Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th February 2017, 22:26   #1  |  Link
Floatingshed
Registered User
 
Join Date: Nov 2008
Posts: 324
joining many clips

I need to join almost 500 short video clips in a random order.
Some are avi, some are mp4. My first thought was to convert them all to the same format and use a file joining tool, this works but a/v sync is a bit variable and there is some data corruption in the video.

Ideally I would load them as individual clips in an avs script but that would take a huge amount of time, and the concatenate line (vid1++vid2++vid3++vid4.....++vid488) would be huge and probably illegal.

Clearly I am approaching this from entirely the wrong direction.
Can anyone suggest anything please?
Floatingshed is offline   Reply With Quote
Old 26th February 2017, 23:10   #2  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,079
You could try DMMediaConverter:
http://dmsimpleapps.blogspot.de/2014...converter.html
which has a "Merge Different" feature.

Quote:
merge different kind of files (different codecs, resolution, etc) into one file. It chooses an output with the biggest width of all source files.
Worked for me a couple of times, but of course with less than 10 files to merge.


Good luck
manolito
manolito is offline   Reply With Quote
Old 26th February 2017, 23:38   #3  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Maybe ffmpeg can help. It seems you can do some crazy stuff with it.

https://trac.ffmpeg.org/wiki/Concatenate


my first thought was: use vapoursynth ... but there is no audio support build in :/
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 27th February 2017, 00:14   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Are they all same DAR, FrameRate, Audio Channels ? (Is anything constant ?)

EDIT: Avisynth [EDIT: 32bit] is limited to about 24 [EDIT: physical] clips open at any one time.

EDIT: A starting point might be to get all files in AVI
Code:
setlocal

REM Where to Find ffmpeg
set FFMPEG="C:\BIN\ffmpeg.exe"

REM Where to get input file, No terminating Backslash, "." = current directory (ie same as dir .bat file)
set INDIR="."


REM Where to place output file, No terminating Backslash. "." would be same as .bat file
set OUTDIR="D:"

FOR %%A IN (*.h264 *.vob *.wmv *.asf *.mpg *.m2v *.avi *.flv *.mov *.mp4 *.m4v *.RAM *.RM *.mkv *.TS *.y4m *.yuv *.webm) DO (
  %FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"

)

Pause
Above does not like SPACES in file names (use some bulk renamer to change spaces to eg '_' first).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 27th February 2017 at 02:20.
StainlessS is offline   Reply With Quote
Old 27th February 2017, 00:29   #5  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,420
Quote:
Originally Posted by Floatingshed View Post
I need to join almost 500 short video clips in a random order.
Some are avi, some are mp4. My first thought was to convert them all to the same format and use a file joining tool, this works but a/v sync is a bit variable and there is some data corruption in the video.

Ideally I would load them as individual clips in an avs script but that would take a huge amount of time, and the concatenate line (vid1++vid2++vid3++vid4.....++vid488) would be huge and probably illegal.

Clearly I am approaching this from entirely the wrong direction.
Can anyone suggest anything please?
There shouldn't be any problems with # of clips to import and concatenate (so long as you have enough memory; 64-bit makes that easier), but there would be tricks to speed up the process of writing a script like that or of making the script legible.

Open bash and run the following in the directory with the video files:
Code:
for n in *.avi *.mp4 ; do
echo FFmpegSource2\(\"$n\",atrack=-1\)>"${n%.*}.avs"
done
You could also use that for block to make sure the scripts it produces force their outputs to a specific framerate, resolution, etc.

Mass-rename all the created *.avs files to sequential numbers (001.avs, 002.avs, etc.). Or do that first with the AVI or MP4 files, if there's a possibility of the *.avs filenames colliding because the original filenames overlap.

In the new script which concatenates the clips together:
Code:
Import("001.avs") ++ \
Import("002.avs") ++ \
Import("003.avs") ++ \
etc.
You can even use a spreadsheet like Calc to duplicate that all the way down, possibly even incrementing the filename numbers at the same time.

At that point, the biggest part would be re-arranging the clip order in the concat script (if you didn't bake the random order into the sequential filenames when you did that), but since each Import is on a new line, it's basically just shuffling lines around (or going down line-by-line-by-line in a text editor and changing the *.avs filename being imported).

Last edited by qyot27; 27th February 2017 at 00:34.
qyot27 is offline   Reply With Quote
Old 27th February 2017, 00:34   #6  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,496
Quote:
Originally Posted by StainlessS View Post
EDIT: Avisynth is limited to about 24 clips open at any one time.
I thought around 70 was the limit for AVIs. And I've definitely had more than 24 mpegs2sourced clips open.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 27th February 2017, 00:41   #7  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
My experience is less than 30 for 32 bit Avisynth (it kinda varies a bit).
(above was prior to standard v2.6 FINAL, not tried since)

There is a demo to splice 1000 clips (all same size etc) in TWriteAVI v2.0 AVS folder. [Includes 1000 clip generator]

EDIT: See here:- https://forum.doom9.org/showthread.p...ight=TWriteAVI
and here:- https://forum.doom9.org/showthread.p...od#post1682093

EDIT:
David, tried your Mpeg2Source thing out with exact same d2v clip every time.
Code:
FN="Test.d2v"

SSS="""CLIPNAME="Test_Clip_%06d" """

GSCript("""
    c=0
    for(i=1,1000) {
        S=RT_String(SSS,i)
        Eval(S)
        Eval(RT_String("%s=MPeg2Source(FN)",CLIPNAME))
        Eval(RT_String("c = c.IsClip ? c + %s : %s",CLIPNAME,CLIPNAME))
        RT_DebugF("Open OK on clip %s FrameCount=%d",CLIPNAME,c.FrameCount,name="TESTING: ")  
    }
    return c
""")
Produces this, and then Access violation after 96 clips.
Code:
00000006    1.67969370  [3296] TESTING: Open OK on clip Test_Clip_000001 FrameCount=616 
00000007    1.68210435  [3296] TESTING: Open OK on clip Test_Clip_000002 FrameCount=1232    
00000008    1.68450701  [3296] TESTING: Open OK on clip Test_Clip_000003 FrameCount=1848    
00000009    1.68690670  [3296] TESTING: Open OK on clip Test_Clip_000004 FrameCount=2464    
00000010    1.68931365  [3296] TESTING: Open OK on clip Test_Clip_000005 FrameCount=3080    

...

00000090    1.89479208  [3296] TESTING: Open OK on clip Test_Clip_000085 FrameCount=52360   
00000091    1.89751649  [3296] TESTING: Open OK on clip Test_Clip_000086 FrameCount=52976   
00000092    1.90024400  [3296] TESTING: Open OK on clip Test_Clip_000087 FrameCount=53592   
00000093    1.90296245  [3296] TESTING: Open OK on clip Test_Clip_000088 FrameCount=54208   
00000094    1.90568399  [3296] TESTING: Open OK on clip Test_Clip_000089 FrameCount=54824   
00000095    1.90841627  [3296] TESTING: Open OK on clip Test_Clip_000090 FrameCount=55440   
00000096    1.91114259  [3296] TESTING: Open OK on clip Test_Clip_000091 FrameCount=56056   
00000097    1.91389704  [3296] TESTING: Open OK on clip Test_Clip_000092 FrameCount=56672   
00000098    1.91664517  [3296] TESTING: Open OK on clip Test_Clip_000093 FrameCount=57288   
00000099    1.91938639  [3296] TESTING: Open OK on clip Test_Clip_000094 FrameCount=57904   
00000100    1.92212272  [3296] TESTING: Open OK on clip Test_Clip_000095 FrameCount=58520   
00000101    1.92487681  [3296] TESTING: Open OK on clip Test_Clip_000096 FrameCount=59136
2nd time of posting above, screwed up first test script.
So your post of 70+ Mpeg2Source clips proves true, at least as far as opening exact same clip. (I dont have 70+ mpegs available).

EDIT:
Did same again as above but with manufactured numbered mpegs with generated d2v's + avs for each, as so
Code:
VideoFileName   ="D:\OUT\Test_000001.d2v"
AudioFileName   ="D:\OUT\Test_000001 Tc0 L1 mono 32 96 DELAY -363ms.mp1"
###
MPEG2Source(VideoFileName)
Audio=NicMPG123Source(AudioFileName,Normalize=False)
return AudioDub(Audio).Trim(0,0)
Access violation after Import of "Test_000095.avs", avs inclusive of audio.

I think the ~24 to 30 was for AviSource + DirectShowSource.

Mpeg Generator
Code:
# Make 150 mpegs in D:\OUT directory (req RT_Stats v2.0 beta for RT_FileDuplicate).

FN1="Test.mpg"
FN2="""OUTNAME="D:\OUT\Test_%06d.mpg" """

DUP="""RT_FileDuplicate(FN1,"%s")"""

GSCript("""
    for(i=1,150) {
        S=RT_String(FN2,i)
        Eval(S)
        Eval(RT_String(DUP,OUTNAME))
    }
""")
Return MessageClip("DONE")
Code:
# Generate avs files after using DGIndex Batcher to create d2v's [I should have created avs via template in DGIndex, forgot]
SSS="""
    CLIPNAME="Test_Clip_%06d"
    VFN="D:\OUT\Test_%06d.d2v"
    AFN="D:\OUT\Test_%06d Tc0 L1 mono 32 96 DELAY -363ms.mp1"
    AVS="D:\OUT\Test_%06d.avs"    
"""

T="""
### DGIndex Template
VideoFileName   ="__vid__"
AudioFileName   ="__aud__"
###
MPEG2Source(VideoFileName)
Audio=NicMPG123Source(AudioFileName,Normalize=False)
return AudioDub(Audio).Trim(0,0) 
"""

Fnd=RT_String("__vid__\n__aud__\n")

GSCript("""
    for(i=1,150) {
        S=RT_String(SSS,i,i,i,i)
        Eval(S)
        RT_DebugF("S=%s",S)
        Rep=RT_String("%s\n%s\n",VFN,AFN)
        T2=RT_StrReplaceMulti(T,Fnd,Rep)
        RT_DebugF("T2=%s",T2)        
        RT_DebugF("AVS=%s",AVS)        
        RT_WriteFile(AVS,"%s",T2,Append=False)
    }
""")
MessageClip("DONE")

Code:
# avs reader
SSS="""
    CLIPNAME="Test_Clip_%06d"
    FN=Chr(34)+"D:\OUT\Test_%06d.avs"+Chr(34)
"""

GSCript("""
    c=0
    for(i=1,150) {
        S=RT_String(SSS,i,i)
        Eval(S)
#        RT_DebugF("S=%s",S)
        S2=RT_String("%s=Import(%s)",CLIPNAME,FN)
#        RT_DebugF("S2=%s",S2)
        Eval(S2)
        S3=RT_String("c = c.IsClip ? c ++ %s : %s",CLIPNAME,CLIPNAME)
#        RT_DebugF("S3=%s",S3)
        Eval(S3)
        RT_DebugF("Open OK on clip %s FrameCount=%d",CLIPNAME,c.FrameCount,name="TESTING: ")  
    }
    return c
""")
EDIT: Mpegs were 480x272@30FPS 20 secs, no idea what part frame size plays in file open limit.

EDIT: Re-did with 150 22 sec clips extracted from PAL DVD 720x576@25FPS + AC3 Audio.
Failed after 84 clips read with audio via AVS Import.
EDIT: Without AC3 audio, failed after 1 additional clip, after clip 85.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 27th February 2017 at 09:43.
StainlessS is offline   Reply With Quote
Old 27th February 2017, 11:03   #8  |  Link
mdalacu
Registered User
 
mdalacu's Avatar
 
Join Date: May 2014
Location: Bucharest, Romania
Posts: 109
Quote:
Originally Posted by manolito View Post
You could try DMMediaConverter:
http://dmsimpleapps.blogspot.de/2014...converter.html
which has a "Merge Different" feature.



Worked for me a couple of times, but of course with less than 10 files to merge.


Good luck
manolito
I have tested it with 18 files and was working just fine...v2.0.1
mdalacu is offline   Reply With Quote
Old 27th February 2017, 18:43   #9  |  Link
rjd0309
Wanted Dead or Dead
 
rjd0309's Avatar
 
Join Date: Apr 2008
Location: Phoenix, Arizona
Posts: 40
Thanks for this thread. Along these same lines, I've always wanted to create a music video by assembling dozens of video-only clips to accompany a piece of music. I've been stumped by the limitation on the number of open files in Avisynth.

Ideally, I'd like to create a list of source files and start/end frame numbers.

Code:
path to D2V of title 1, first frame of clip, last frame of clip
path to D2V of title 2, first frame of clip, last frame of clip
path to D2V of title 3, first frame of clip, last frame of clip
path to D2V of title 4, first frame of clip, last frame of clip
A contrived example:

Code:
"C:/My Hard Drive/I Love Lucy/episode 27/dgindex.d2v", 500, 642  # Lucy stomping grapes
"C:/My Hard Drive/I Love Lucy/episode 13/dgindex.d2v", 123, 456  # Lucy & Viv in shower
"C:/My Hard Drive/I Love Lucy/episode 41/dgindex.d2v", 654, 715  # conveyor belt chocolates
I'm not sufficiently well-versed in Avisynth to be able to make this work, though.
rjd0309 is offline   Reply With Quote
Old 27th February 2017, 21:08   #10  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,079
Quote:
Originally Posted by mdalacu View Post
I have tested it with 18 files and was working just fine...v2.0.1
A little OT but a good opportunity for nagging you...

I have been using v1.9.0 Beta which you gave me last November without any issues. Any time frame for releasing a new stable version?

Cheers
manolito
manolito is offline   Reply With Quote
Old 28th February 2017, 00:32   #11  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Something like Premiere, can open files as it needs them and close files that have been processed, Avisynth basically
requires knowledge of output frame count and so all files need be open at once.
Might be possible to pre-scan source files, calc output framecount, create BlankClip of required size and swap each frame of blankclip
with required file, opening where necessary and closing when no longer in use. (could do something like that in Scriptclip I think,
might require entire process to be carried out via GScript runtime rather than FrameServer mode, output via TWriteAVI v2.0).

Script to join clips or images together, of different sizes, probably a bit limited for video if different framerates:- https://forum.doom9.org/showthread.php?t=165694&page=4
Limited to about 24 clips at once (but more if images, ie slideshow)
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th February 2017 at 00:36.
StainlessS is offline   Reply With Quote
Old 28th February 2017, 01:09   #12  |  Link
ndjamena
Registered User
 
Join Date: Sep 2012
Posts: 366
Would MP_Pipeline be useful here? Presumably it starts extra instances of AVISynth and can therefore override the Source filter count limitations...
ndjamena is offline   Reply With Quote
Old 28th February 2017, 01:32   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
I'm under the impression that the limitation is a machine wide resource one, not Avisynth

There is a demo in TWriteAVI v2.0 AVS folder, that creates RT_Stats DBase for each file opened, (daft demo), each file has attributes for eg
Favorite color, time opened, and during FrameServiing, can show the favorite color of the clip in question, the source clip frame number
related to the overall output clip, plus original clip name, + other stuff.
I guess it would not be impossible with a core ScriptClip instance to check on file open status, and open if necessary, and close when no longer in-use
(probably a bit jittery at clip joins where temporal processing), dont know if I am up to the task, but could be a possibility me thinks).
[Above would require a bunch of DBases to be created before FrameServing starts, implemented pretty much already in the demo].

EDIT: Above sounds slow, but is not terribly slow at all, core ScriptClip only accesses clip DBase when the current clip changes,
main DBase is interrogated to ascertain current clip number only [at each frame], via binary search, very fast
(assuming you dont got gazillions of source clips).
EDIT: Demo plays in better than real time on my crap 2.4GHz core duo.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 28th February 2017 at 01:48.
StainlessS is offline   Reply With Quote
Old 28th February 2017, 02:19   #14  |  Link
ndjamena
Registered User
 
Join Date: Sep 2012
Posts: 366
I got 80 AVISource filters open using MP_Pipeline, where it doesn't work if used in a normal AVS.
ndjamena is offline   Reply With Quote
Old 28th February 2017, 02:22   #15  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Hmm, good to know, perhaps someone knows exactly what the resource limitation is.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 28th February 2017, 03:17   #16  |  Link
mariush
Registered User
 
Join Date: Dec 2008
Posts: 589
Convert all to AVI with a lossless video codec (MagicYUV, Huffyuv, Lagarith) and raw audio (pcm stereo, 16 bit, 48khz or 44.1 khz). Match the framerate and resolution of all.

Name your segments in the order you want them ... file001.avi , file002.avi etc ... Open Virtualdub, hit Open, select the first file and check the box that says Append segments (or something like that). Virtualdub should open file001.avi and append file002.avi and so on and work with hundreds of files as it will open the files as needed.

Compress to your liking.

Virtualdub also has batch processing ... you can specify files you want processed, add filters to change refresh rate or resolution (add black bars if needed to preserve aspect ratio etc), it will load mp4 files with the proper input plugin (available on the 'net)
mariush is offline   Reply With Quote
Old 28th February 2017, 14:59   #17  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by rjd0309 View Post
I'm not sufficiently well-versed in Avisynth to be able to make this work, though.
There is a new Function in RT_Stats v2.0 beta 5
Code:
RT_DBaseReadCSV(String DB, String CSV, String "Separator"=",")

    DB,         FileName of an RT_Stats DBase file.
    CSV,        FileName of a text CSV (Comma Separated Value) file.
    Separator   Default "," ie comma separator (first character only, 0, 13 and 10 illegal).

    This function will extract CSV values in text file, and append them as records to an appropriately formated DBase file.
    Returns Int, the number of records added to the DBase.
    (Maximum line length in CSV text file is 16*1024 characters).

    Example:-
        CSV.txt file containing text something like this
        ###
            1  ,42.0 ,true , "Hello"        # I am a comment
            2  ,43.5 ,false, "Goodbye"
            $FF,3.142,TRUE , "Coconuts"     # So Am I
        ###
        Script:-
        DB  = "My.DB"
        CSV = "CSV.txt"
        RT_DBaseAlloc(DB,0,"ifbs32")        # Allocate Dbase with 0 pre-allocated records, 4 fields, 0)=Int, 1)=Float, 2)=bool, 3=String(maxlen=32).
        Added=RT_DBaseReadCSV(DB,CSV)
        Return MessageClip("Added "+String(Added)+" Records")   # Shows "Added 3 Records".
Using above func, we have here test file CSV.txt
Code:
"D:\OUT\Test_000001.d2v", 10,   19    # Something somthing something
"D:\OUT\Test_000002.d2v", 20,   29    # Something somthing something
"D:\OUT\Test_000003.d2v", 30,   39    # Something somthing something
"D:\OUT\Test_000004.d2v", 40,   49    # Something somthing something
"D:\OUT\Test_000005.d2v", 50,   59    # Something somthing something
"D:\OUT\Test_000006.d2v", 60,   69    # Something somthing something
"D:\OUT\Test_000007.d2v", 70,   79    # Something somthing something
"D:\OUT\Test_000008.d2v", 80,   89    # Something somthing something
"D:\OUT\Test_000009.d2v", 90,   99    # Something somthing something
"D:\OUT\Test_000010.d2v", 100, 109    # Something somthing something
We now create DBase
Code:
#rjd0309_CreateDB.avs

DB  = "My.DB"
CSV = "CSV.txt"
MAXSTRLEN=512       # Max Filename len

TYPESTRING = RT_String("s%dii",MAXSTRLEN)    # As Default create DB typestr as 3 Field per record, "s512ii" ie String[512], Int, Int. 

RT_DBaseAlloc(DB,0,TYPESTRING)   # Allocate DBase with 0 pre-allocated records, 3 fields, 0)=String(maxlen=512), 1)=Int, 2)=Int. 
Added=RT_DBaseReadCSV(DB,CSV)
Return MessageClip("Added "+String(Added)+" Records")   # Shows "Added 10 Records".
and create spliced clip
Code:
# rjd0309_SpliceDBClips.avs

DB  = "My.DB"

ClipCount=RT_DBaseRecords(DB)

c=0         # Dummy, will become clip

GSCript("""
    for(i=0,ClipCount-1) {
        ClipName   = RT_DBaseGetField(DB,i,0)
        StartFrame = RT_DBaseGetField(DB,i,1)
        EndFrame   = RT_DBaseGetField(DB,i,2)
        newC       = MPeg2Source(ClipName)
        newC       = newC.Trim(StartFrame,EndFrame)
        newC       = newC.Subtitle(ClipName)
        c = (c.IsClip) ? c ++ newC : newC           # If c is a clip, then add splice newc else replace with newc
    }
""")
c
Sorted.

EDIT: Actually I think I corrected a bug in beta 5, IIRC it did not cope with -ve int nor -ve float, Ill update to v2 beta 6 in a few moments.
I also changed above RT_DBaseReadCSV a little bit (above script works fine as it is in beta 5).

EDIT: RT_Stats updated to v2.0 beta 6.

EDIT: I think I corrected the above mentioned bug before release of 2.0 beta 5.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 2nd March 2017 at 02:40.
StainlessS is offline   Reply With Quote
Old 1st March 2017, 07:27   #18  |  Link
Floatingshed
Registered User
 
Join Date: Nov 2008
Posts: 324
Plenty to chew-over there!
Thanks very much everyone.
Floatingshed is offline   Reply With Quote
Old 1st March 2017, 16:40   #19  |  Link
rjd0309
Wanted Dead or Dead
 
rjd0309's Avatar
 
Join Date: Apr 2008
Location: Phoenix, Arizona
Posts: 40
Many thanks for your DBase and Splicing scripts, StainlessS!
rjd0309 is offline   Reply With Quote
Old 1st March 2017, 21:17   #20  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Quote:
Originally Posted by rjd0309 View Post
Many thanks for your DBase and Splicing scripts, StainlessS!
I'm actually doing a bit of work on that right now, guess it might be another monster thread as functionality is improving (the DB utility functions in AVS\LIBS dir in TWriteAVI zip), added ability to use your example CSV type thing, and to trim where necessary + other stuff.
Will probably require an awful amount of time to document it, I'de rather do some examples so people can figure it out themselves, I hate docs.

EDIT: I think, why cant these people work it out for themselves, my time is better spent doing something constructive
and not explaining how something works, of course that is totally wrong, but I still hate doing that kinda stuff.

I'm guessin' most devs feel EXACTLY the same way.

EDIT: Update below script replacing previous ones for rjd0309 query.

CSV.TXT
Code:
"D:\OUT\Test_000001.d2v", 10,   19    # Something somthing something
"D:\OUT\Test_000002.d2v", 20,   29    # Something somthing something
"D:\OUT\Test_000003.d2v", 30,   39    # Something somthing something
"D:\OUT\Test_000004.d2v", 40,   49    # Something somthing something
"D:\OUT\Test_000005.d2v", 50,   59    # Something somthing something
"D:\OUT\Test_000006.d2v", 60,   69    # Something somthing something
"D:\OUT\Test_000007.d2v", 70,   79    # Something somthing something
"D:\OUT\Test_000008.d2v", 80,   89    # Something somthing something
"D:\OUT\Test_000009.d2v", 90,   99    # Something somthing something
"D:\OUT\Test_000010.d2v", 100, 109    # Something somthing something
rjd0309_CreateDB_And_Clip_and_Demo.avs
Code:
#rjd0309_CreateDB_And_Clip_and_Demo.avs

# Added two more Int fields to DBase for DemoScan().
# Now Requires RT_Stats v2.0 Beta 7 update [added args StartField and EndField to RT_DBaseReadCSV()]
# Also req TWriteAVI v2.0 to write AVI file Audio + Video. 

DB        = "My.DB"
CSV       = "CSV.txt"
OUTAVI    = "OUTAVI.AVI"
MAXSTRLEN = 512       # Max Filename len
FileNameField   = 0
TrimStartField  = 1
TrimEndField    = 2
# Added fields to locate sequence info in DBase.
SeqStartField   = 3 
SeqEndField     = 4

TYPESTRING = RT_String("s%diiii",MAXSTRLEN)    # As Default create DB typestr as 5 Field per record, "s512iiii" ie String[512], Int,Int,Int,Int. 

RT_DBaseAlloc(DB,0,TYPESTRING)  # Allocate DBase with 0 pre-allocated records, 5 fields, 0)=String[maxlen=512], 1 To 4)=Int 

# Fill in Fields 0 -> 2, Fields 3 and 4 currently left in null state (ie int = 0), we will fill them in later.
Added=RT_DBaseReadCSV(DB,CSV,StartField=0,EndField=2)   # Default StartField=0, EndField=4(in this case, ie last field)

### NOW, we will update DBase fields 3 and 4 to hold start and end frame within outputclip, of the input clips.
### We will also create clip and call DemoScan() on result.

# Get clip of sequences Added to DBase, and set fields SeqStartField and SeqEndField.                
Get_DB_Sequences(DB,FileNameField,TrimStartField,TrimEndField,SeqStartField,SeqEndField)


# TWrite OUTAVI Lossless file Audio PCM, Video UT_Video YV12 rec601
TWriteAvi(OUTAVI,overwrite=True,FourCC="ULY0")      # Play result clip, and simultaneously write OUTAVI using UT_Video Codec FourCC"ULY0".
ForceProcessAVI()                                   # Forcibly write OUTAVI, instead of playing clip 


#################################
# Play clip Demo (Uncomment when DB and AVI exist and can use below on its own)
# Fields as gotten from CSV file. 
# NOTE, not same order as used in DemoScan, DemoScan functionality relies upon SeqStartField, and SeqEndField, so those
# are before optional TrimStartField, and TrimEndField.
#FileNameField   = 0
#TrimStartField  = 1
#TrimEndField    = 2
# Added fields to locate sequence info in DBase.
#SeqStartField   = 3 
#SeqEndField     = 4
#################################
#DB        = "My.DB"
#OUTAVI    = "OUTAVI.AVI"
#Avisource(OUTAVI)
# Play clip Demo
DemoScan(DB,FileNameField,SeqStartField,SeqEndField,TrimStartField,TrimEndField)

Return Last

Function Get_DB_Sequences(String DB,int FileNameField,int TrimStartField,int TrimEndField,int SeqStartField,int SeqEndField) {
    # Get Sequences Added to DBase, and set fields SeqStartField, and SeqEndField
    ClipCount=RT_DBaseRecords(DB)
    c=0         # Dummy, will become clip    
    GSCript("""
        SeqStart = 0                                       
        for(i=0,ClipCount-1) {
            ClipName       = RT_DBaseGetField(DB,i,FileNameField)  # As Set via RT_DBaseReadCSV() 
            TrimStartFrame = RT_DBaseGetField(DB,i,TrimStartField) # Ditto
            TrimEndFrame   = RT_DBaseGetField(DB,i,TrimEndField)   # Ditto
                        
            newC       = MPeg2Source(ClipName)
            newC       = newC.Trim(TrimStartFrame,TrimEndFrame)
            newC       = newC.Subtitle(ClipName,Align=5)
                                                                                                            
            Len        = newC.FrameCount                      # Len of current sequence.
            RT_DBaseSetField(DB,i,SeqStartField,SeqStart)     # Start frame of current record/clip (input clip) within entire output clip.
            RT_DBaseSetField(DB,i,SeqEndField,SeqStart+Len-1) # End frame of current record/clip within entire output clip.
            SeqStart = SeqStart + Len                         # Next clip Start frame within entire output clip.        
    
            c = (c.IsClip) ? c ++ newC : newC                 # If c is a clip, then add splice newc else replace with newc
        }
    """)
    Return c
}


Function DemoScan(clip c,String DB,Int FileNameField,SeqStartField,SeqEndField,Int "TrimStartField",Int "TrimEndField") {
    myName="DemoScan: "
    c
    Assert(0 < SeqStartField,RT_String("%s 0 < SeqStartField(%d)",myName,SeqStartField))
    Assert(0 < SeqEndField,  RT_String("%s 0 < SeqEndField(%d)",myName,SeqEndField))
    Assert(DB!="" && Exist(DB),RT_String("%s DB is empty string or does not exist",myName))
    TrimStartField=Default(TrimStartField,-1)   TrimEndField=Default(TrimEndField,-1)
    DB=RT_GetFullPathName(DB)        
    ShowTrim=(TrimStartField>=0 && TrimEndField>=0)
    # Shows records as zero relative
    Fmt="%d/%d]\nSeq:-%d/%d {%d:%d}[%d/%d]\n%s" + ((ShowTrim) ? "\a!.Trim(%d,%d)" : "") 
    FMX=FrameCount-1
    FuncS="""
        Function Fn@@@(clip clp,String DB,Int FileNameField,Int SeqStartField,Int SeqEndField,
                    \ Int TrimStartField,Int TrimEndField,String Fmt,bool Showtrim,Int FMX) {
            clp     n=current_frame
            if(PREV_FRAME@@@ != n) {
                if(PREV_FRAME@@@ + 1 != n || n > REC_E@@@) {
                    # User jumped, OR new sequence, need interrogate DBase (get out the nipple clamps boys).
                    Rec                       = DB_FindSeq(DB,SeqStartField,SeqEndField,n)
                    Global REC_S@@@           = RT_DBaseGetField(DB,rec,SeqStartField)
                    Global REC_E@@@           = RT_DBaseGetField(DB,rec,SeqEndField)
                    Global RECORD@@@          = Rec
                    Global REC_NAME@@@        = RT_DBaseGetField(DB,Rec,FileNameField)
                    Global TRIM_START@@@      = (ShowTrim) ? RT_DBaseGetField(DB,Rec,TrimStartField) : 0
                    Global TRIM_END@@@        = (ShowTrim) ? RT_DBaseGetField(DB,Rec,TrimEndField)   : 0
                }            
            }
            seqf=n-REC_S@@@
            seqmx=REC_E@@@-REC_S@@@
            (Showtrim)
                \ ? RT_Subtitle(Fmt,n,FMX,RECORD@@@,RECORDS@@@-1,REC_S@@@,REC_E@@@,seqf,seqmx,REC_NAME@@@,TRIM_START@@@,TRIM_END@@@)
                \ : RT_Subtitle(Fmt,n,FMX,RECORD@@@,RECORDS@@@-1,REC_S@@@,REC_E@@@,seqf,seqmx,REC_NAME@@@)  
            Return Last
        }
        #######################################
        Global  PREV_FRAME@@@   = -666                # Unique Global Variables Initialization, Force Init
        Global RECORDS@@@       = RT_DBaseRecords(DB) # Constant
        #######################################
        ARGS = "DB,FileNameField,SeqStartField,SeqEndField,TrimStartField,TrimEndField,Fmt,Showtrim,FMX"
        Last.GScriptClip("Fn@@@("+ARGS+")", local=true, args=ARGS)
    """
    GIFunc ="DemoScan"                 # Function Name, Supply unique name for your multi-instance function.
    GIName =GIFunc+"_InstanceNumber"   # Name of the Instance number Global
    RT_IncrGlobal(GIName)              # Increment Instance Global (init to 1 if not already exists)
    GID    = GIFunc + "_" + String(Eval(GIName))
    InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
#   RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
    Return GScript(InstS)
}

/*
############
 DB_FindSeq(String DB,Int S_Field,Int E_Field,Int Frame,Int "Low"=0,Int "High"=RT_DBaseRecords(DB)-1)
 
 DB_FindSeq Returns clip (or record) number containing movie frame Frame, using Binary Search.
   
   DB,      DBase FileName,
   S_Field, DB field that contains the first frame of the record clip/sequence (within entire movie).
   E_Field, DB field that contains the Last frame of the record clip/sequence (within entire movie).
   Low,     Default 0. Lowest record number to scan (likely almost always 0).
   High,    Default RT_DBaseRecords(DB)-1. Hightest record number to scan (likely almost always as default).
   
   Return of -1 = RECORD NOT FOUND.
   
   eg If you have 4 clips in DBase, each of 10 frames then fields would contain data EXACTLY as below (must be ordered).
      Record     S_Field   E_Field
         0           0        9
         1          10       19
         2          20       29
         3          30       39
############
*/

Function DB_FindSeq(String DB,Int S_Field,Int E_Field,Int Frame,Int "Low",Int "High") {
    myName="DB_FindSeq: "
    Records = RT_DBaseRecords(DB)
    low     = Default(Low,0)
    high    = Default(High,Records - 1)
    Assert(0 <= Low <= High < Records,RT_String("%s0 <= Low(%d) <= High(%d) < Records(%d)",myName,Low,High,Records))
    result = -1                                                     # Init NOT FOUND
    GSCript("""
        while(low <= high) {
            mid = (low + high) / 2
            if(RT_DBaseGetField(DB,mid,E_Field) < Frame) {
                low = mid + 1
            } Else If (RT_DBaseGetField(DB,mid,S_Field) > Frame) {
                high = mid - 1
            } Else {
                low = high + 1                                      # Force exit
                Result = mid
            }
        }
    """)
    return result
}
Everything in one scipt.

EDIT: You can comment out the TWriteAVI and ForceProcessAVI lines if you want, still works, just does not auto write output clip.
EDIT: Also, After DB and AVI creation, the stuff after TWriteAVI should work on its own, so long as it also has the filenames set too, + an
Avisource(OUTAVI) before DemoScan.

EDIT: Changed DB_FindSeq() slightly, no change to functionality. I'll probably add similar to RT_Stats.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 3rd March 2017 at 15:48. Reason: Update
StainlessS is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.