How to open a (EDL)text file and parse it?

[tin3tin]

I been thinking about how to open a text file and how to parse it afterwards - things which would be useful for working with .edl or unsupported subtitle formats directly in avisynth script.

Some years ago avisynth strings seemed to be able to hold a limited number of characters, now it seems to be higher(anyone know how many?). Anyway this opens a possibility to work with huge scripts as strings which can be executed with the Eval command. So that part seems to be possible.

However I can't find a way to load/read/import a text file into a string. And then run through the string - until the end of the string - parsing it on a line by line basis.

Ex. script(not working):

My_edl_string=ImportString("My_edl_file.edl")

My_edl_script=""#global variable the final avisynth script

My_edl_script=ParseString(My_Parser, My_edl_string)# Run through string and apply the named function to a line at a time until the end of the string.

function My_Parser(string CurrentLine)
{
findstr(CurrentLine, "avi")=0 ? :My_edl_script=My_edl_script&"Directshowsource("&chr(34)&CurrentLine&chr(34)&")"#example adding of parser script to My_edl_script
}

Eval(My_edl_string)# And finally convert the the generated script into a clip.

Well as you can see I'm missing idears for ways to import textfiles into a string and also parse the string one line at the time until the end of the string.

I guess if I somehow could count the number of line breaks in the string I could make a blankclip with a length of that number and use animate to call the parser function for each line, but how would I get it to count the number of line breaks and also how would I get the parser function to find and parse the line with the animated frame corresponding number?

[Gavino]

The script language provides no primitives for reading files, but it would be trivial to implement ImportString as a plugin function in C++ - just read the file contents into a buffer and return it as a string.

Tasks like parsing, unless the format is very simple, are probably best handled by a plugin too.
If you really want to do it in the script language, the outer part (looping over the lines) is relatively simple using a recursive function.
Pseudocode:

function ParseLines(string text) {
  if (text contains NL) #use findStr( text, chr(10))
    Parse(leftStr(...)) + ParseLines(rightStr(...))
  else
    Parse(text)
}

Personally (YMMV), I would go for combining the whole thing into a single plugin which both reads the file and parses its contents.

[tin3tin]

I'm still close to nowhere when it comes to C++, so pluginwriting is out of my limits.

Actually a part of what I'm after has allready been worked on here: An EDL2AVS converter - it was never finished. It imports all source clips with DirectShowSource. I guess most people would like to be able to change how source clips are imported. Ex. if the source clip is a .avs you would want to use 'Import' instead.

What I'm dreaming of is a way to simplify the link between a NLE GUI and avisynth script and some sort of EDL parser would make that process much easier. So I'm thinking why not use the EDL format most NLEs are using for that purpose? Info on the EDL format here.

The reason for keeping the string importer and the string parser separate is that you then could code a string inside your script you would want to parse.

And the reason for having the parser separate from the 'Eval' is of course that you could edit it or writetofile and such.

Here is an edl importer for Blender written in python.

[tin3tin]

Well, I've been trying to figure the C++ stuff out, but it takes time to figure the most simple things out when you're trying to figure a new coding language out. And this compiling stuff is no fun for a noob like me...

Anyway, I've come across this C++ EDL to AAF project - which might be giant step forward for someone with some C++ knowledge in making a EDLsource plugin for avisynth. It seems to come down to change the aaf stuff to avisynth commands and then output it as a video clip(There are EDL examples included in the EDLaaf download).

Link here.

[Gavino]

If you're not comfortable using C++, why not try a different approach? Use a language you are familiar with to write a standalone EDL to AVS converter whose output is an Avisynth script.

It may be less user-friendly than doing the conversion directly inside Avisynth, but doing it this way will let you concentrate on the details of the task rather than fighting with an unfamiliar language. At some later point, when you are more proficient in C++, you could then rewrite the program in that language using the experience gained the first time around, and convert it to a plugin.

[tin3tin]

My DVD slideshow GUI is basically an Avisynth script generator for making slideshows(GUI in wxBasic). However the resulting scripts have become insanely complicated and therefore not really the way to progress if you would want to make an NLE like gui for Avisynth.

I should get started learning some C++ and I'm working on it. And so far I've figured out how to load the edl file into the buffer n C++.

The two links above is sourcecode for edl to blender written in python (like AvsP) and the sourcecode for edl to aaf in C++.

[tin3tin]

With StainlessS' new RT_Stats plugin it is now possible to read and pass textfiles, so I had to try to write up a proof of concept import of EDL script. It only reads video clips with cuts for now.
I think working with arrays are needed to just open videofiles once and to pass the various functions. Let me know if you have some ideas on how to pass the lines in a clever way.

# Load a CMX formatted EDL into Avisynth
# As a proof of concept here's a script reading the video clips with cuts and pass them to avs script which finally is eval'ed and a clip is returned.
# Gscript by Gavino and RT_Stats by StainlessS is needed.

#~ THE EDL FORMAT
    #~ Comments can appear at the beginning of the EDL file (header) or between the edit lines in the EDL. The first block of comments in the file is defined to be the header comments and they are associated with the EDL as a whole. Subsequent comments in the EDL file are associated with the first edit line that appears after them.
    #~ Edit Entries
    #~ <filename|tag>  <EditMode>  <TransitionType>[num]  [duration]  [srcIn]  [srcOut]  [recIn]  [recOut]
    #~ * <filename|tag>: Filename or tag value. Filename can be for an MPEG file, Image file, or Image file template. Image file templates use the same pattern matching as for command line glob, and can be used to specify images to encode into MPEG. i.e. /usr/data/images/image*.jpg
    #~ * <EditMode>: 'V' | 'A' | 'VA' | 'B' | 'v' | 'a' | 'va' | 'b' which equals Video, Audio, Video_Audio edits (note B or b can be used in place of VA or va).
    #~ * <TransitonType>: 'C' | 'D' | 'E' | 'FI' | 'FO' | 'W' | 'c' | 'd' | 'e' | 'fi' | 'fo' | 'w'. which equals Cut, Dissolve, Effect, FadeIn, FadeOut, Wipe.
    #~ * [num]: if TransitionType = Wipe, then a wipe number must be given. At the moment only wipe 'W0' and 'W1' are supported.
    #~ * [duration]: if the TransitionType is not equal to Cut, then an effect duration must be given. Duration is in frames.
    #~ * [srcIn]: Src in. If no srcIn is given, then it defaults to the first frame of the video or the first frame in the image pattern. If srcIn isn't specified, then srcOut, recIn, recOut can't be specified.
    #~ * [srcOut]: Src out. If no srcOut is given, then it defaults to the last frame of the video - or last image in the image pattern. if srcOut isn't given, then recIn and recOut can't be specified.
    #~ * [recIn]: Rec in. If no recIn is given, then it is calculated based on its position in the EDL and the length of its input.
    #~ [recOut]: Rec out. If no recOut is given, then it is calculated based on its position in the EDL and the length of its input. first frame of the video. 
    #~ For srcIn, srcOut, recIn, recOut, the values can be specified as either timecode, frame number, seconds, or mps seconds. i.e.
    #~ [tcode | fnum | sec | mps], where:
    #~ * tcode : SMPTE timecode in hh:mm:ss:ff
    #~ * fnum : frame number (the first decodable frame in the video is taken to be frame 0).
    #~ * sec : seconds with 's' suffix (e.g. 5.2s)
    #~ * mps : seconds with 'mps' suffix (e.g. 5.2mps). This corresponds to the 'seconds' value displayed by Windows MediaPlayer.

# Example from Lightworks
    #~ TITLE: Edit #1   FORMAT: CMX3600
    #~ 001  VJ0K006W V    C         00:00:19:05 00:00:41:07 00:00:00:00 00:00:22:02
    #~ 002  E00B00CU AA   C         00:00:19:05 00:00:41:07 00:00:00:00 00:00:22:02
     #~ COMMENT: AUDIO MAPPING   FROM CH1 TO CH2
    #~ 003  VJ0K006W V    C         00:00:00:00 00:00:18:18 00:00:22:02 00:00:40:20
    #~ 003  900      V    D    025  00:00:01:00 00:00:03:06 00:00:40:20 00:00:43:01
    #~ 004  E00B00CU AA   C         00:00:00:00 00:00:19:05 00:00:22:02 00:00:41:07
     #~ COMMENT: AUDIO MAPPING   FROM CH1 TO CH2
    #~ 005  E00B00CU AA   C         00:00:34:12 00:00:36:06 00:00:41:07 00:00:43:01
     #~ COMMENT: AUDIO MAPPING   FROM CH1 TO CH2
    #~ 006     BL    AA/V C         00:00:00:00 00:00:22:02 00:00:43:01 00:01:05:03

loadplugin("gscript.dll")
loadplugin("RT_Stats.dll")

MY_EDL="Edit.edl"                                                                  # The Edl file to be converted
MY_DIRECTORY="C:\MATERIAL\"                                                 # Where the sourcefiles are located
MY_EXTENTION="AVI"                                                               # The extention of all the source files
MY_FPS=25                                                                              # FPS for the project
blankclip(1,20,20)

EDL2AVS( MY_EDL, MY_DIRECTORY, MY_EXTENTION, MY_FPS)

function EDL2AVS(MY_EDL, MY_DIRECTORY, MY_EXTENTION, MY_FPS)
{
GScript("""
    MY_AVSSCRIPT=""+chr(13)+chr(10)
    EDLLIST=RT_ReadTxtFromFile(MY_EDL)                    # Read the EDL file
    LINESNR=RT_TxtQueryLines(EDLLIST)                     # Query Number of lines in String.

        CLIPNR=0
        For(i=0,LINESNR-1) {                               #Count video clips
            LINE=RT_TxtGetLine(EDLLIST,i)                     # Get one line at a time
            if (FindStr(RightStr(LINE,StrLen(LINE)-4)," V    C")>0){
             CLIPNR=CLIPNR+1
            }
        }
        if (CLIPNR>0) {MY_AVSSCRIPT=MY_AVSSCRIPT+"video = "+CHR(92)+chr(13)+chr(10)}    # Add Video first line    

        CLIPCNT=1
        For(i=0,LINESNR-1) 
        {
            LINE=RT_TxtGetLine(EDLLIST,i)                     # Get one line at a time
            #if ( LeftStr(LINE,4)==string(int(value(LeftStr(LINE,4)))) ) 
            #{          
                if (FindStr(RightStr(LINE,StrLen(LINE)-4)," V    C")>0){          # Find Vide and Cut
                    IMPFILE=MidStr(LINE,6,FindStr(RightStr(LINE,StrLen(LINE)-4)," V    C")-2)
                    HI=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9,2))         # Read the INPOINT
                    MI=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+3,2))
                    SI=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+6,2))
                    FI=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+9,2))
                    INPOINT=tc(INT(HI), INT(MI), INT(SI), INT(FI), MY_FPS)
                    HO=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+12,2))      # Read the OUTPOINT
                    MO=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+15,2))
                    SO=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+18,2))
                    FO=VALUE(MidStr(LINE,FindStr(LINE, "         ")+9+21,2))
                    OUTPOINT=tc(INT(HO), INT(MO), INT(SO), INT(FO), MY_FPS)                    
                    MY_AVSSCRIPT=MY_AVSSCRIPT+"DirectshowSource("+CHR(34)+MY_DIRECTORY+IMPFILE+"."+MY_EXTENTION+CHR(34)+").TRIM("+string(INPOINT)+","+string(OUTPOINT)+")"
 
                     IF (CLIPCNT<CLIPNR) 
                     {
                              MY_AVSSCRIPT=MY_AVSSCRIPT+" + "+CHR(92)+chr(13)+chr(10)
                              CLIPCNT=CLIPCNT+1
                     } else if (CLIPCNT==CLIPNR){MY_AVSSCRIPT=MY_AVSSCRIPT+chr(13) + chr(10)}             
                }     
            #}
        }
        if (CLIPCNT>1) {MY_AVSSCRIPT=MY_AVSSCRIPT+"Video"+chr(13) + chr(10)} 
        
        return geval(MY_AVSSCRIPT)                        
""")
#WriteFileStart("MY_AVSSCRIPT.txt", """ MY_AVSSCRIPT """)
}


Function tc(int h, int m, int s, int f, int fps)
{
mt = h*60+m
todrop = (mt/10)*18 + ((mt%10==0) ? 0 : 2*(mt%10-1))
result = (mt*60+s)*fps+f - todrop
return result
}

Example EDL from Lightworks:

TITLE: Edit #1   FORMAT: CMX3600
001  VJ0K006W V    C         00:00:19:05 00:00:41:07 00:00:00:00 00:00:22:02
002  E00B00CU AA   C         00:00:19:05 00:00:41:07 00:00:00:00 00:00:22:02
 COMMENT: AUDIO MAPPING   FROM CH1 TO CH2
003  VJ0K006W V    C         00:00:00:00 00:00:18:18 00:00:22:02 00:00:40:20
003  900      V    D    025  00:00:01:00 00:00:03:06 00:00:40:20 00:00:43:01
004  E00B00CU AA   C         00:00:00:00 00:00:19:05 00:00:22:02 00:00:41:07
 COMMENT: AUDIO MAPPING   FROM CH1 TO CH2
005  E00B00CU AA   C         00:00:34:12 00:00:36:06 00:00:41:07 00:00:43:01
 COMMENT: AUDIO MAPPING   FROM CH1 TO CH2
006     BL    AA/V C         00:00:00:00 00:00:22:02 00:00:43:01 00:01:05:03

For testing - just change VJ0K006W to a video file(without extention) on your computer.

[StainlessS]

If working with larger strings, may come across this problem:
http://forum.doom9.org/showthread.php?t=164212

Shall also be adding a func to write a file.
As far as the RT_Stats thing is concerned, Chr(13) is removed on input from text file leaving only chr(10)
as the line separator.
However, internally it (RT_TxtGetLine & RT_TxtQueryLines) will use either Chr(13), Chr(10) or the pair of them
(in either order) as a separator.
When writing using Avisynth WriteFileStart, chr(10) will be replaced by Chr(13),Chr(10).
Never write Chr(10),Chr(13) sequence as a line separator for file output, notepad and other software
will get very confused if you do (EDIT: I think but not sure, that using Chr(13),Chr(10) in WriteFileStart
may result in Chr(13),Chr(13),Chr(10) on output, and a confused Notepad etc,check it out).

Might be best to avoid using Chr(13) and stick to Chr(10) only for everything.

Further:
RT_TxtGetLine strips Any Chr(13) or Chr(10) and return just a bare string.

Have implemented multi wildcards in extension for RT_WriteFileList, eg

"D:\VID\*.AVI|MP*G"
"D:\Pics\*.BMP|JP*G|PNG"

Next version.

EDITED:

[tin3tin]

Thank you again for your plugin! Doing this stuff would have been impossible without it. On Chr(13)+Chr(10). I'll have to check if geval/eval will work with only chr(10).

[StainlessS]

Eval works fine with just Chr(10), Dont know but guess that Avisynth strips Carriage Returns during script load,
they are not really necessary. (File read ops in text mode auto strip CR)
Have changed the QueryBorderCrop() thing to use Chr(10) (did use 13) for next release.
At the time I did that, was considering a sort of two dimensional array but cannot ensure that a user would
stick to that.

[StainlessS]

Update is here:
http://forum.doom9.org/showthread.php?t=165479

[Zarxrax]

tin3tin, not sure if you would be interested but I believe you can achieve an edl importer via a macro in Avspmod. Avspmod macros basically let you write code in Python and then create or modify an avs script with it. Python is generally considered to be a fairly easy language.

[tin3tin]

That might be an idea, however I can't code python. There is a Python script here for importing edl in Blender. So all the parsing is done in Python code all ready. What is left is to change the Blender commands to Avisynth.