MakeMultiPartScripts

[StainlessS]

Two Functions to processes avs script in multiple parts.

ZIP containing scripts @ MediaFire below this post (~4.5KB).

To Create Multi-Part Scripts, USE VDub2 or any Player Except VLC, and MPC-HC only if nothing else available. Do NOT USE AvsPMod (it dont work well with RT_FselOpen file selector).

MakeMultiPartScripts.avs # Requires RT_Stats, Avs+ OR GScript.

# MakeMultiPartScripts.avs : v1.00
#######
PARTS=4 # Edit as appropriate
#######
myName="MakeMultiPartScripts: "
Assert(PARTS>1,RT_String("MakeMultiPartScripts.avs: PARTS MUST be greater than 1 (%d)",PARTS))
IsAvsPlus=(FindStr(VersionString, "AviSynth+")>0 || FindStr(VersionString, " Neo")>0)
HasGScript=RT_FunctionExist("GScript")
Assert(IsAvsPlus || HasGScript,RT_String("%sNeed either GScript or AVS+",myName))
#######
FSEL_TITLE="Select AVS files"
FSEL_DIR="."
FSEL_FILT="AVS files|*.AVS"
FSEL_MULTI=True
FILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(!(FILE_LIST.IsInt && FILE_LIST==0),RT_String("%sUser Cancelled in FileSelector - No action taken",myName))
Assert(FILE_LIST.IsString,RT_String("%sRT_FSelOpen Error=%s",myName,String(FILE_LIST)))

SCRIPT_TMPT = """
PART = %d
PARTS = %d
Import("%s")
FC = FrameCount
Start = ((part==0 ) ? -1 : part * (FC-1) / PARTS) + 1
End = ((part==PARTS-1) ? FC-1 : (part+1) * (FC-1) / PARTS)
End = (End == Start) ? -1 : End
Trim(Start,End)
# Subtitle("Trim("+String(Start)+","+String(End)+")")
Return Last
"""

GS="""
NLINES = RT_TxtQueryLines(FILE_LIST)
for(i=0,NLINES-1) {
PARTSNAME = RT_TxtGetLine(FILE_LIST,i)
for(part=0,PARTS-1) {
PartName = RT_String("%s_PART_%d.avs",PARTSNAME,part+1)
ScriptInst=RT_String(SCRIPT_TMPT,part,PARTS,PARTSNAME)
RT_WriteFile(PartName,"%s",ScriptInst)
}
}
"""
HasGScript ? GScript(GS) : Eval(GS) # Use GSCript if installed (loaded plugs override builtin)

MessageClip(RT_String("ALL DONE, Processed %d avs files of %d parts each",NLINES,PARTS))


EDIT: For this script, split scripts are self calculating for split points, ie if changing trims in source script then dont need to re-create split scipts.

[StainlessS]

2nd function to split processing so as to start each section on I frames.
Prompted by this thread:- https://forum.doom9.org/showthread.php?t=176239

FFMS_MakeMultiPartScripts.avsi

# FFMS_MakeMultiPartScripts.avsi

Function FFMS_MakeMultiPartScript(String FnList,String "OutPath",Int "Parts") {
/*
FFMS_MakeMultiPartScripts v0.04 ::: Requires RT_Stats, Avs+ OR GScript.

FFMS_MakeMultiPartScript(String FnList,String "OutPath"="",Int "Parts"=4)
FnList, No default. Multi-line string of avs filenames(1 or more).
Source scripts to split must use FFMS2 source filter.
Source scripts clip length must be at least (20 * Parts) frames else error abort.
This function sets a Global FFMS_MakeMultiPartScript_Glb variable, can be detected in source
avs scripts so as to determine if called by this function (for whatever reason).
OutPath, Default "", Same Directory as source file directory. Path (must exist) where output scripts written.
Where OutPath = default "", and multiple input files with different directories given in FnList, OutPath will use input file directory
for each input file (even when different directories, will not be different paths where fileSelector client script used).
Use "." for current directory.
Parts, Default 4. Number of parts that source script is split into.
*/
myName="FFMS_MakeMultiPartScript: "
Global FFMS_MakeMultiPartScript_Glb="Sausages" # *** NOTE *** Exported as Global for possible use via avs Script that will be split.
OutPath=Default(OutPath,"") Parts=Default(Parts,4)
Assert(Parts>1,RT_String("MakeMultiPartScripts.avs: PARTS MUST be greater than 1 (%d)",Parts))
IsAvsPlus=(FindStr(UCase(versionString),"AVISYNTH+")!=0) HasGScript=RT_FunctionExist("GScript")
Assert(IsAvsPlus || HasGScript,RT_string("%sEssential Avs+ OR GScript installed",myName))
Assert(FnList!="",RT_String("%sFn Cannot be ''",myName))
Script_Tmpt = """Return Import("%s").Trim(%d,%d) # PART %d of %d"""
ORG_OutPath=OutPath
GS="""
NLINES = RT_TxtQueryLines(FnList)
for(i=0,NLINES-1) { # Check Input files Exist
InFn=RT_TxtGetLine(FnList,i).RT_GetFullPathName
Assert(Exist(InFn),RT_String("%sFn Does Not Exist\n'%s'",myName,InFn))
}
for(i=0,NLINES-1) {
InFn = RT_TxtGetLine(FnList,i).RT_GetFullPathName
OutPath = (ORG_OutPath=="") ? InFn.RT_FilenameSplit(3) : ORG_OutPath.RT_GetFullPathName
OutPath=OutPath.RevStr
While(OutPath.FindStr("\")==1 || OutPath.FindStr("/")==1) { OutPath=OutPath.MidStr(2) }
OutPath=OutPath.RevStr
Assert(Exist(OutPath),RT_String("%sOutPath Does Not Exist\n'%s'",myName,OutPath))
OutPath=OutPath + "\"
OutFn = OutPath + RT_FilenameSplit(InFn,4)
c = Import(InFn)
FC = c.FrameCount
c.RT_AverageLuma(n=FC-1,w=1,h=1)
Assert(RT_VarExist("FFPICT_TYPE"),RT_String("%sFFVideoSource FFPICT_TYPE does not exist\n'%s'",myName,InFn))
Assert(FC>=20*PARTS,RT_String("%sClip length(%d) must be at least 20*Parts(%d) frames\n'%s'",myName,FC,20*PARTS,InFn))
End = -1
for(part=0,PARTS-1) {
Start = End + 1
if(Part == Parts-1) {
End = FC -1
} Else {
ChkEnd = (FC - Start) / (PARTS - part) + Start
ChkLim = 3 * (FC - Start) / (2*(PARTS - part)) + Start
End=-1
for(j=ChkEnd+1,ChkLim) {
c.RT_AverageLuma(n=j,w=1,h=1)
if(FFPICT_TYPE == 73) { End = j-1 j=FC }
}
if(End<0) {
ES=RT_String("%sCannot find I Frame for part %d\n'%s'",myName,Part+1,InFn)
RT_DebugF("%s",ES,name=myName)
for(j=0,part-1) {
OutPartName = RT_String("%s_PART_%d.avs",OutFn,j+1)
RT_DebugF("Deleting %s",OutPartName,name=myName)
RT_FileDelete(OutPartName)
}
Assert(False,ES)
}
}
OutPartName = RT_String("%s_PART_%d.avs",OutFn,part+1)
ScriptInst=RT_String(Script_Tmpt,InFn,Start,End,part+1,PARTS)
RT_DebugF("Writing '%s'\n '%s'",OutPartName,ScriptInst,name=myName)
RT_WriteFile(OutPartName,"%s",ScriptInst)
}
}
"""
HasGScript ? GScript(GS) : Eval(GS) # Use GScript if installed (loaded plugs override builtin)
Return MessageClip(RT_String("ALL DONE, Processed %d avs files of %d parts each",NLINES,PARTS))
}


FFMS_MakeMultiPartScripts_Client.avs

# FFMS_MakeMultiPartScripts_Client.avs

Import(".\FFMS_MakeMultiPartScripts.avsi")

#######
# To Create Multi-Part Scripts, USE VDub2 or any Player Except VLC, and MPC-HC only if nothing else available. Do NOT USE AvsPMod (it dont work well with RT_FselOpen file selector).
#######
PARTS = 4 # Part Scripts to create.
FSEL_DIR = "." # Start Path for File Selector, current directory.
OUTPATH = "" # Output Path where avs files created, ""=Same directory as source files. ["." = current directory]
#######
myName="FFMS_MakeMultiPartScript_Client: "
FSEL_TITLE="Select AVS files"
FSEL_FILT="AVS files|*.AVS"
FSEL_MULTI=True # Allow Multi-Select of input avs files.
FILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(!(FILE_LIST.IsInt && FILE_LIST==0),RT_String("%sUser Cancelled in FileSelector - No action taken",myName))
Assert(FILE_LIST.IsString,RT_String("%sRT_FSelOpen Error=%s",myName,String(FILE_LIST)))
###
Return FFMS_MakeMultiPartScript(FILE_LIST,OutPath=OUTPATH,Parts=PARTS)


Source script to split, requires something like below [EDIT: The FFPICT_TYPE/Subtitle stuff just to show that frame 0 of split script is an I frame]

FFMS_MultiPart_Source.avs

# FFMS_MultiPart_Source.avs

fn=".\tears_of_steel_1080pwebm.webm"

FFVideoSource(Fn)
#FFMpegSource2(Fn)

myName="FFMS_MultiPart_Source.avs: "

(RT_VarExist("FFMS_MakeMultiPartScript_Glb"))
\ ? RT_DebugF("Script called via FFMS_MakeMultiPartScripts.avsi",name=myName)
\ : RT_DebugF("Script NOT called via FFMS_MakeMultiPartScripts.avsi",name=myName)

SSS="""
T=Chr(FFPICT_TYPE)
S=String(current_frame,"%.0f]") + " " + T
return Subtitle(S)
"""
return Scriptclip(SSS,After_Frame=True)


Result of the FFMS_MakeMultiPartScripts_Client.avs script
https://i.postimg.cc/LsyHLC3x/FFMS-Make-Multi-Part-Scripts-Client-00.jpg (https://postimages.org/)

Debug output from FFMS_MakeMultiPartScripts_Client.avs script [EDIT: Non coloured is some of the output from dll loader avsi in plugins directory]

00000005 0.11214769 [4936] InitExternalPlugins: .\DGDecode\DGDecode_x86.DLL LOADED
00000006 0.12637249 [4936] FFMS2 avs plugin: Initializing...
00000007 0.12701488 [4936] InitExternalPlugins: .\FFMS_C\ffms2.dll LOADED
00000008 0.12901194 [4936] InitExternalPlugins: .\FFMS_C\ffms2.avsi LOADED
00000009 0.13371623 [4936] InitExternalPlugins: Loading .\LSMASH_CPP\LSMASHSource.dll
00000010 0.17118925 [4936] InitExternalPlugins: .\DGDecode\DGDecode_x86.DLL LOADED
00000011 0.17142294 [4936] InitExternalPlugins: .\FFMS_C\ffms2.dll LOADED
00000012 0.17346205 [4936] InitExternalPlugins: .\FFMS_C\ffms2.avsi LOADED
00000013 0.17369319 [4936] InitExternalPlugins: Loading .\LSMASH_CPP\LSMASHSource.dll
00000014 1.91770518 [4936] FFMS_MultiPart_Source.avs: Script called via FFMS_MakeMultiPartScripts.avsi
00000015 5.91507673 [4936] FFMS_MultiPart_Source.avs: Writing 'D:\MakeMultiPartScripts\FFMS_MultiPart_Source_PART_1.avs'
00000016 5.91516447 [4936] FFMS_MultiPart_Source.avs: 'Return Import("D:\MakeMultiPartScripts\FFMS_MultiPart_Source.avs").Trim(0,4472) # PART 1 of 4'
00000017 6.75278282 [4936] FFMS_MultiPart_Source.avs: Writing 'D:\MakeMultiPartScripts\FFMS_MultiPart_Source_PART_2.avs'
00000018 6.75284672 [4936] FFMS_MultiPart_Source.avs: 'Return Import("D:\MakeMultiPartScripts\FFMS_MultiPart_Source.avs").Trim(4473,8869) # PART 2 of 4'
00000019 8.38078785 [4936] FFMS_MultiPart_Source.avs: Writing 'D:\MakeMultiPartScripts\FFMS_MultiPart_Source_PART_3.avs'
00000020 8.38088036 [4936] FFMS_MultiPart_Source.avs: 'Return Import("D:\MakeMultiPartScripts\FFMS_MultiPart_Source.avs").Trim(8870,13267) # PART 3 of 4'
00000021 8.38193226 [4936] FFMS_MultiPart_Source.avs: Writing 'D:\MakeMultiPartScripts\FFMS_MultiPart_Source_PART_4.avs'
00000022 8.38198757 [4936] FFMS_MultiPart_Source.avs: 'Return Import("D:\MakeMultiPartScripts\FFMS_MultiPart_Source.avs").Trim(13268,17619) # PART 4 of 4'



3rd of 4 resultant scripts.
FFMS_MultiPartSourceScript_PART_3.avs

Return Import("D:\MakeMultiPartScripts\FFMS_MultiPart_Source.avs").Trim(8870,13267) # PART 3 of 4


Debug output from FFMS_MultiPartSourceScript_PART_3.avs

00000015 0.44113362 [1540] FFMS_MultiPart_Source.avs: Script NOT called via FFMS_MakeMultiPartScripts.avsi


A crop section of 1st frame from part #3 script showing first frame is I frame.
https://i.postimg.cc/kGLPp9LL/FFMS-Multi-Part-Source-PART-3-00.jpg (https://postimages.org/)

EDIT: For this script, split scripts are NOT self calculating for split points, ie if changing trims in source script then DO need to re-create split scipts.

EDIT: A few comments from users in another thread.
I did a quick test with four instances of Virtualdub, setting CPU affinity in Windows task manager. On a CPU heavy script I went from approximately 3 fps single-threaded to 4 fps combined. The downside is that the system became unresponsive. My CPU is a i5-2500k.

Set each encoder instance's priority to "Low" (Task Manager) / "Idle" (Process Manager (https://technet.microsoft.com/en-us/sysinternals/processexplorer.aspx)) / "LOW" (cmd.exe's "start" command when using x264 etc.).

@jriker1

I thought I had a problem (bug) with RT_FSelOpen due to the previous MakeMultiPartScript.avs but turns out not a bug.
I was using Media Player Classic - Home Cinema to open, and if I clicked Cancel then it would immediately bring up another File Selector.
Thought was bug in RT_FSelOpen but seems that MPC-HC will immediately retry because of Assert() alert (due to user Cancel) without any
frames being delivered from script. Does not happen with other players and seems that MPC-HC is just trying to be helpful in re-starting what
it thinks is glitch in the system. Anyway, just pointing it out if you use MPC-HC (just updated to latest stable and still does it), but does not
happen in other players that I've tried.
So because of above quote, to generate the individual multipart scripts, dont use with MPC-HC,
and also dont use with AvsPMod (weird things happen, AvsPMod inserts filename into script [any script using RT_FSelOpen] and buggers up further script functionality).
Ideally use any other player (other than MPC-HC) except VLC (which dont work with avs , leastwise the one I use dont work), or use VDub2.

[StainlessS]

Post #2 update v0.02.

[StvG]

Thanks for your time. It works flawless.
Is it possible to change default OUTPATH to the same directory as the source script?

[StainlessS]

Post #2 new version FMMS script v0.04
(download in ZIP form from MediaFire below this post)


Is it possible to change default OUTPATH to the same directory as the source script?

Done, "." = current dir, "" default = Same path as source files even when multiple different source paths [ie when not using FileSelector].


FFMS_MakeMultiPartScripts v0.04 ::: Requires RT_Stats, Avs+ OR GScript.

FFMS_MakeMultiPartScript(String FnList,String "OutPath"=".",Int "Parts"=4)
FnList, No default. Multi-line string of avs filenames(1 or more).
Source scripts to split must use FFMS2 source filter.
Source scripts clip length must be at least (20 * Parts) frames else error abort.
This function sets a Global FFMS_MakeMultiPartScript_Glb variable, can be detected in source
avs scripts so as to determine if called by this function (for whatever reason).
OutPath, Default "", Same Directory as source file directory. Path (must exist) where output scripts written.
Where OutPath = default "", and multiple input files with different directories given in FnList, OutPath will use input file directory
for each input file (even when different directories, will not be different paths where fileSelector client script used).
Use "." for current directory.
Parts, Default 4. Number of parts that source script is split into.


EDIT: Arh damn, In doc says Outpath default = ".", I'll correct next version. [should be ""]
EDIT: Zip and post updated with fixed doc.

[StvG]

Much appreciated. Thanks.

[Blankmedia]

When I use Prefetch and SetMemoryMax, should I put it in the script to be split or the splitted script?

I already modified that part to add my pluggin directory :

SCRIPT_TMPT = """
ClearAutoloadDirs()
AddAutoloadDir("D:\\A encoder\\avs_plugin_372")

Does it matter?

Thank you

[StainlessS]

When I use Prefetch and SetMemoryMax, should I put it in the script to be split or the splitted script?

I would say put it in script to be split [the script has to work successfully prior to being split ::: Import() and Trim() of the splitting script should make little difference to Prefetch].

If whatever you do works (without error), then it should be OK.

[StainlessS]

This below, posted in another thread, [ https://forum.doom9.org/showthread.php?p=1969927#post1969927 ]

# ===========================

OK, here extracts PartNo from multi-part Filename, only tested with given script.


Function GetMultipartFileNamePartNo(String FN) {
FN = RT_FilenameSplit(FN,get=4) # Name node only, no path, no extension eg, "FFMS_MultiPart_Source_PART_1"
RFN = RevStr(FN)
S="" # Init
C = RFN.MidStr(1,1)
While("0" <= C <= "9") {
S = C + S
RFN = RFN.MidStr(2)
C = RFN.MidStr(1,1)
}
Return Eval("0" + S)
}


FN = "FFMS_MultiPart_Source_PART_123.avs"

PartNo = GetMultipartFileNamePartNo(FN)
Assert(PartNo > 0, "Bad PartNo")

BlankClip
Subtitle("Got PartNo = " + String(PartNo))


Produces clip Subtitled with "123".

Extracts the bit in RED as int.