RT_Stats, Compile-time/Runtime Functions v1.43 - 08 Oct 2014

[StainlessS]

Some parts may be liberated from Avisynth source code.
http://avisynth2.cvs.sourceforge.net/avisynth2/

v1.00, Implemented RGB with internal RGB -> YUV-Y conversion.
v1.01, Added RT_Debug function.
v1.02, Added several more functions.
v1.03, Added several more functions.
v1.04, Added RT_Call.
v1.05, Added RT_YInRange. Ucase bug fixed.
v1.07beta, 11 Dec 2012. Added Lots of stuff, Testing 1-2-3.
v1.08 Add some funcs
v1.09 Fixed RT_RgbChanAve w==1 bug (always blue channel).
v1.10 Added funcs. Fixed RT_TxtSort null string bug, Matrix default now all (width<=720?2:3)
v1.11 Fixed broken RT_LumaDifference & RT_LumaCorrelation, "must have two clips" bug.
Added temporary RT_LumaSNSSD()
v1.12 Added RT_LumaSSSD(), left RT_LumaSNSSD() in situ, probably removed in next version.
v1.13 RT_QueryBorderCrop CropLess/More modified. Experimental RT_LumaSSSD removed.
v1.14, Added RT_FindStr. Mods:- RT_Debug, StrReplace fns, RT_NumberValue and RT_HexValue.
v1.15, RT_YInRangeLocate, internally, Baffle separately limited to dimensions on both axis.
v1.16, RT_LumaSceneChange added.
v1.17, Added RT_LumaMovement. RT_Ord mod.
v1.18, Fixed RT_QueryBorderCrop RGB "Matrix NOT @ PC levels - Check!" bug.
Added RT_ColorSpaceXMod/YMod. Added RT_RgbChan family.
v1.19, RT_LumaSceneChange() moved cap @ 255.0 to just before return instead of on Lut, better metrics.
Originally like:- mt_lutxy(mt_polish("((x-y)/4)^2")).AverageLuma()
Changed RT_QueryBorderCrop() Thresh Massaging.
v1.20, Changed the way RT_LumaSceneChange works. Added Bias->Pord args. Added RT_Subtitle().
v1.21, RT_Subtitle FIX: single backslash @ end of '\n' terminated string let '\n' pass without conversion to '\r'.
Mod, Strictly eg "\n" and not "\N" interpreted as linefeed etc.
Added FSEL functions.
v1.22, Fixed RT_RGBChanXXXX channels. Changed RT_YStdev() to Standard Deviation from Sample Standard Deviation.
v1.23, Mod RT_Debug, added Repeat arg.
Mod RT_YInRangeLocate, added Baffle_W and Baffle_H args.
Added RT_RgbInRange func.
Added RT_RgbInRangeLocate func.

v1.24, 28/9/13, Fixed Interlaced Y stepping.
Added Mask args to Y/RGB stats, Added RT_YPNorm, RT_RgbChanPNorm.
Added RT_TxtFindStr and RT_FileFindStr and RT_String.
v1.25, 28/9/13, Afterthough, changed RT_string(bool Esc) to int.
v1.26, 15/10/13, Fixed typo, XTRA_MASKMIN should have been XTRA_MASKMAX.
Convert all default func matrix args to switch rec709 if height >= 600
v1.27, Added RT_FunctionExist, RT_DebugF. Changed all returned Global vars to Local vars except for RT_SignalDAR.
v1.28, Added Array Funcs + lots of others. Added Esc arg to RT_Subtitle.
v1.30, 19 Dec 2013, Added DBase Functions, Increased Array attributes to 256, improved error messages.
v1.40, 9 Sept 2014, Fixed, RT_DBaseGetAttrib returned float instead of Int (for int attrib).
Added args Start and End to RT_QueryBorderCrop and RT_QueryLumaMinMax. Added RT_WriteFile + other funcs.
Now 1024 Array & DBase Attributes. Array and DBase now supports type double as private type for RT_Stats
internal routines, set or get as type Float from Avisynth script.
DBase max number of fields increased to 1024. Added DBase/Array String Attributes.
Added RT_QwikScan + other stuff.
v1.41, 15 Sept 2014, Fixed borked v1.40, (released wrong source + binary), RT_QwikScan PC, PD related stuff.
Changed RT_QwikScan LumaTol arg to float.
v1.42, 19 Sept 2014. Modified Float LumaTol imlementation. Added additional XP mode to RT_QwikScan, and affects
BEST MATCH results. Added RT_AvgLumaDif.
v1.43, 07 Oct 2014. Added Dbase/Array ID and RT_QwikScanEstimateLumaTol().



See post #2 for a (OLD) stripped down partial version of docs due to draconian 16kb limit.


Zip contains v2.58 and v2.60 dll's, + source + avs scripts.
RT_YStats_TestTimer.avs # Speed comparison script (eg YPlaneMin/RT_YPlaneMin).
ImageSplicer.avs
PlanetCrop.avs
QueryBorderCrop.avs
QueryLumaMinMax.avs
YInRangeLocate.avs
DAR.avs
LocateFrames.avs
MatchFrames.avs
s_ExLogo() # RT_Stats NOT required
TweenFramesI.avs
SecurityCamExtractMotion.AVS # v1.2, Update
DynaCrop.avs # New, with SC_SCSelect()
RT_LumaSceneChange_Graffer # added, + vb source
v1.21
Added FSel_DEMO avs files.




RT_Stats_Function_List v1.43


There follows a list of all function names together with CPP style argument specifiers that inform
Avisynth the argument types and optional names. Optional arguments have square brackets surrounding
their name as in [name] and are followed by a type specifier character that gives the type.
Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args,
followed by optional 'arg1' of type bool and optional 'arg2' of type int.

# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
###################################


RT_ColorSpaceXMod "c"
RT_ColorSpaceYMod "c[Laced]b"
RT_Stats "c"
RT_GraphLink "cc+b*"
RT_Subtitle "cs.*[align]i[x]i[y]i[vcent]b[expx]b[expy]b[esc]i"
RT_BitNOT "i"
RT_BitAND "ii"
RT_BitOR "ii"
RT_BitXOR "ii"
RT_BitASL "ii"
RT_BitASR "ii"
RT_BitLSL "ii"
RT_BitLSR "ii"
RT_BitTST "ii"
RT_BitCLR "ii"
RT_BitSET "ii"
RT_BitCHG "ii"
RT_BitROR "ii"
RT_BitROL "ii"
RT_BitSetCount "i"
RT_Hex "i[width]i"
RT_HexValue "s[pos]i"
RT_NumberString "i[base]i[width]i"
RT_NumberValue "s[base]i[pos]i"
RT_Call "s[Hide]b[Debug]b"
RT_GetLastError ""
RT_GetLastErrorString ""
RT_Debug "s+[name]b[repeat]b"
RT_DebugF "s.*[name]s[tabsz]i"
RT_GetProcessName "[Parent]b[debug]b"
RT_Undefined ""
RT_VarExist "s"
RT_FunctionExist "s"
RT_Ord "s[pos]i"
RT_Timer ""
RT_IncrGlobal "s[Init]i"
RT_Version ""
RT_VersionString ""
RT_VersionDll ""
RT_PluginDir ""
RT_PluginFunctions ""
RT_InternalFunctions ""
RT_ScriptName ""
RT_ScriptFile ""
RT_ScriptDir ""
RT_PluginParam "s"
RT_GetSystemEnv "s"
RT_GetFileTime "si"
RT_LocalTimeString "[file]b"
RT_VarType "."
RT_VarIsSame "..+[sig]b"
RT_TimerHP ""
RT_FileQueryLines "s"
RT_ReadTxtFromFile "s[Lines]i[Start]i"
RT_WriteFileList "ss[append]b"
RT_TxtWriteFile "ss[append]b"
RT_FileDelete "s"
RT_FileFindStr "ss[sig]b[pos]i[start]i[lines]i"
RT_WriteFile "ss.*[Append]b"
RT_FileRename "ss"
RT_FSelOpen "[title]s[dir]s[filt]s[fn]s[multi]b[debug]b"
RT_FSelSaveAs "[title]s[dir]s[filt]s[fn]s[debug]b"
RT_FSelFolder "[title]s[dir]s[debug]b"
RT_YDifference "c[n]i[delta]i[x]i[y]i[w]i[h]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaCorrelation "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaDifference "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaSceneChange "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[Bias]f[Gain]f[Cont]f[RPow]f[SPow]f"
"[SPMid]f[Pord]b"
RT_LumaMovement "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[mode]i[blkw]i[blkh]i[Bias]f[Gain]f"
"[Cont]f[RPow]f[SPow]f[SPMid]f[Pord]b"
RT_RGB32AsFloat "c[n]i[delta]i[x]i[y]i"
RT_RGB32AsInt "c[n]i[delta]i[x]i[y]i"
RT_FloatAsRGBA "f"
RT_YankChain "c[n]i[delta]i"
RT_TxtQueryLines "s"
RT_TxtGetLine "s[Line]i"
RT_StrAddStr "ss+"
RT_TxtAddStr "ss+"
RT_QuoteStr "s"
RT_StrReplace "sss[sig]b"
RT_StrReplaceDeep "sss[sig]b"
RT_StrReplaceMulti "sss[sig]b"
RT_StrPad "si[c]s"
RT_FindStr "ss[sig]b[pos]i"
RT_TxtFindStr "ss[sig]b[pos]i[start]i[lines]i"
RT_String "s.*[esc]i"
RT_TxtSort "s[mode]i"
RT_GetWorkingDir ""
RT_FilenameSplit "s[get]i"
RT_GetFullPathName "s"
RT_GetFileExtension "s"
RT_QueryLumaMinMax "c[Samples]i[Ignore]f[Prefix]s[Debug]b[X]i[Y]i[W]i[H]i[Matrix]i[Start]i[End]i"
RT_QueryBorderCrop "c[Samples]i[Thresh]f[Laced]b[xMod]i[yMod]i[wMod]i[hMod]i[Relative]b[Prefix]s[RLBT]i[Debug]b[Ignore]f[Matrix]i"
"[ScanPerc]f[Baffle]i[ScaleAutoThreshRGB]b[ScaleAutoThreshYUV]b[ATM]f[Start]i[End]i"
RT_YInRangeLocate "c[n]i[delta]i[x]i[y]i[w]i[h]i[baffle]i[thresh]f[lo]i[hi]i[matrix]i[prefix]s[debug]b[baffle_w]i[baffle_h]i"
RT_RgbInRange "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Rlo]i[Rhi]i[Glo]i[Ghi]i[Blo]i[Bhi]i"
RT_RGBInRangeLocate "c[n]i[delta]i[x]i[y]i[w]i[h]i[baffle]i[thresh]f[Rlo]i[Rhi]i[Glo]i[Ghi]i[Blo]i[Bhi]i[prefix]s[debug]b[baffle_w]i"
"[baffle_h]i"
RT_GetSAR "cf"
RT_GetDAR "cf"
RT_SignalDAR "f"
RT_SignalDAR2 "ii"
RT_GetCropDAR "cf[x]f[y]f[w]f[h]f"
RT_ArrayAlloc "s[Type]i[Dim1]i[Dim2]i[Dim3]i[StringLenMax]i"
RT_ArrayGetDim "s[Dim]i"
RT_ArrayGetType "s"
RT_ArrayGetElSize "s"
RT_ArrayGetNels "s"
RT_ArrayGetNelMax "s"
RT_ArrayGetDim1Max "s"
RT_ArrayGet "si[ix2]i[ix3]i"
RT_ArraySet "s.i[ix2]i[ix3]i"
RT_ArrayExtend "s[Add]i"
RT_ArrayAppend "s."
RT_ArrayGetAttrib "si"
RT_ArraySetAttrib "si."
RT_ArrayGetStrAttrib "si"
RT_ArraySetStrAttrib "sis"
RT_ArrayTypeName "i"
RT_ArrayGetID "si"
RT_ArraySetID "si."
RT_DBaseAlloc "sis[StringLenMax]i"
RT_DBaseFields "s"
RT_DBaseFieldType "si"
RT_DBaseFieldSize "si"
RT_DBaseRecords "s"
RT_DBaseRecordSize "s"
RT_DBaseRecordsMax "s"
RT_DBaseGetField "sii"
RT_DBaseSetField "sii."
RT_DBaseSet "si.+"
RT_DBaseExtend "s[Add]i"
RT_DBaseAppend "s.+"
RT_DBaseGetAttrib "si"
RT_DBaseSetAttrib "si."
RT_DBaseGetStrAttrib "si"
RT_DBaseSetStrAttrib "sis"
RT_DBaseTypeName "i"
RT_DBaseGetID "si"
RT_DBaseSetID "si."
RT_RgbChanMin "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMax "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMinMaxDifference "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMedian "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanAve "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanStdev "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanInRange "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[lo]i[hi]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanPNorm "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mu]f[d]i[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanStats "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[lo]i[hi]i[flgs]i[prefix]s[mu]f[d]i"
"[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMin "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMax "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMinMaxDifference "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMedian "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_AverageLuma "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneStdev "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YInRange "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[lo]i[hi]i[mask]c[maskmin]i[maskmax]i"
RT_YPNorm "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mu]f[d]i[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_YStats "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[lo]i[hi]i[flgs]i[prefix]s[mu]f[d]i"
"[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_ChanAve "c[n]i[Prefix]s"
RT_FrameDifference "cc[n]i[n2]i[ChromaWeight]f"
RT_LumaPixelsDifferentCount "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[Thresh]i"
RT_LumaPixelsDifferent "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[Thresh]i"
RT_QwikScanCreate "cs[PrevDB]s[NextDB]s[Matrix]i[Debug]b"
RT_QwikScan "ciciss[LumaTol]f[Flags]i[LC]f[LD]f[FD]f[PD]f[PC]i[XP]i[ChromaWeight]f[PDThresh]i[MaxDistance]i[Inclusive]b"
"[Prefix]s"
RT_QwikScanEstimateLumaTol "cc[n]i[n2]i[matrix]i"
RT_AvgLumaDif "c[n]i[slices]b[matrix]i"



EDIT: RT_Stats v2.00 Beta 12 also available via sig below.
See MediaFire or SendSpace in sig.

[StainlessS]

RT_Stats, Short description: EDIT: Many additions missing, see later posts. (again hit 16kb limit)

RT_GraphLink(clip source, clip c1, ... , clip cn, bool b1, ... , bool bn) Equiv "Echo()" in 2.6a4
The clips, c1 to cn are by default forcibly linked into the filter graph.

RT_Debug(string s1, ... , string sn,bool "name"=true)
Takes one or more strings, outputs the combined string to OutputDebugString()

RT_Call(String Cmd,bool "Hide"=false,bool "Debug"=false)
Run an external program using supplied command.

RT_YankChain(clip,int "n"=current_frame,int "delta"=0)
Process frame (n + delta) of a clip ie forcibly process the filter graph chain for frame n + delta.

RT_GetProcessName(bool "parent"=false,bool "debug"=false)
Returns string, name of current process.

RT_StrAddStr(String, String s1, ... , String sn)
Function to concatenate 2 or more strings.

RT_TxtAddStr(String, String s1, ... , String sn)
Function to concatenate 2 or more strings with Chr(10) line separators.

RT_TxtQueryLines(String)
String, the multiline string that you require a line count from.
Returns the number of Newline [Chr(10)] separated lines in a multiline string.

RT_TxtGetLine(String, Int "Line"=0)
Extract a single line from a multiline Newline[Chr(10)] separated string.

RT_StrReplace(string source,string find,string replace,bool "sig"=True)
Takes a source string, searches for all occurrences of find string and replaces the found strings with the replace string.

RT_StrReplaceDeep(string source,string find,string replace,bool "sig"=True)
Takes a source string, searches for all occurrences of find string and replaces the found strings with the replace string.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Differs from RT_StringReplace in that if length of replace < length of find string will rescan after a replacement to see if
the replaced string combines with currently existing string to create another find string, If length replace >= find, then calls
RT_StringReplace instead. Useful to replace eg " " with " ", (two spaces with 1 space), where more than two consecutive
spaces would end up with more than one consecutive resulting space if using plain RT_StrReplace.

RT_StrReplaceMulti(string source,string find,string replace,bool "sig"=True)
Takes a source string, searches for all occurrences of find substrings and replaces the found substrings with the replace substrings.
Find string and Replace string are both multiline [Chr(10) separated] strings, both of n lines where find[i] and replace[i]
are matched pairs, ie the first line of a multiline find is replaced by the first line of a multiline found, etc.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Differs from RT_StrReplace, in that both find and replace strings are multiline strings and MUST match in number of lines.
Cannot use either carriage return [Chr(13)] or newline [Chr(10)] in find or replace as these are interpreted as end of current line,
you would need to use RT_StrReplace() instead.

RT_QuoteStr(string)
Encloses supplied string arg in double quotes.

RT_TxtSort(String,int 'mode'=0)
Function to sort a Chr(10) separated multiline string, 12 modes.

RT_FileQueryLines(String)
Returns number of lines in a text file.

RT_TxtWriteFile(String,String,bool "append"=false)
The first arg is a string of text that you want to write to a file.
The second arg is the output filename.

RT_ReadTxtFromFile(String ,Int "Lines"=0,Int "Start"=0)
String Filename, Name of text file to load into a string.

RT_WriteFileList(String,String,bool "append"=false)
Writes to a file, a list of filenames matching a wildacard eg "D:\Pics\*.BMP|JP*G|PNG"

RT_Hex(int , int "width"=0)
First arg is an integer to convert to a hexadecimal string.
Width, (0, 0 -> 8) is the minimum width of the returned string.

RT_HexValue(String,"pos"=1)
Returns an int conversion of the supplied hexadecimal string.

RT_NumberString(int ,int "base"=10, int "width"=0)
Converts a number (base 2->36) into a string.

RT_NumberValue(String,int "base"=10,"pos"=1)
Returns an int conversion of the supplied number base string.
Base, (10, 2 -> 36) is the number base that the string uses.

RT_BitNOT(int)
Returns the integer result of a bitwise NOT operation on it's int argument.

RT_BitAND(int, int)
Returns the integer result of a bitwise AND operation on the two int arguments.

RT_BitOR(int, int)
Returns the integer result of a bitwise OR operation on the two int arguments.

RT_BitXOR(int, int)
Returns the integer result of a bitwise XOR operation on the two int arguments.

RT_BitLSL(int, int)
Returns the integer result of a Logical Shift Left on the first integer, by a shift count of the second int.
An Arithmetic Shift Left and a Logical Shift Left, are essentially the same.

RT_BitLSR(int, int)
Returns the integer result of a Logical Shift Right on the first integer, by a shift count of the second int.
Logical shift right will shift zeros into the high bits.

RT_BitASL(int, int)
Returns the integer result of an Arithmetic Shift Left on the first integer, by a shift count of the second int.
An Arithmetic Shift Left and a Logical Shift Left, are essentially the same.

RT_BitASR(int, int)
Returns the integer result of an Arithmetic Shift Right on the first integer, by a shift count of the second int.
Arithmetic shift right will shift into the high bits, zeros for a +ve number and ones for a -ve number, so -ve
numbers will stay -ve and +ve stay +ve.

RT_BitTST(int, int)
Returns the bool result of a bit test on first integer, the bit number is given in the second int.
A zero in the bit returns False and a 1 returns True.

RT_BitCLR(int, int)
Returns the integer result of a bit clear (make 0) on first integer, the bit number is given in the second int.

RT_BitSET(int, int)
Returns the integer result of a bit Set (make 1) on first integer, the bit number is given in the second int.

RT_BitCHG(int, int)
Returns the integer result of a bit change (0->1, 1->0) on first integer, the bit number is given in the second int.

RT_BitROR(int, int)
Returns the integer result of a bit Rotate Right on first integer, the bit rotate count is given in the second int.

RT_BitROL(int, int)
Returns the integer result of a bit Rotate Left on first integer, the bit rotate count is given in the second int.

RT_BitSetCount(int)
Return an int, the count of the number of set bits (1's) in arg int.

RT_Undefined()
Returns same as undefined() in v2.6. Defined(RT_Undefined()) would return false.

RT_FloatAsRGBA(float)
Given a float arg, returns an int formatted as for use as a color to eg BlankClip.
Can later be recovered from that clip using RT_RGB32AsFloat().

RT_RGB32AsFloat(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0)
Given an RGB32 clip that had a pixel value created via RT_FloatAsRGBA(), gets that pixel and returns as the
original float given to RT_FloatAsRGBA.

RT_RGB32AsInt(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0)
Given an RGB32 clip that had a pixel value created from an Int, gets that pixel and returns as the
original Int.

RT_RgbChanAve(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0, int "h"=0,bool "interlaced"=false,int "chan"=0)
Return the float average value (0.0 -> 255.0) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).

RT_AverageLuma(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value average luma (0.0 -> 255.0) in frame(n+delta) for area x,y,w,h.

RT_YPlaneMin(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=)
Returns int value minimum luma (0 -> 255) in frame(n+delta) for area x,y,w,h.
Threshold is a percentage, stating how many percent of the pixels are allowed below minimum. The threshold is optional and defaults to 0.0

RT_YPlaneMax(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=)
Returns int value maximum luma (0 -> 255) in frame(n+delta) for area x,y,w,h.
Threshold is a percentage, stating how many percent of the pixels are allowed above maximum. The threshold defaults to 0.0

RT_YPlaneMinMaxDifference(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
bool "interlaced"=false,float "threshold"=0.0,int "Matrix"=)
Returns int value luma range (maximum - minimum difference) (0 -> 255) in frame(n+delta) for area x,y,w,h.
Threshold is a percentage, stating how many percent of the pixels are allowed below minimum or above maximum. The threshold defaults to 0.0

RT_YPlaneMedian(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "Matrix"=)
Returns int value luma median [equiv RT_YPlaneMin(threshold=50.0)] (0 -> 255) in frame(n+delta) for area x,y,w,h.
Threshold is a percentage, stating how many percent of the pixels are allowed above or below minimum. The threshold is optional and defaults
to 0.0

RT_YPlaneStdev(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "Matrix"=)
Returns FLOAT value luma Standard Deviation (0.0 -> 255.0) in frame(n+delta) for area x,y,w,h.

RT_YDifference(clip,int "n"=current_frame,int "delta"=1,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
bool "interlaced"=false,int "Matrix"=)
Returns FLOAT value luma difference (0.0 -> 255.0) between frame n area x,y,w,h, and frame (n+delta) area x2,y2,w,h.
Note, by default it will be equivalent to YDifferenceToNext as delta defaults to 1 and x,y,w,h defaults full frame.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.

RT_LumaDifference(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=)
Returns FLOAT value luma difference (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.

RT_YInRange(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "Matrix"=,
int "lo"=128,int "hi"=lo)
Returns float value (0.0 -> 1.0) being the amount of pixels in the range "lo" to "hi" (inclusive), 1.0 is equivalent to 100%.
Implemented as requested by Martin53 (thank you), NOTE, differs from other funcs that return range 0.0 to 255.0.
NOTE, "hi" defaults to "lo".

RT_LumaCorrelation(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "n2"=current_frame,
int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width>=720?2:3))
Returns FLOAT value luma correlation (-1.0 -> 1.0) between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.

RT_GetLastError()
Return a error number describing previous system error (if available). Perhaps use after eg RT_Call.

RT_GetLastErrorString()
Return a string describing previous system error (if available).

RT_StrPad(String,Int n,string "c"=" ")
Pads an unnamed string to at least int n (unnamed) length with first character of string c (Default SPACE).

RT_GetWorkingDir()
Return current Working directory, directory of script, or dir set by SetWorkingDir().

RT_GetFullPathName(string "name")
Creates an absolute or full path name for the specified relative path 'name'.

RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the parts joined together.

RT_GetFileExtension(string s)
Returns filename extension from the supplied filename, including '.' prefix.

RT_FileDelete(string fn)
Delete unnamed file fn.

RT_VarExist(string)
Given the name (string) of the variable that you want to test for existence, returns true if exists, else false.

RT_Ord(string)
Returns same as 'Ord' in v2.6, the ASCII code for the 1st letter in string (0 if empty string "").

RT_Timer()
Returns time in seconds since start of process.

RT_Ystats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=(width<=720?2:3),int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="YS_")
Returns multiple results as for above single frame Luma sampling functions as Global Variables (prefixed with the prefix string arg).

RT_GetSAR(clip,dar float)
Gets Sample (pixel) Aspect Ratio from DAR Display Aspect Ratio.

RT_GetDAR(clip,sar float)
Gets DAR Display Aspect Ratio from SAR, Sample (pixel) Aspect Ratio.

RT_SignalDAR(float) # See Megui wiki (name change from SetDAR)
Signal DAR to MEGUI. Sets global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

RT_GetCropDAR(clip,float DAR,float "X"=0,float "Y"=0,Float"W"=0,float "H"=0)
Call prior to Crop/Resize with (possibly fractional) cropping to calc resultant DAR, X,Y,W,H are cropping coords

RT_QueryLumaMinMax(clip c,int "Samples"=24,float "Ignore"=0.2,String "Prefix"="QLMM",bool "DEBUG"=false,int "X"=0,int "Y"=0,int "W"=0,int "H"=0, \
int "Matrix"=(width<=720?2:3))
Prescan function to get Luma Min,Max for clip scanning Samples frames area x,y,w,h.

Function RT_QueryBorderCrop(clip c,int "Samples"=24,Float "Thresh"=-32,bool "Laced"=true,int "XMod",int "YMod",int "WMod",int "HMod", \
bool "Relative"=false, String "Prefix"="QBCrop",int "RLBT"=15,bool "DEBUG"=false,float "Ignore"=0.2,int "Matrix"=(width<=720?2:3), \
Float "ScanPerc"=49.0,int "Baffle"=4, bool "ScaleAutoThreshRGB"=true,bool "ScaleAutoThreshYUV"=false)
Prescan function to get coords for eg cropping black borders by sampling Samples frames.

Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(width<=720?2:3),String "Prefix"="YIRL_",bool "Debug"=false)
Clip function to scan frame n+delta for an area where percentage of pixels in luma range Lo to Hi (inclusive) breaks a threshold over at least Baffle
consecutive number of scanlines (all four sides).

RT_FindStr(String S,string Substr,bool "Sig"=True,int "Pos"=1)
Finds unnamed string Substr in unnamed string S.
Optional bool Sig: Default=True, does case significant comparison, insignificant if false.
Optional int Pos: Default=1, is start position in string S to find string Substr. If Pos smaller than 1 or greater than length of string s,
then will return 0, Not Found.
Returns position (1 relative) of found string within the FULL LENGTH string S, or 0 if not found.
Differs from Avisynth FindStr in Sig and Pos args and in that if either string S or string Substr is "" then will ALWAYS return 0, 'Not Found'.

ADDED IN v1.16

RT_LumaSceneChange(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value (0.0 -> 255.0) scene change detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Is implemented sort of like:- mt_lutxy(mt_polish("((x-y)/4)^2")).AverageLuma() where x=(n+delta) and y=(n2+delta2).

List of All function names current @ v1.23 here:
http://forum.doom9.org/showthread.php?p=1643148#post1643148

[Gavino]

Nice work, Mr S.

Runtime Filters v0.0b 24 July "012.
It's worth pointing out that these are not just runtime filters(*), but (by virtue of the n and n2 parameters) can also be used at compile-time, as your example with GScript shows.

As I have previously described, this can be done with the standard runtime functions by manually setting current_frame, but this is really 'cheating' the runtime environment. Your functions allow this to be done more cleanly.

(*)Actually, they are not strictly filters either, as they don't return a clip.
The wiki (http://avisynth.org/mediawiki/Runtime_environment) distinguishes runtime filters (eg ScriptClip) from the runtime functions (like AverageLuma).

[StainlessS]

Arh Mr G,
A rose by any other name would smell as sweet, No Not You, the damn filters/functions. :)

OK, Ill edit the description, good job I only gave them an 'RT_' prefix.

Unfortunately, I'm getting something like '-IND000000' errors in the correlation thing which
I think is some kind of 'NaN' Not A Number type error. I might have to resort to the two pass
method which uses a lot less huge numbers, but I'm not gonna give up just yet, trying to
force it into working correctly. I used the 4th equation in the 1st link provided which can be done
in one scan of the frame.
A line that might produce the error (especially at eg 'fade to black'):

RT_Correlation(last,last.invert)

on live video.

[Wilbert]

A line that might produce the error (especially at eg 'fade to black'):

RT_Correlation(last,last.invert)

double div = sqrt((double)(Pixels * Sx2 - (Sx*Sx))) * sqrt((double)(Pixels * Sy2 - (Sy*Sy)));

Perhaps the stuff in the sqrt gets negative for some reason?

[jmac698]

I hope you realize that's it's normal to get a 0 for divisor when doing a correlation. It is valid for a correlation to be undefined. I think you could get this by comparing, 2,3 and 2,2.

[StainlessS]

double div = sqrt((double)(Pixels * Sx2 - (Sx*Sx))) * sqrt((double)(Pixels * Sy2 - (Sy*Sy)));

Perhaps the stuff in the sqrt gets negative for some reason?

Yep, -ve in one of the sqrt fns, I think one of subracts ran out of
precision producing a -ve result [Sy2 -(Sy*Sy)], I think.
EDIT: You must have magical powers, my mediafire account shows zero downloads for the plug.

I hope you realize that's it's normal to get a 0 for divisor when doing a correlation. It is valid for a correlation to be undefined. I think you could get this by comparing, 2,3 and 2,2.

Problem is, I dont know how to check for eg '-IND00000000' or
whatever it was.

EDIT: Gotta be 20 years since I last did much floating point stuff.

EDIT: Found this in Float.h, looks like what I want.


/* IEEE recommended functions */

_CRTIMP double __cdecl _copysign (double, double);
_CRTIMP double __cdecl _chgsign (double);
_CRTIMP double __cdecl _scalb(double, long);
_CRTIMP double __cdecl _logb(double);
_CRTIMP double __cdecl _nextafter(double, double);
_CRTIMP int __cdecl _finite(double);
_CRTIMP int __cdecl _isnan(double);
_CRTIMP int __cdecl _fpclass(double);

#define _FPCLASS_SNAN 0x0001 /* signaling NaN */
#define _FPCLASS_QNAN 0x0002 /* quiet NaN */
#define _FPCLASS_NINF 0x0004 /* negative infinity */
#define _FPCLASS_NN 0x0008 /* negative normal */
#define _FPCLASS_ND 0x0010 /* negative denormal */
#define _FPCLASS_NZ 0x0020 /* -0 */
#define _FPCLASS_PZ 0x0040 /* +0 */
#define _FPCLASS_PD 0x0080 /* positive denormal */
#define _FPCLASS_PN 0x0100 /* positive normal */
#define _FPCLASS_PINF 0x0200 /* positive infinity */

[mastrboy]

No assembler opimizations, so slower.


I see, so is assembler (sse[x]) optimizations on the roadmap?

[StainlessS]

@Mastrboy,

Not by me, sorry. (would feel like going back to Z80 assembler, I feel sick just thinking about it. M68K spoilt me).

[Wilbert]

double div = sqrt((double)(Pixels * Sx2 - (Sx*Sx))) * sqrt((double)(Pixels * Sy2 - (Sy*Sy)));
if (div==0) {div=.000001;}
return (float)(num/div);


Something like the following should do it


double div;
double div1 = abs(Pixels * Sx2 - Sx*Sx);
double div2 = abs(Pixels * Sy2 - Sy*Sy);
if ((div1<0.0001)||(div2<0.0001))
{
div = .000001;
} else {
div = sqrt(div1) * sqrt(div2);
}
return (float)(num/div);

[StainlessS]

Thankyou Wilbert, very close to what I currently have.
EDIT: Although I forgot about the 'abs'
EDIT: (which should have been 'fabs', did fabs by hand rather
than calling function).
Works fine.

[mastrboy]

@Mastrboy,

Not by me, sorry. (would feel like going back to Z80 assembler, I feel sick just thinking about it. M68K spoilt me).

Lol, M68K isn't that the CPU used in the sega megadrive? what the hell did you code for that? :P

[StainlessS]

MC68K Used in eg Sinclair QL and family, Atari ST and family, Commodore Amiga and family, early Mac's.
Up until joining D9, had not done any coding for about 14/15 years. (EDIT: unless you count a little vb)
EDIT: Sega Megadrive, no idea.

New Version, see first post.

[StainlessS]

Having perused the source, I think I would be happy to remove the beta status of the plugin immediately.

[StainlessS]

SEE Update Version Here:
http://forum.doom9.org/showthread.php?p=1586857#post1586857

Here's a nice little script function written using both GScript and RT_Stats, I suspect that those
two plugins are quite often going to be used together. The function, "QueryBorderCrop()" returns the crop coords
to remove letter boxing or other black bars around the edges of a clip.
The return value is a string and you can 'read' the values in Avisynth by using an Eval on the string.

Heres the script:


Function QueryBorderCrop(clip c,int "Samples",Float "Thresh",bool "Laced",int "XMod",int "YMod",int "WMod",int "HMod",bool "CropMore", \
bool "Relative", String "Prefix",int "RLBT",bool "DEBUG") {
# QueryBorderCrop v1.06, By StainlessS, Requires GScript & RT_Stats v1.01 Plugins.
# Prescan function to get coords for cropping black borders. Planar, YUY2, RGB.
# Returns a string holding crop coords eg "QBCropX=10 QBCropY=8 QBCropW=640 QBCropH=480", can use eg Eval(QueryBorderCrop()) to set values
# into QBCropX, QBCropY,QBCropW,QBCropH variables. v1.05 Added return variable eg QBCropThresh, perhaps useful if auto Thresh used.
# http://forum.doom9.org/showthread.php?p=1584626#post1584626
#
# Samples=32, Number of frames to sample from source clip c
# Thresh= -18.0, Average luma value below or equal to which is considered possible black border, above considered definitely image.
# A zero or negative value is 'AutoThresh', will prescan 'Samples' frames (in suspect border area eg height/100), looking for minimum
# pixel luma value (but ignoring the darkest 1%) and set 'Thresh' to that value plus the negated AutoThresh value,
# ie default sets Thresh to minimum found Y value + 18.0.
# Auto cropping will likely fail on END CREDITS ONLY clips, where on black background, and will probably crop off the sides that are no
# different in average luma to any letterbox borders, if you cannot see the borders, then neither can QueryBorderCrop(), even setting the
# auto Thresh to eg -1.0 or 0.0 is quite likely to fail. (See RLBT edge bitflags).
# Laced=true, set true for Interlaced. (Modifies defaults for YMod,HMod, explicit YMod/HMod will override).
# XMod, Default: Planar 2, YUY2=2, RGB=1
# YMod, Default: Planar=2, YUY2=1, RGB=1: BUT, Doubled if laced=true.
# WMod, Default: Planar=4, YUY2=2, RGB=1
# HMod, Default: Planar=2, YUY2=1, RGB=1: BUT, Doubled if laced=true.
# CropMore=false, False = Leave some black for XMod,YMod,WMod,HMod, true=Crop off all black and a bit of image for XMod,WMod,WMod,HMod.
# Relative=false, False returns Width and Height, true returns Width/Height relative eg QBCropW=-6 QBCropH=-4.
# Prefix="QBCrop", string for returned variable names, only use valid variable name characters eg NO SPACES. Default returns eg "QBCropX".
# RLBT=15=All Borders, Bitflags of edges to crop, 15 ($0F) crops all four. Each letter in the name 'RLBT' represents an edge and bit position
# starting with 'R' in bit 3 representing the value 8 (2^3=8). 'L' = bit 2=4 (2^2=4), 'B' = bit 1=2 (2^1=2), 'T' = bit 0=1 (2^0=1).
# To calculate the RLBT bitflags, for 'R'(Right) add 8, for 'L'(Left) add 4, for 'B'(Bottom) add 2, and for 'T'(Top) add 1.
# Add all of the bit flags together 8+4+2+1 (=15) crops all four edges, 8+4 crops only Right & Left, and 2+1 crops only Bottom & Top.
# DEBUG=False=No Debug. Set True for debugging info, need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
#
# If not cropping enough border, then increase Samples. If cropping too much border, then decrease Thresh.
VERS="QueryBorderCrop v1.06 - 7 Aug 2012"
c
Laced = Default(Laced,True)
XM = IsPlanar() ? 2 : IsYUY2() ? 2 : 1 WM = IsPlanar() ? 4 : IsYUY2() ? 2 : 1
YM = IsPlanar() ? 2 : IsYUY2() ? 1 : 1 YM = (Laced) ? 2 * YM : YM
Samples = Default(Samples,32) Thresh= Float(Default(Thresh,-18.0)) DEBUG=default(DEBUG,False)
XMod = Default(XMod,XM) YMod = Default(YMod,YM)
WMod = Default(WMod,WM) HMod = Default(HMod,YM)
Prefix= Default(Prefix,"QBCrop")
CropMore = Default(CropMore,false) Relative= Default(Relative,false) RLBT=Default(RLBT,15)
w=Width h=Height Frames = Framecount()
Assert((Samples >= 1), "QueryBorderCrop: Requires Samples>=1")
Assert(XMod>0 && YMod>0 && WMod>0 && HMod>0,"QueryBorderCrop: XMod,YMod,WMod,HMod Must be at least 1")
Assert(RLBT>=1 && RLBT<=15,"QueryBorderCrop: RLBT 1->15 Only")
flgR=(RLBT>=8) RLBT=RLBT % 8 flgL=(RLBT>=4) RLBT=RLBT % 4 flgB=(RLBT>=2) RLBT=RLBT % 2 flgT=(RLBT>=1)
Samples = (Samples > Frames)? Frames : Samples
CX=0 CY=0 CW=0 CH=0 Rgt=w/2 Bot=h/2 Lft=Rgt-1 Top=Bot-1
GScript("""
if(DEBUG){RT_Debug(":") RT_Debug(VERS,"- By StainlessS") RT_Debug(":")}
if(Thresh <= 0.0) {
thmin=255.0 prw=(w/100>=1) ? w/100 : 1 prh=(h/100>=1) ? h/100 : 1
for(Samp = 1,Samples) {
Frm = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5)
if(flgT){th=c.RT_YPlaneMin(Frm,h=prh, threshold=1.0) if (thmin>th){thmin=th}}
if(flgB){th=c.RT_YPlaneMin(Frm,y=h-prh,threshold=1.0) if (thmin>th){thmin=th}}
if(flgL){th=c.RT_YPlaneMin(Frm,w=prw, threshold=1.0) if (thmin>th){thmin=th}}
if(flgR){th=c.RT_YPlaneMin(Frm,x=w-prw,threshold=1.0) if (thmin>th){thmin=th}}
if(DEBUG){RT_Debug("QBC_AutoThresh:",String(Samp)+")","["+String(Frm)+"]","YPlaneMin="+String(thmin))}
if(thmin==0.0){Samp = Samples + 1} # No smaller possible, Break.
}
Thresh = thmin + (-Thresh) if(DEBUG){RT_Debug("QBC_AutoThresh:","Thresh="+String(Thresh))}
}
for(Samp = 1,Samples) {
Frm = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5)
if(flgT){cnt=0 For(y=0,Top){th=c.RT_AverageLuma(Frm,y=y,h=1,x=CX,w=CW) if(th>Thresh){if(Cnt>=2){Top=y-2 CY=Top y=h
if(DEBUG){RT_Debug("QBC_Top:",String(Samp)+")","["+String(Frm)+"]","AveY="+String(th),"CY="+String(CY))}
} else {Cnt=Cnt+1}} else {Cnt=0}}}
if(flgB){cnt=0 For(y=h-1,Bot,-1){th=c.RT_AverageLuma(Frm,y=y,h=1,x=CX,w=CW) if(th>Thresh){if(Cnt>=2){Bot=y+2 CH=-(h-1-Bot) y=0
if(DEBUG){RT_Debug("QBC_Bot:",String(Samp)+")","["+String(Frm)+"]","AveY="+String(th),"CH="+String(CH))}
} else {Cnt=Cnt+1}}else {Cnt=0}}}
if(flgL){cnt=0 For(x=0,Lft){th=c.RT_AverageLuma(Frm,x=x,w=1,y=CY,h=CH) if(th>Thresh){if(Cnt>=2){Lft=x-2 CX=Lft x=w
if(DEBUG){RT_Debug("QBC_Lft:",String(Samp)+")","["+String(Frm)+"]","AveY="+String(th),"CX="+String(CX))}
} else {Cnt=Cnt+1}}else {Cnt=0}}}
if(flgR){cnt=0 For(x=w-1,Rgt,-1){th=c.RT_AverageLuma(Frm,x=x,w=1,y=CY,h=CH) if(th>Thresh){if(Cnt>=2){Rgt=x+2 CW=-(w-Rgt-1) x=0
if(DEBUG){RT_Debug("QBC_Rgt:",String(Samp)+")","["+String(Frm)+"]","AveY="+String(th),"CW="+String(CW))}
} else {Cnt=Cnt+1}}else {Cnt=0}}}
# if((!flgT||Top==0) && (!flgB||Lft==0) && (!flgL||Bot==h-1) && (!flgR||Rgt==w-1)) # Ooops v1.05
if((!flgT||Top==0) && (!flgL||Lft==0) && (!flgB||Bot==h-1) && (!flgR||Rgt==w-1))
{Samp = Samples + 1} # Does Not Require Cropping, Break.
}
XMode = (CropMore) ? XMod-1 : 0 YMode = (CropMore) ? YMod-1 : 0
CX = int((CX+XMode)/XMod)*XMod CY = int((CY+YMode)/YMod)*YMod
WW = w - CX HH = h - CY CW = WW + CW CH = HH + CH
WMode = (CropMore)?0:(WMod-1 < WW-CW) ? WMod-1 : WW-CW
HMode = (CropMore)?0:(HMod-1 < HH-CH) ? HMod-1 : HH-CH
CW = int((CW+WMode)/WMod)*WMod CH = int((CH+HMode)/HMod)*HMod
if(Relative) {CW = -(WW - CW) CH = -(HH - CH)}
RS=Prefix+"X="+String(CX) + " " + Prefix+"Y=" + String(CY) + " " + Prefix+"W=" + String(CW) + " " + Prefix+"H=" + String(CH)
RS=RS+ " " + Prefix+"Thresh="+String(Thresh) # Return used Thresh, might be useful to avoid 2nd prescan for AutoThresh.
if(DEBUG){RT_Debug("QBC_Return:",RS)}
""")
Return RS
}


And a test snippit:

import("D:\avs\bank\QueryBorderCrop.avs")
avisource("D:\avs\avi\1.avi")
Addborders(10,8,6,4)

DEBUG=True # Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027

CropString=QueryBorderCrop(debug=DEBUG)

Eval(CropString) # Assign return values to QBCropX QBCropY QBCropW QBCropH and QBCropThresh
Crop(QBCropX,QBCropY,QBCropW,QBCropH)
Subtitle(String(QBCropX),Align=7)
Subtitle(String(QBCropY),Align=8)
Subtitle(String(Width),Align=9)
Subtitle(String(Height),Align=2)
Subtitle(String(CropString),Align=5)
return last


v1.03, Added auto 'Thresh' functionality.
v1.04, Added RLBT edge Bitflags.
v1.05, Now returns used Thresh as eg QBCropThresh. Added Debug mode.
v1.06, EDIT: Fixed silly bug in v1.05, hilighted in source

SEE Update v1.07 Version Here:
http://forum.doom9.org/showthread.php?p=1586857#post1586857

If you want the coords for a resize without any black (instead of cropping of image) then set
XMod, YMod, WMod and HMod to 1 and use returned coords in a resize function. The below linked
function can be called with the crop coords prior to resize, to calculate the resultant Display Aspect Ratio
after resize and signaled to MeGUI so that you do not have to set DAR manually in MeGUI.

Below a post with script function to calc resultant Display Aspect Ratio after crop/resize with crop.
http://forum.doom9.org/showthread.php?p=1576193#post1576193

[StainlessS]

Previous post update for "QueryCrop()"

[Gavino]

avisource("D:\avs\avi\1.avi")
Addborders(10,8,6,4)
CropString=QueryCrop()

Eval(CropString) # Assign return values to CROPX CROPY, CROPW and CROPH
Crop(CropX,CropY,CropW,CropH)
ScriptClip("""
Subtitle(String(CropX),Align=7)
Subtitle(String(CropY),Align=8)
Subtitle(String(CropW),Align=9)
Subtitle(String(CropH),Align=2)
Subtitle(String(CropString),Align=5)
""")
return last
Why do you use ScriptClip() here when CropX, etc are calculated once only and fixed at compile time?

[jmac698]

It would apply if crop changes, but better to pick a few frames at compile time and see what the most frequent answer is and use that.
Some dark scenes could mess it up, also scrolling credits.
This wouldn't work as is on a real letterboxed movie.

[StainlessS]

@Gavino, Thanks. I modded an existing script and guess I did not give that part any thought. Fixed.

@JMac,
If croppings change, then would pick outermost cropping, ie least cropped.
If dark scenes, then would have to be real dark and all of the way though the clip.
You could always play with the Thresh & samples options. If a clip consisted of purely credits then would
fail, but what is the likely hood of that and that would likely defeat any attempt as autocropping.

This wouldn't work as is on a real letterboxed movie.

Care to elaborate.

EDIT: Had previously only tried on example provided, just tried it on NTSC version of Return Of The King, No problem.

EDIT: Tried 6 more PAL DVDs and all cropped correctly without issue, perhaps you can ID a DVD that fails
(I don't have that much time to waste on trying to find one, suggest look for clip at night time in a coal mine with the lights out.).

EDIT: Tried on 6 DVDs trimming out everything but credits. Failed quite badly on all of them, but, I probably could not tell
where left and right borders were with vertically scrolling credits. Advise is, dont use on DVDs that consist solely of credits.

[StainlessS]

New Version RT_Stats, See first post.




Why do you use ScriptClip() here when CropX, etc are calculated once only and fixed at compile time?
Just twigged, I seem to do it all of the time. Go Figure. :o

[StainlessS]

Here An updated QueryBorderCrop(), unfortunately, the site screws up at a simple 16K script, so
I'm splitting it into several posts.

Here Some of the Text I will have to strip out of the AVS file.


Function QueryBorderCrop(clip c,int "Samples",Float "Thresh",bool "Laced",int "XMod",int "YMod",int "WMod",int "HMod", \
bool "Relative", String "Prefix",int "RLBT",bool "DEBUG",float "Ignore",int "Matrix") {
# QueryBorderCrop v1.08, By StainlessS, Requires GScript & RT_Stats v1.01 Plugins.
# http://forum.doom9.org/showthread.php?p=1586858#post1586858
#
# Prescan function to get coords for eg cropping black borders. Planar, YUY2, RGB.
# Simultaneously returns 4 sets of strings holding crop coords, here is one set: "QBCropX=8 QBCropY=8 QBCropW=640 QBCropH=480".
# String sets are Chr(10) separated, the 'exact found' coords set as above, not rounded at all and possibly throwing an error if
# you try to crop, although could use in eg resize.
# Second set using eg "QBCropXL=8", which is CropLess("L" trailer), ie when rounding for Xmod,WMod etc may leave some black border.
# Third set using eg "QBCropXM=8", which is CropMore, ie when rounding for Xmod,WMod etc may crop some image.
# Forth set using eg "QBCropXP=8", which is CropPlus, moves border positions an extra 2 pixels inwards on each edge and then as CropMore.
# The non-exact coords try to center the coords when cropping for larger WMod/HMod's so as to evenly crop from eg both left and right instead of
# just one side. Also returned in the return string is the used Threshold, perhaps set by AutoThresh, as eg "QBCropThresh=32.0"
# You can use eg Eval(QueryBorderCrop()) to set values into QBCropX, QBCropY,QBCropW,QBCropH etc variables.
#
# Args:-
#
# Samples=24, Number of frames to sample from source clip c.
#
# Thresh= -32.0, Average luma value below or equal to which is considered possible black border, above considered possible image
# (if 3 adjacent pixel rows/columns above Thresh, then taken as image).
# A zero or negative value is 'AutoThresh', will prescan 'Samples' frames looking for minimum pixel luma value (but ignoring a percentage
# of the darkest pixels, see Ignore) and set 'Thresh' to that value plus the negated AutoThresh value, ie defaults sets Thresh to minimum
# found Y value + 32.0.
# Auto cropping will likely fail on END CREDITS ONLY clips, where on black background, and will probably crop off the sides that are no
# different in average luma to any letterbox borders, if you cannot see the borders, then neither can QueryBorderCrop(), even setting the
# auto Thresh to eg -1.0 or 0.0 is quite likely to fail. (See RLBT edge bitflags).
# v1.08, If 'samples' < 12 [samples would have been reduced if greater than framecount], and Thresh is not specified, then default
# AutoThresh will be adjusted.
# Thresh = -(((samples - 1) * (31.5/(12-1))) + 0.5) # samples==1 comes out as -0.5 : samples==12 would come out at -32.0
# So if eg single frame, would use an AutoThresh of -0.5
# If RGB and user specified a PC matrix then AutoThresh would also be scaled up a little (by (255.0/(235-16)).
# The adjustment to auto Thresh where samples < 12, is to reduce the possibility of overcropping on eg a dark single 'Samples' clip,
# but could result in not cropping enough border, its a bit of a balancing act really.
# As an observation, most clips have good border recogition within the 1st 2 sampled frames using the default -32 AutoThresh
# although have noticed some dark clips that required ~8 samples (or more) for full recognition @ default Thresh = -32.
# We use a default Samples=24, because we are intrinsically paranoid.
# To speed up, you may want to leave Auto-Thresh alone and reduce samples, but there is danger that it will not detect all borders
# (overcrop).
# NOTE, The plugin AutoCrop() uses a Thresh of 40.0 and samples == 5 by default (No AutoThresh), but the logic is not too dissimilar.
#
# Laced=true, set true for Interlaced. (Modifies defaults for YMod,HMod, explicit YMod/HMod will override).
#
# XMod, Default: Planar 2, YUY2=2, RGB=1
# YMod, Default: Planar=2, YUY2=1, RGB=1: BUT, Doubled if laced=true.
# WMod, Default: Planar=4, YUY2=2, RGB=1
# HMod, Default: Planar=2, YUY2=1, RGB=1: BUT, Doubled if laced=true.
# The above XMod etc set rounding for the non-exact cropping coords.
# The only real reason for default planar WMod=4 is VirtualDubMod dont like it less than that (Vdub latest, OK).
#
# Relative=false, False returns Width and Height, true returns Width/Height relative eg QBCropW=-6 QBCropH=-4.
#
# Prefix="QBCrop", string for returned variable names, only use valid variable name characters eg NO SPACES. Default returns eg "QBCropX".
#
# RLBT=15=All Borders, Bitflags of edges to crop, 15 ($0F) crops all four. Each letter in the name 'RLBT' represents an edge and bit position
# starting with 'R' in bit 3 representing the value 8 (2^3=8). 'L' = bit 2=4 (2^2=4), 'B' = bit 1=2 (2^1=2), 'T' = bit 0=1 (2^0=1).
# To calculate the RLBT bitflags, for 'R'(Right) add 8, for 'L'(Left) add 4, for 'B'(Bottom) add 2, and for 'T'(Top) add 1.
# Add all of the bit flags together 8+4+2+1 (=15) crops all four edges, 8+4 crops only Right & Left, and 2+1 crops only Bottom & Top.
#
# DEBUG=False=No Debug. Set True for debugging info, need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
#
# Ignore=0.2, percentage of darkest pixels to ignore during Full image AutoThresh scan.
#
# Matrix, For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
# Default for RGB is 2(PC601) if width <= 720 else 1(PC709) : YUV not used
# The defaults are for PC601 & PC709 range 0-255.
# So as to not require different Thresh for RGB, if clip c is RGB and matrix is PC range, then Thresh will be scaled
# to RGB full range ie Thresh = Thresh * (255.0/(235.0-16.0))
#
# If cropping too much border, then increase Samples or reduce Thresh
# If not cropping enough border then border is being seen as image, increase Thresh.



The Bare Script Follows In next post

[StainlessS]

QueryBorderCrop(), See Also previous and following posts.


Function QueryBorderCrop(clip c,int "Samples",Float "Thresh",bool "Laced",int "XMod",int "YMod",int "WMod",int "HMod", \
bool "Relative", String "Prefix",int "RLBT",bool "DEBUG",float "Ignore",int "Matrix") {
VERS="QueryBorderCrop v1.08 - 29 Aug 2012"
c
GScript("""
DEBUG=default(DEBUG,False)
Laced = Default(Laced,True)
XM=IsPlanar()?2:IsYUY2()?2:1 WM=IsPlanar()?4:IsYUY2()?2:1 YM=IsPlanar()?2:IsYUY2()?1:1 YM=(Laced)?2*YM:YM
XMod = int(Default(XMod,XM)) YMod = int(Default(YMod,YM)) WMod = int(Default(WMod,WM)) HMod = int(Default(HMod,YM))
Prefix= Default(Prefix,"QBCrop") Relative= Default(Relative,false) RLBT=int(Default(RLBT,15)) w=Width h=Height Frames = Framecount()
Ignore=Float(Default(Ignore,0.2))
Samples = int(Default(Samples,24))
Samples = (Samples > Frames)? Frames : Samples
Mat = (width <= 720) ? 2 : 3 # Default to PC levels
Matrix = Default(Matrix,Mat)
Assert((Samples >= 1), "QueryBorderCrop: Requires Samples>=1")
Assert(XMod>0 && YMod>0 && WMod>0 && HMod>0,"QueryBorderCrop: XMod,YMod,WMod,HMod Must be at least 1")
Assert(WMod%XMod==0,"QueryBorderCrop: WMod must be a multiple of XMod")
Assert(HMod%YMod==0,"QueryBorderCrop: HMod must be a multiple of YMod")
Assert(RLBT>=1 && RLBT<=15,"QueryBorderCrop: RLBT 1->15 Only")
flgs=RLBT flgR=(flgs>=8) flgs=flgs % 8 flgL=(flgs>=4) flgs=flgs % 4 flgB=(flgs>=2) flgs=flgs % 2 flgT=(flgs>=1)
CX=0 CY=0 CW=0 CH=0 Rgt=w/2 Bot=h/2 Lft=Rgt-1 Top=Bot-1
DBGS1="" DBGS2=""
if(!Defined(Thresh)) {
if(samples < 12) {
Thresh = -(((samples - 1) * (31.5/(12-1))) + 0.5) # samples==12 would come out at -32
DBGS1="QBC: Low Samples count, AutoThresh reduced from default -32 to "+String(Thresh)
}
else {Thresh = -32.0}
}
if(isRGB() && Matrix >= 2) {
Thresh=Thresh *(255.0/(235.0-16.0)) # Scale up a bit for PC levels
DBGS2="QBC: RGB PC Levels Thresh Auto Scaled to "+String(Thresh)
}
if(DEBUG){RT_Debug("QBC:") RT_Debug("QBC:",VERS,"- By StainlessS") RT_Debug("QBC:")
RT_Debug(DBGS1)
RT_Debug(DBGS2)
RT_Debug("QBC:","Samples="+String(Samples),"Thresh="+String(Thresh,"%.2f"),"Laced="+String(Laced),"Matrix="+String(Matrix))
RT_Debug("QBC:","XMod="+String(XMod),"YMod="+String(YMod),"WMod="+String(WMod),"HMod="+String(HMod))
RT_Debug("QBC:","Relative="+String(Relative),"Prefix="+String(Prefix),"RLBT="+String(RLBT),"Ignore="+String(Ignore,"%.2f"))
RT_Debug("QBC:")}
if(Thresh < 0.0) { thmin = 255.0
if(IsPlanar() && Width < 4096 && Height < 4096) { if(DEBUG){RT_Debug("QBC:","AutoThresh Using Avisynth Planar Luma Sampling")}
for(Samp = 1,Samples) { current_frame = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5)
th=YPlaneMin(threshold=Ignore) if (thmin>th){thmin=th}
if(DEBUG){
RT_Debug("QBC:","AutoThresh",String(Samp,"%-2.0f")+")","["+String(current_frame,"%-5.0f")+ "]","YPlaneMin="+String(thmin))}
if(thmin==0.0){Samp = Samples + 1} # No smaller possible, Break.
}
} else { if(DEBUG) {RT_Debug("QBC:","AutoThresh Using RT_Stats Luma Sampling")}
for(Samp = 1,Samples) { Frm = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5)
th=RT_YPlaneMin(Frm,threshold=Ignore,matrix=Matrix) if (thmin>th){thmin=th}
if(DEBUG){ RT_Debug("QBC:","AutoThresh",String(Samp,"%-2.0f")+")","["+String(frm,"%-5.0f")+"]","YPlaneMin="+String(thmin))}
if(thmin==0.0){Samp = Samples + 1} # No smaller possible, Break.
}
}
Thresh = thmin + (-Thresh) if(DEBUG) {RT_Debug("QBC:","AutoThresh","Thresh="+String(Thresh,"%.2f")) RT_Debug("QBC:")}
}
GotF=-1 GotN=0 # Track Number of frames where coords 1st found
for(Samp = 1,Samples) {
Frm = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5) # Sample frame
th2=0.0 th1=0.0 th=0.0 # Init, Previous luma values (pass the parcel)
# Only Y-Top Commented
if(flgT && Top>=2) { # Required and some more to do ?
cnt=0 # Init, got none yet
For(y=0,Top) { # Searching from outside inwards
th2=th1 th1=th th=RT_AverageLuma(Frm,y=y,h=1,x=CX,w=CW,matrix=Matrix) # Pass the parcel
if(th>Thresh) { # Possible image
if(Cnt>=2) { # Is Image, (found 3 adjacent scanlines all passing Thresh, No Incr this time)
CY=y-2 # The outer scanline that passed first (th2 was th at that time).
Top=(y<=2)?0:y-1 # If no top border at all, no more search: else, next time search max scanline y-1.
if(Frm<>GotF) {
GotF=Frm # Rememeber Last active Frame where image found
GotN=GotN+1 # Incr number of active frames
}
y=h # Force loop exit. (No more to do this frame, found 1st match scanning inwards)
if(DEBUG) {
RT_Debug("QBC:","Top",String(Samp)+")","["+String(Frm,"%-5.0f")+"]","AveY="+String(th2,"%3.2f"),"CY="+String(CY))
}
} else {
Cnt=Cnt+1 # Incr count, scanlines passed OK as possible image
}
} else {
Cnt=0 # Failed, maybe border.
}
}
}
if(flgB && Bot<=(h-1)-2){
cnt=0 For(y=h-1,Bot,-1){th2=th1 th1=th th=RT_AverageLuma(Frm,y=y,h=1,x=CX,w=CW,matrix=Matrix)
if(th>Thresh){if(Cnt>=2){CH= -((h-1)-(y+2)) Bot=((h-1)-y<=2)?h-1: y+1 if(Frm<>GotF) {GotF=Frm GotN=GotN+1} y=-1
if(DEBUG){RT_Debug("QBC:","Bot",String(Samp)+")","["+String(Frm,"%-5.0f")+"]","AveY="+String(th2,"%3.2f"),"CH="+String(CH))}
} else {Cnt=Cnt+1}}else {Cnt=0}}}
if(flgL && Lft>=2){
cnt=0 For(x=0,Lft){th2=th1 th1=th th=RT_AverageLuma(Frm,x=x,w=1,y=CY,h=CH,matrix=Matrix)
if(th>Thresh){if(Cnt>=2){CX=x-2 Lft=(x<=2)?0:x-1 if(Frm<>GotF) {GotF=Frm GotN=GotN+1} x=w
if(DEBUG){RT_Debug("QBC:","Lft",String(Samp)+")","["+String(Frm,"%-5.0f")+"]","AveY="+String(th2,"%3.2f"),"CX="+String(CX))}
} else {Cnt=Cnt+1}} else {Cnt=0}}}
if(flgR && Rgt<=(w-1)-2){
cnt=0 For(x=w-1,Rgt,-1){th2=th1 th1=th th=RT_AverageLuma(Frm,x=x,w=1,y=CY,h=CH,matrix=Matrix)
if(th>Thresh){if(Cnt>=2){CW= -((w-1)-(x+2)) Rgt=((w-1)-x<=2)?w-1:x+1 if(Frm<>GotF) {GotF=Frm GotN=GotN+1} x=-1
if(DEBUG){RT_Debug("QBC:","Rgt",String(Samp)+")","["+String(Frm,"%-5.0f")+"]","AveY="+String(th2,"%3.2f"),"CW="+String(CW))}
} else {Cnt=Cnt+1}}else {Cnt=0}}}
if((!flgT||Top==0) && (!flgL||Lft==0) && (!flgB||Bot==h-1) && (!flgR||Rgt==w-1))
{Samp = Samples + 1} # No further cropping possible, Break.
}
if(DEBUG){ RT_Debug("QBC:","Active Frames (where image coords first found) = "+String(GotN))
RT_Debug("QBC:","Sampled Borders:","X="+String(CX),"Y="+String(CY), \
"W="+String(CW)+"("+String(w-CX+CW)+")","H="+String(CH)+"("+String(h-CY+CH)+")")
RT_Debug("QBC:")
}

# Crop Exact # Make Exact found Coords: Default, QBCropX, QBCropY, QBCropW, QBCropH
CX0 = CX # Just make dupes
CY0 = CY
CW0 = w - CX + CW # Dont change CW and CH as needed for CropMore/CropLess/CropPlus
CH0 = h - CY + CH
if(DEBUG){RT_Debug("QBC:","CropExact:","X ="+String(CX0,"%2.0f"),"Y ="+String(CY0,"%2.0f"), \
"W ="+String(CW0,"%3.0f")+"("+String(CX0+CW0-w)+")","H ="+String(CH0,"%3.0f")+"("+String(CY0+CH0-h)+")")}

# CropLess X,W # Make CropLess Coords: Default, QBCropXL, QBCropYL, QBCropWL, QBCropHL
CX1 = int(CX / XMod)
CW1_Lim = int(w / XMod) - CX1
CW1 = int((w - (CX1*XMod) + CW + WMod - 1) / XMod)
CW1 = (CW1 > CW1_Lim) ? CW1_Lim : CW1
CW1_D = int(WMod/XMod)
CW1_M = int((CW1 % CW1_D) / 2)
CW1_Off = (CX1 < CW1_M) ? CX1 : CW1_M
CX1 = CX1 - CW1_Off
CW1 = CW1 + CW1_Off
CX1 = CX1 * XMod
CW1 = int(CW1 / CW1_D) * CW1_D * XMod
# CropLess Y,H
CY1 = int(CY / YMod)
CH1_Lim = int(h / YMod) - CY1
CH1 = int((h - (CY1*YMod) + CH + HMod - 1) / YMod)
CH1 = (CH1 > CH1_Lim) ? CH1_Lim : CH1
CH1_D = int(HMod/YMod)
CH1_M = int((CH1 % CH1_D) / 2)
CH1_Off = (CY1 < CH1_M) ? CY1 : CH1_M
CY1 = CY1 - CH1_Off
CH1 = CH1 + CH1_Off
CY1 = CY1 * YMod
CH1 = int(CH1 / CH1_D) * CH1_D * YMod
if(DEBUG) {
if(CW1_Off<>0) {RT_Debug("QBC:","CropLess: "," XL Centered - "+String(CW1_Off))}
if(CH1_Off<>0) {RT_Debug("QBC:","CropLess: "," YL Centered - "+String(CH1_Off))}
RT_Debug("QBC:","CropLess: "," XL="+String(CX1,"%2.0f"),"YL="+String(CY1,"%2.0f"), \
"WL="+String(CW1,"%3.0f")+"("+String(CX1+CW1-w)+")","HL="+String(CH1,"%3.0f")+"("+String(CY1+CH1-h)+")")
}
# CropMore X,W # Make CropMore Coords: Default, QBCropXM, QBCropYM, QBCropWM, QBCropHM
CX2 = int((CX+XMod-1)/XMod)*XMod # Round up CX2
CW2_Tmp = int(((w - CX2 + CW)/XMod))*XMod # wid Mod XMod
CW2 = int(CW2_Tmp/WMod)*WMod # wid Mod WMod
CW2_Dif = int((CW2_Tmp - CW2) / XMod) # Diff due to WMod in XMod steps
CW2_Off = (int(CW2_Dif/2)) * XMod # Half difference to try and center
CX2 = CX2 + CW2_Off # Center, try to share out cropping due to big WMod
# CropMore Y,H
CY2 = int((CY+YMod-1)/YMod)*YMod # Round up CY2
CH2_Tmp = int(((h - CY2 + CH)/YMod))*YMod # height Mod YMod
CH2 = int(CH2_Tmp/HMod)*HMod # height Mod HMod
CH2_Dif = int((CH2_Tmp - CH2) / YMod) # Diff due to HMod in YMod steps
CH2_Off = (int(CH2_Dif/2)) * YMod # Half difference to try and center
CY2 = CY2 + CH2_Off # Center, try to share out cropping due to big HMod
if(DEBUG) {
if(CW2_Off<>0) {RT_Debug("QBC:","CropMore: "," XM Centered + "+String(CW2_Off))}
if(CH2_Off<>0) {RT_Debug("QBC:","CropMore: "," YM Centered + "+String(CH2_Off))}
RT_Debug("QBC:","CropMore: "," XM="+String(CX2,"%2.0f"),"YM="+String(CY2,"%2.0f"), \
"WM="+String(CW2,"%3.0f")+"("+String(CX2+CW2-w)+")","HM="+String(CH2,"%3.0f")+"("+String(CY2+CH2-h)+")")
}
# CropPlus X,W # Make CropPlus Coords: Default, QBCropXP, QBCropYP, QBCropWP, QBCropHP
CXP = CX+2 # CropPlus Over Cropping, dont touch orig vars
CYP = CY+2 # Cropping extra 2 pixels from each edge
CWP = CW-2 # After that, as CropMore
CHP = CH-2
CX3 = int((CXP+XMod-1)/XMod)*XMod # Round up CX3
CW3_Tmp = int(((w - CX3 + CWP)/XMod))*XMod # wid Mod XMod
CW3 = int(CW3_Tmp/WMod)*WMod # wid Mod WMod
CW3_Dif = int((CW3_Tmp - CW3) / XMod) # Diff due to WMod in XMod steps
CW3_Off = (int(CW3_Dif/2)) * XMod # Half difference to try and center
CX3 = CX3 + CW3_Off # Center, try to share out cropping due to big WMod
# CropPlus Y,H
CY3 = int((CYP+YMod-1)/YMod)*YMod # Round up CY3
CH3_Tmp = int(((h - CY3 + CHP)/YMod))*YMod # height Mod YMod
CH3 = int(CH3_Tmp/HMod)*HMod # height Mod HMod
CH3_Dif = int((CH3_Tmp - CH3) / YMod) # Diff due to HMod in YMod steps
CH3_Off = (int(CH3_Dif/2)) * YMod # Half difference to try and center
CY3 = CY3 + CH3_Off # Center, try to share out cropping due to big HMod
if(DEBUG) {
if(CW3_Off<>0) {RT_Debug("QBC:","CropPlus: "," XP Centered + "+String(CW3_Off))}
if(CH3_Off<>0) {RT_Debug("QBC:","CropPlus: "," YP Centered + "+String(CH3_Off))}
RT_Debug("QBC:","CropPlus: "," XP="+String(CX3,"%2.0f"),"YP="+String(CY3,"%2.0f"), \
"WP="+String(CW3,"%3.0f")+"("+String(CX3+CW3-w)+")","HP="+String(CH3,"%3.0f")+"("+String(CY3+CH3-h)+")")
}

if(Relative) { CW0 = CX0 + CW0 - w CH0 = CY0 + CH0 - h CW1 = CX1 + CW1 - w CH1 = CY1 + CH1 - h
CW2 = CX2 + CW2 - w CH2 = CY2 + CH2 - h CW3 = CX3 + CW3 - w CH3 = CY3 + CH3 - h}
RS0=Prefix+"X=" +String(CX0)+" "+Prefix+"Y=" +String(CY0)+" "+Prefix+"W=" +String(CW0)+" "+Prefix+"H=" +String(CH0)
RS1=Prefix+"XL="+String(CX1)+" "+Prefix+"YL="+String(CY1)+" "+Prefix+"WL="+String(CW1)+" "+Prefix+"HL="+String(CH1)
RS2=Prefix+"XM="+String(CX2)+" "+Prefix+"YM="+String(CY2)+" "+Prefix+"WM="+String(CW2)+" "+Prefix+"HM="+String(CH2)
RS3=Prefix+"XP="+String(CX3)+" "+Prefix+"YP="+String(CY3)+" "+Prefix+"WP="+String(CW3)+" "+Prefix+"HP="+String(CH3)
RST=Prefix+"Thresh="+String(Thresh) # Return used Thresh, might be useful to avoid 2nd prescan for AutoThresh.
if(DEBUG){RT_Debug("QBC:") RT_Debug("QBC:","Return:",RS0) RT_Debug("QBC:","Return:",RS1) RT_Debug("QBC:","Return:",RS2)
RT_Debug("QBC:","Return:",RS3) RT_Debug("QBC:","Return:",RST)}
""")
Return RS0+Chr(10)+RS1+Chr(10)+RS2+Chr(10)+RS3+Chr(10)+RST+Chr(10)
}

[StainlessS]

QueryLumaMinMax(), And a couple of other script functions. See Previous and following posts.



Function QueryLumaMinMax(clip c,int "Samples",float "Ignore",String "Prefix",bool "DEBUG",int "X",int "Y",int "W",int "H",int "Matrix") {
# QueryLumaMinMax v1.01, By StainlessS, Requires GScript & RT_Stats v1.01 Plugins.
# Prescan function to get Luma Min,Max. Planar, YUY2, RGB.
# Returns luma min/max as string Default eg "QLMMMin=25 QLMMMax=244", use eg "Eval(QueryLumaMinMax())" to set variables for
# use in Avisynth.
# http://forum.doom9.org/showthread.php?p=1586859#post1586859
#
# Samples=24=frames to sample.
#
# Ignore=0.2=Percentage of extreme pixels to ignore.
#
# Prefix="QLMM"=Prefix for return values.
#
# X=Y=W=H=0=Full screen, as crop.
#
# DEBUG=false= dont show. True=debug info. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
#
# Matrix, For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
# Default for RGB is 2(PC601) if width <= 720 else 3(PC709) : YUV not used
# For RGB, it probably does not make sense to use anything other than PC levels, unless studio RGB.
#
# Usage:
#
# # Auto levels using Levels() filter.
# AUTOLEVEL_STRENGTH = 0.5 # 0.0 -> 1.0
# Eval(QueryLumaMinMax()) # using default values, sets QLMMMin and QLMMMax.
# if(IsRGB()) { # Requires GScript
# CSMin = 0
# CSMax = 255
# } else {
# CSMin = 16
# CSMax = 235
# }
# ALMin = Int(CSMin - ((CSMin - QLMMMin) * AUTOLEVEL_STRENGTH) + 0.5) # Round Up
# ALMax = Int(CSMax - ((CSMax - QLMMMax) * AUTOLEVEL_STRENGTH)) # Round down
# Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring
#
VERS="QueryLumaMinMax v1.01 - 29 Aug 2012"
c
Samples = Default(Samples,24) Ignore=Float(Default(Ignore,0.2)) Prefix = Default(Prefix,"QLMM") DEBUG=default(DEBUG,False)
X=int(Default(X,0)) Y=int(Default(Y,0)) W=int(Default(W,0)) H=int(Default(H,0))
W = (W<=0) ? Width - X + W : W H = (H<=0) ? Height - Y + H : H
Mat = (width <= 720) ? 2 : 3 # Default, PC601 or PC709 for RGB
Matrix = Default(Matrix,Mat)
GScript("""
Assert((Samples >= 1), "QueryLumaMinMax: Requires Samples>=1")
Frames=FrameCount mn=255.0 mx=0.0 Samples = (Samples > Frames)? Frames : Samples
if(DEBUG){RT_Debug("QLMM:") RT_Debug("QLMM:",VERS,"- By StainlessS") RT_Debug("QLMM:")
RT_Debug("QLMM:","Samples="+String(Samples),"Ignore="+String(Ignore,"%.2f"),"Prefix="+String(Prefix))
RT_Debug("QLMM:","X="+String(X),"Y="+String(Y),"W="+String(W),"H="+String(H),"Matrix="+String(Matrix))
RT_Debug("QLMM:")}
if(IsPlanar==True && X==0 && Y==0 && W==Width && H==Height && Width < 4096 && Height < 4096) {
if(DEBUG){RT_Debug("QLMM:","Using Avisynth Planar Full Frame Luma Sampling")}
for(Samp = 1,Samples) {
current_frame = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5)
Mn_Tmp = YPlaneMin(threshold=ignore) Mx_Tmp = YPlaneMax(threshold=ignore)
if(DEBUG){RT_Debug("QLMM:","LumaRange:",String(Samp,"%-2.0f")+")","["+String(current_frame,"%-5.0f")+"]", \
"LumaMin = "+String(Mn_Tmp),"LumaMax = "+String(Mx_Tmp))}
mn=(Mn_Tmp<mn)?Mn_Tmp:mn mx=(Mx_Tmp>mx)?Mx_Tmp:mx}
} else {if(DEBUG){RT_Debug("QLMM:","Using RT_Stats Luma Sampling")}
for(Samp = 1,Samples) {
n = int(Samp * ((Frames - 1.0) / (Samples+1)) + 0.5)
Mn_Tmp = RT_YPlaneMin(n,x=X,y=Y,w=W,h=H,threshold=ignore,matrix=Matrix)
Mx_Tmp = RT_YPlaneMax(n,x=X,y=Y,w=W,h=H,threshold=ignore,matrix=Matrix)
if(DEBUG){RT_Debug("QLMM:","LumaRange:",String(Samp,"%-2.0f")+")","["+String(n,"%-5.0f")+"]", \
"LumaMin = "+String(Mn_Tmp),"LumaMax = "+String(Mx_Tmp))}
mn=(Mn_Tmp<mn)?Mn_Tmp:mn mx=(Mx_Tmp>mx)?Mx_Tmp:mx}
}
RS=Prefix+"Min="+String(Mn) + " " + Prefix+"Max=" + String(mx)
if(DEBUG){RT_Debug("QLMM:") RT_Debug("QLMM:","Return:",RS)}
""")
Return RS
}
#--------------

# Get File extension from filename, use to eg select Source filter for audio file.
Function GetFileExtension(String fn) {len = FindStr(RevStr(fn),".") - 1 Return Len>=1 ? RightStr(fn,Len) : ""}

Function GetCropDAR(clip c,float DAR,float "X",float "Y",float "W",float "H") {
# Call prior to Crop/Resize with (possibly fractional) cropping to calc resultant DAR, X,Y,W,H are cropping coords
# DAR = FAR * SAR ::: FAR = DAR / SAR ::: SAR = DAR / FAR
#
X=Float(Default(X,0.0)) Y=Float(Default(Y,0.0)) W=Float(Default(W,0.0)) H=Float(Default(H,0.0))
W=W<=0.0?c.width+W-X:W H=H<=0.0?c.height+H-Y:H
# Irrespective of what various resizers in various Avisynth versions silently correct, we dont allow eg -ve X
Assert(X>=0.0&&X < c.width, "GetCropDAR: Invalid X("+String(X)+")")
Assert(Y>=0.0&&Y < c.height,"GetCropDAR: Invalid Y("+String(Y)+")")
Assert(W> 0.0&&X+W<=c.width, "GetCropDAR: Invalid W("+String(W)+")")
Assert(H> 0.0&&Y+H<=c.height,"GetCropDAR: Invalid H("+String(H)+")")
Return c.GetSAR(DAR) * W / H
}

#--------------

# From MeGUI Wiki:
Function GetDAR(clip c, float SAR) { return Float(c.width) * SAR / Float(c.height)} # Gets the DAR from the SAR
Function GetSAR(clip c, float DAR) { return DAR * Float(c.height) / Float(c.width) } # Gets the SAR from the DAR
Function SignalDAR(float DAR){global MeGUI_darx=Round(1000*DAR) global MeGUI_dary=1000} # Signal DAR for MEGUI (Name change from SetDar)

[StainlessS]

A Little Demo script for QueryBorderCrop() and QueryLumaMinMax (), See previous few posts.


Import ("D:\avs\QueryBorderCrop.avs")
Import ("D:\avs\QueryLumaMinMax.avs")
GSCript("""
####################################
DAR = 16.0/9.0 # Input Full Frame Display Aspect Ratio
SIZE_X = 640 # Output X dim
SIZE_Y = 480 # Output Y dim
DEBUG = True # Debug ON
TFF = True
AUTOLEVEL_STRENGTH = 0.5 # Strength of auto leveling, 0.0 -> 1.0
####################################
VideoFileName =LCase("D:\TEST_VOBS\1.d2v")
AudioFileName =LCase("D:\TEST_VOBS\1 T80 3_2ch 448Kbps DELAY 0ms.ac3")
AudioDelay =Value(LCase("0.000"))

###############
MPEG2Source(VideoFileName,cpu2="xxxxxx")
AudioExt=GetFileExtension(AudioFileName)

Audio= (AudioExt=="ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
\ : (AudioExt=="mpa"||AudioExt=="mp1"||AudioExt=="mp2"||AudioExt=="mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
\ : (AudioExt=="wav") ? RaWavSource(AudioFileName)
\ : 0
(!isInt(Audio))?AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0):NOP

(AudioRate() <> 44100 && AudioRate() <> 48000)?ResampleAudio(44100):NOP

(TFF)?AssumeTFF():AssumeBFF

Eval(QueryBorderCrop(laced=false,Debug=DEBUG)) # Get Return string and initialize QBCropX type variables in Avisynth
PreCrop=Crop(QBCropXP,QBCropYP,QBCropWP,QBCropHP) # Get OverCropped coord clip using CropPlus Coords ('P' trailer character)

# Get Auto Levels Min/Max from OVERCROPPED Clip (Ignore black borders)
Eval(QueryLumaMinMax(PreCrop,Debug=DEBUG))

if(IsYUV()) {
CSMin = 16
CSMax = 235
} else {
CSMin = 0
CSMax = 255
}
ALMin = Int(CSMin - ((CSMin - QLMMMin) * AUTOLEVEL_STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * AUTOLEVEL_STRENGTH)) # Round down
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # AutoLevel, DONT USE Coring


if(DEBUG) {
RT_Debug("AutoLevel: Levels("+String(ALMin)+",1.0,"+String(ALMax)+","+String(CSMin)+","+String(CSMax)+")")
}
if(QBCropXM<>0||QBCropYM<>0||QBCropWM<>SIZE_X||QBCropHM<>SIZE_Y) {
DAR=GetCropDAR(DAR,QBCropXM,QBCropYM,QBCropWM,QBCropHM) # Calc New DAR after Resize with CropMore ('M' Trailer Char)
Spline36Resize(SIZE_X,SIZE_Y,QBCropXM,QBCropYM,QBCropWM,QBCropHM)
}

if(DEBUG){RT_Debug("SIGNALDAR:","MeGUI DAR="+String(DAR))}
SignalDAR(DAR) # Tell MeGUI the new Resized/Cropped Display Aspect Ratio
return last
""")



And the RT_Stats, RT_Debug Output


00000005 10:53:34 RT_Debug: QBC:
00000006 10:53:34 RT_Debug: QBC: QueryBorderCrop v1.08 - 29 Aug 2012 - By StainlessS
00000007 10:53:34 RT_Debug: QBC:
00000008 10:53:34 RT_Debug: QBC: Samples=24 Thresh=-32.00 Laced=false Matrix=2
00000009 10:53:34 RT_Debug: QBC: XMod=2 YMod=2 WMod=4 HMod=2
00000010 10:53:34 RT_Debug: QBC: Relative=false Prefix=QBCrop RLBT=15 Ignore=0.20
00000011 10:53:34 RT_Debug: QBC:
00000012 10:53:34 RT_Debug: QBC: AutoThresh Using Avisynth Planar Luma Sampling
00000013 10:53:34 RT_Debug: QBC: AutoThresh 1 ) [715 ] YPlaneMin=16
00000014 10:53:35 RT_Debug: QBC: AutoThresh 2 ) [1430 ] YPlaneMin=16
00000015 10:53:35 RT_Debug: QBC: AutoThresh 3 ) [2145 ] YPlaneMin=16
00000016 10:53:35 RT_Debug: QBC: AutoThresh 4 ) [2860 ] YPlaneMin=16
00000017 10:53:35 RT_Debug: QBC: AutoThresh 5 ) [3575 ] YPlaneMin=16
00000018 10:53:36 RT_Debug: QBC: AutoThresh 6 ) [4290 ] YPlaneMin=16
00000019 10:53:36 RT_Debug: QBC: AutoThresh 7 ) [5004 ] YPlaneMin=16
00000020 10:53:36 RT_Debug: QBC: AutoThresh 8 ) [5719 ] YPlaneMin=16
00000021 10:53:36 RT_Debug: QBC: AutoThresh 9 ) [6434 ] YPlaneMin=16
00000022 10:53:36 RT_Debug: QBC: AutoThresh 10) [7149 ] YPlaneMin=16
00000023 10:53:37 RT_Debug: QBC: AutoThresh 11) [7864 ] YPlaneMin=16
00000024 10:53:37 RT_Debug: QBC: AutoThresh 12) [8579 ] YPlaneMin=16
00000025 10:53:37 RT_Debug: QBC: AutoThresh 13) [9294 ] YPlaneMin=16
00000026 10:53:37 RT_Debug: QBC: AutoThresh 14) [10009] YPlaneMin=16
00000027 10:53:37 RT_Debug: QBC: AutoThresh 15) [10724] YPlaneMin=16
00000028 10:53:38 RT_Debug: QBC: AutoThresh 16) [11439] YPlaneMin=16
00000029 10:53:38 RT_Debug: QBC: AutoThresh 17) [12154] YPlaneMin=16
00000030 10:53:38 RT_Debug: QBC: AutoThresh 18) [12869] YPlaneMin=16
00000031 10:53:38 RT_Debug: QBC: AutoThresh 19) [13583] YPlaneMin=16
00000032 10:53:39 RT_Debug: QBC: AutoThresh 20) [14298] YPlaneMin=16
00000033 10:53:39 RT_Debug: QBC: AutoThresh 21) [15013] YPlaneMin=16
00000034 10:53:39 RT_Debug: QBC: AutoThresh 22) [15728] YPlaneMin=16
00000035 10:53:39 RT_Debug: QBC: AutoThresh 23) [16443] YPlaneMin=16
00000036 10:53:40 RT_Debug: QBC: AutoThresh 24) [17158] YPlaneMin=16
00000037 10:53:40 RT_Debug: QBC: AutoThresh Thresh=48.00
00000038 10:53:40 RT_Debug: QBC:
00000039 10:53:44 RT_Debug: QBC: Top 2) [1430 ] AveY=65.15 CY=75
00000040 10:53:44 RT_Debug: QBC: Bot 2) [1430 ] AveY=61.18 CH=-76
00000041 10:53:44 RT_Debug: QBC: Lft 2) [1430 ] AveY=48.26 CX=46
00000042 10:53:44 RT_Debug: QBC: Rgt 2) [1430 ] AveY=48.14 CW=-77
00000043 10:53:45 RT_Debug: QBC: Lft 3) [2145 ] AveY=120.76 CX=0
00000044 10:53:45 RT_Debug: QBC: Rgt 3) [2145 ] AveY=106.21 CW=0
00000045 10:53:58 RT_Debug: QBC: Active Frames (where image coords first found) = 2
00000046 10:53:58 RT_Debug: QBC: Sampled Borders: X=0 Y=75 W=0(720) H=-76(425)
00000047 10:53:58 RT_Debug: QBC:
00000048 10:53:58 RT_Debug: QBC: CropExact: X = 0 Y =75 W =720(0) H =425(-76)
00000049 10:53:58 RT_Debug: QBC: CropLess: XL= 0 YL=74 WL=720(0) HL=426(-76)
00000050 10:53:58 RT_Debug: QBC: CropMore: XM= 0 YM=76 WM=720(0) HM=424(-76)
00000051 10:53:58 RT_Debug: QBC: CropPlus: XP= 2 YP=78 WP=716(-2) HP=420(-78)
00000052 10:53:58 RT_Debug: QBC:
00000053 10:53:58 RT_Debug: QBC: Return: QBCropX=0 QBCropY=75 QBCropW=720 QBCropH=425
00000054 10:53:58 RT_Debug: QBC: Return: QBCropXL=0 QBCropYL=74 QBCropWL=720 QBCropHL=426
00000055 10:53:58 RT_Debug: QBC: Return: QBCropXM=0 QBCropYM=76 QBCropWM=720 QBCropHM=424
00000056 10:53:58 RT_Debug: QBC: Return: QBCropXP=2 QBCropYP=78 QBCropWP=716 QBCropHP=420
00000057 10:53:58 RT_Debug: QBC: Return: QBCropThresh=48.000000
00000058 10:53:58 RT_Debug: QLMM:
00000059 10:53:58 RT_Debug: QLMM: QueryLumaMinMax v1.01 - 29 Aug 2012 - By StainlessS
00000060 10:53:58 RT_Debug: QLMM:
00000061 10:53:58 RT_Debug: QLMM: Samples=24 Ignore=0.20 Prefix=QLMM
00000062 10:53:58 RT_Debug: QLMM: X=0 Y=0 W=716 H=420 Matrix=2
00000063 10:53:58 RT_Debug: QLMM:
00000064 10:53:58 RT_Debug: QLMM: Using Avisynth Planar Full Frame Luma Sampling
00000065 10:53:58 RT_Debug: QLMM: LumaRange: 1 ) [715 ] LumaMin = 16 LumaMax = 23
00000066 10:53:58 RT_Debug: QLMM: LumaRange: 2 ) [1430 ] LumaMin = 18 LumaMax = 163
00000067 10:53:58 RT_Debug: QLMM: LumaRange: 3 ) [2145 ] LumaMin = 83 LumaMax = 199
00000068 10:53:58 RT_Debug: QLMM: LumaRange: 4 ) [2860 ] LumaMin = 21 LumaMax = 103
00000069 10:53:58 RT_Debug: QLMM: LumaRange: 5 ) [3575 ] LumaMin = 25 LumaMax = 159
00000070 10:53:58 RT_Debug: QLMM: LumaRange: 6 ) [4290 ] LumaMin = 17 LumaMax = 164
00000071 10:53:58 RT_Debug: QLMM: LumaRange: 7 ) [5004 ] LumaMin = 19 LumaMax = 170
00000072 10:53:58 RT_Debug: QLMM: LumaRange: 8 ) [5719 ] LumaMin = 17 LumaMax = 145
00000073 10:53:58 RT_Debug: QLMM: LumaRange: 9 ) [6434 ] LumaMin = 17 LumaMax = 159
00000074 10:53:58 RT_Debug: QLMM: LumaRange: 10) [7149 ] LumaMin = 16 LumaMax = 176
00000075 10:53:58 RT_Debug: QLMM: LumaRange: 11) [7864 ] LumaMin = 16 LumaMax = 191
00000076 10:53:58 RT_Debug: QLMM: LumaRange: 12) [8579 ] LumaMin = 17 LumaMax = 196
00000077 10:53:58 RT_Debug: QLMM: LumaRange: 13) [9294 ] LumaMin = 20 LumaMax = 195
00000078 10:53:58 RT_Debug: QLMM: LumaRange: 14) [10009] LumaMin = 18 LumaMax = 197
00000079 10:53:58 RT_Debug: QLMM: LumaRange: 15) [10724] LumaMin = 18 LumaMax = 155
00000080 10:53:58 RT_Debug: QLMM: LumaRange: 16) [11439] LumaMin = 16 LumaMax = 143
00000081 10:53:58 RT_Debug: QLMM: LumaRange: 17) [12154] LumaMin = 16 LumaMax = 153
00000082 10:53:58 RT_Debug: QLMM: LumaRange: 18) [12869] LumaMin = 20 LumaMax = 191
00000083 10:53:58 RT_Debug: QLMM: LumaRange: 19) [13583] LumaMin = 24 LumaMax = 127
00000084 10:53:58 RT_Debug: QLMM: LumaRange: 20) [14298] LumaMin = 27 LumaMax = 204
00000085 10:53:58 RT_Debug: QLMM: LumaRange: 21) [15013] LumaMin = 21 LumaMax = 191
00000086 10:53:58 RT_Debug: QLMM: LumaRange: 22) [15728] LumaMin = 20 LumaMax = 184
00000087 10:53:58 RT_Debug: QLMM: LumaRange: 23) [16443] LumaMin = 18 LumaMax = 200
00000088 10:53:58 RT_Debug: QLMM: LumaRange: 24) [17158] LumaMin = 29 LumaMax = 191
00000089 10:53:58 RT_Debug: QLMM:
00000090 10:53:58 RT_Debug: QLMM: Return: QLMMMin=16 QLMMMax=204
00000091 10:53:58 RT_Debug: AutoLevel: Levels(16,1.0,219,16,235)
00000092 10:53:58 RT_Debug: SIGNALDAR: MeGUI DAR=2.415094

[StainlessS]

New version RT_Stats() v1.02, see first post.

[StainlessS]

New version RT_Stats() v1.03, see first post.

Also updated QueryBorderCrop() and QueryLumaMinMax(), a few posts earlier (Also now contained in RT_Stats zip).

[StainlessS]

New Version RT_Stats, See first post.
Added RT_Call() function, calls an executable.

[wiseant]

@StainlessS

Nice!

Keep up the great work on this . . .

[wiseant]

using RT_Call with MediaInfo and template

http://forum.doom9.org/showthread.php?p=1591158#post1591158

[martin53]

StainlessS,
tried RT_call, because you gave it a 'hide' option :). It performs different from 'Call', like I wrote in that thread (http://forum.doom9.org/showthread.php?p=1583270#post1583270). What I mean is shown in this example:
ColorBars()
ShowFrameNumber()
ConvertToRGB24()
ScriptClip("""
c = last
Trim(current_frame, -1)
ImageWriter("C:\Temp\", 0, 0, type="tif")
Call("C:\PortableApps\ImageMagick\Mogrify.exe -flip C:\Temp\000000.tif", "0")
#RT_Call("C:\PortableApps\ImageMagick\Mogrify.exe -flip C:\Temp\000000.tif",true)
StackHorizontal(last,ImageSource("C:\Temp\000000.tif")).Crop(c.width,0,0,0)
""")

(For the newbies who happen to read this thread: The StackHorizontal... trick forces the 'last' frame with the ImageWriter and Call commands to be requested first, and then the ImageSource clip - so ImageWriter and Call can prepare the file for ImageSource within the same run of the realtime environment)

You will see the flipped frame. Now move the comment marker to Call, and you see that the frame is no longer flipped!

In my humble words: It seems as if RT_Call, like Run, are executed when the realtime environment is compiled, rather than when a frame is requested. Call however is executed during frame #0 request, and because the clip is always trimmed to start with the current frame, every frame is frame #0 for Call.

Of course I would be happy if you decided to extend RT_Call to my need, because the 'hide' option is very nice indeed.

And btw., here I demonstrate how all Imagemagick filters in fact can even be used inside the runtime environment. In the thread where I first asked for an interface and then proposed a solution, I had not yet found the StackHorizontal... trick and hence needed a separate ScriptClip to read the file back.

[StainlessS]

@Martin53,

See CallCmd() v0.0 beta, source included here:-
http://forum.doom9.org/showthread.php?p=1593132#post1593132

NOTE, your above example fails if 000000.tif does not already exist before script executed,
ImageSource() checks for it in its constructor (EDIT: before tif created) and fails if absent.

EDIT:
Also, mogrify.exe produces below text in console window, seems to be an incompatibility between ImageWriter
and mogrify.exe, presume in EXIF (or similar) data.


mogrify.exe: incorrect count for field "DateTime" (21, expecting 20); tag ignored. (C:\Temp\000000.tif).

[martin53]

@StainlessS,
sorry for the inconvenience. Yes, it seems that the file must be there in advance. I propose this approach (red line added).
ColorBars()
ShowFrameNumber()
ConvertToRGB24()
Exist("C:\Temp\000000.tif") ? last : ImageWriter("C:\Temp\", 0, 0, type="tif")
ScriptClip("""
c = last
Trim(current_frame, -1)
ImageWriter("C:\Temp\", 0, 0, type="tif")
Call("C:\PortableApps\ImageMagick\Mogrify.exe -flip C:\Temp\000000.tif", "0")
#RT_Call("C:\PortableApps\ImageMagick\Mogrify.exe -flip C:\Temp\000000.tif",true)
StackHorizontal(last,ImageSource("C:\Temp\000000.tif")).Crop(c.width,0,0,0)
""")
As for the mogrify error message, I do not experience this - or I am not aware of it. Does the example still work and the message can be ignored, or does the exampe fail? Maybe it's an ImageMagick version issue ...
Anyway, tif format is not mandatory. I just chose it because I expected that it fulfills the requirements (speed, compatibility, no quality loss due to compression)

[StainlessS]

Nice Exist solution.

I recently downloaded ImageMagick (already got a copy but could not find it), perhaps you have newer version.

From Mogrify.exe

Version: @(#)ImageMagick 5.4.6 06/01/02 Q:8 http://www.imagemagick.org
Copyright: Copyright (C) 2002 ImageMagick Studio LLC

It was only a warning anyway, did not affect result.

[martin53]

Mine says

Version: ImageMagick 6.7.8-3 2012-07-11 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2012 ImageMagick Studio LLC

Never gave version issues a thought - just downloaded and used.

I tested the command line Mogrify.exe -flip C:\Temp\000000.tif
- no warnings, works as expected.

[StainlessS]

@Martin53
The initial Mogrify.exe I was using from 2002 (v5.4.6) was from here:
http://www.dylanbeattie.net/magick/downloads.html

Works fine and only 2.3MB.

Portable one I just downloaded from ImageMagick.org (ImageMagick-6.7.9-9-Q16-windows.zip) 44MB
http://imagemagick.org/script/binary-releases.php#windows

But this one displays mogrify'ed images as bright green with a little darker color around font edges when loaded into any
MS apps except MS Paint. Other paint type programs I tried all loaded them OK, so strangely its only
the more capable MS apps that screw up. Think I'll stick with the older smaller portable version.

EDIT: the newer one is listed as "16 bits-per-pixel", strange that MS Paint (XP32) has no problems and MS OFFICE 2003
picture manager fails. (I dont like office 2007+).

[martin53]

Thanks for the testing, that might help users who want to use the interface and struggle with the incompatible capabilities.

[martin53]

StainlessS,
I never said :thanks: for RT_Stats. I do now!

I am just facing a problem: I use MVTools2's MMask(kind=5) to get a U/V frame of object motion.

MMask leaves U/V at the default 128 for blocks it can not assign a valid motion vector to.

Now I need the average motion of a part of the image.
Aside from needing crop and AverageChromaU, or UToY and RT_AverageLuma - so two steps either way (I am always in the RTE with my projects), the pixels with 128 spoil the result. I could use

if (RT_AverageLuma>128) {
#start a while () loop with RT_YPlaneMin() and ascending threshold to get the %age of pixels with 128, then recalculate the average ond the basis of that amount
} else {
#RT_YPlaneMax()...
}
but that would fail if some blocks move in one direction, and some in the other.

I could convert to RGBA, ResetMask(), ColorMask() to create a copy of these pixels in the mask clip.

That's all so complicated. Do you have a better idea? Maybe a new function I give two numbers and the function tells me how many % of the pixels are in that range? That would also allow cool histogram functions.

[StainlessS]

Maybe a new function I give two numbers and the function tells me how many % of the pixels are in that range? That would also allow cool histogram functions.

OK, but it's your job to think up a name, my plugs nearly all start out having names like "Fred" or "Test", same for arg names.

I'll implement "Greater or equal to low" and "below high", if not as desired, say so. I'll look here again before I post link.

EDIT: Ps, hope you saw the edit in the developer Call() thread concerning Exist().

[StainlessS]

Here tis, given a moniker of YBetwixt(), (EDIT: "betwixt", usage archaic, "in the interval", ie "between")
EDIT: Link removed, See 1st post, renamed func to RT_YInRange().


As for RT_Stats funcs


YBetwixt(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "Matrix"=0,
int "lo"=64,int "hi"=192)

Returns Float % population of pixels above or equal to "lo" and below "hi".

If hi <= lo then will return 0.0


EDIT: If OK, I may perhaps add to RT_Stats plug as RT_YBetwixt or another name if you have one.
EDIT: If I do add it to RT_Stats, may return value in range 0.0-255.0 comparable to other RT_Stats funcs,
can easily convert to percent by *100.0/255.0

[wiseant]

@Stainless

Re RT_Stats

One thing I noticed is that RT_Stats won't work reading an imagelist if the imagefile name is e.g. %big_file06.jpg - the problem being the %
[probably some other funky characters as well]

Is there a way to overcome this without having to rename the files in question?

TIA

[StainlessS]

Bit pissed at the moment, but, not sure what you mean.
Not aware of using "%" in any RT_Stats plugs as any kind of special character.
What func in particular are you talking about?, give an exact example, just filenames and commands.
Your post is a bit non specific.

[wiseant]

@StainlessS

Sorry for the confusion - this time I tested things with just one file

The bmpList file -> 82539.List:
D:\ImageTest\%myfile.jpg

excerpt from DebugView:
[3436] RT_Debug: Found BMP files = 1
[3436] RT_Debug:
[3436] RT_Debug: Processing BMP File D:\ImageTest\%myfile.jpg

so the bmpList file is OK and RT_Debug shows the correct filename, but I get an error when opening my avs file is either VirtualDub, AVSPMod, MPlayerClassic

VirtualDub Error
ImageReader: error 'Could not open file' in DevIL library
reading file D:\ImageTest\myfile.jpg

the filename is %myfile.jpg - so the problem is with the % character

It isn't a problem with RT_Stats, but rather with ImageReader


loadplugin("GScript.dll")
loadplugin("RT_Stats.dll")
fps = 1
end = fps-1
GScript("""
picclip=0
result=RT_WriteFileList("D:\ImageTest\*.jpg","82539.List")
Assert((result!=0),"No BMP files found")
BMPLIST=RT_ReadTxtFromFile("82539.List")
FILES=RT_TxtQueryLines(BMPLIST)
RT_Debug("Found BMP files = " + String(FILES))
RT_Debug(" ")
canvaswidth = 640
canvasheight = 480
For(i=0,FILES-1) {
FN=RT_TxtGetLine(BMPLIST,i)
RT_Debug("Processing BMP File",FN)
k=ImageSource(FN,end=end).ConvertToRGB32()
iHeight=k.height iWidth=k.width
if (iHeight >= iWidth) {
iHeight1 = canvasheight
iWidth1 = (canvasheight * iWidth) / iHeight
x = canvaswidth - iWidth1
x1 = x/2
if (iWidth1 % 2 > 0) {
x1 = (x/2) + 1
}
k = k.bilinearresize(iWidth1,canvasheight).addborders(x1,0,x/2,0,$101010)
}
if (iWidth > iHeight) {
iWidth1 = canvaswidth
iHeight1 = (canvaswidth*iHeight)/iWidth
y = canvasheight-iHeight1
y1 = y/2
if (iHeight1 % 2 >0) {
y1 = (y/2) + 1
}
k = k.bilinearresize(iWidth1,iHeight1).addborders(0,y1,0,y/2,$101010)
}
if (iwidth1 > canvaswidth) {
iwidth1 = canvaswidth
iheight1 = (canvaswidth*iheight)/iwidth
y = canvasheight-iheight1
y1 = y/2
if (iheight1 % 2 >0) {
y1 = (y/2) + 1
}
k = k.bilinearresize(iWidth1,iHeight1).addborders(0,y1,0,y/2,$101010)
}
if (iHeight1 > canvasheight) {
iHeight1 = canvasheight
iWidth1 = (canvasheight*iWidth)/iHeight
x = canvaswidth-iWidth1
x1 = x/2
if (iWidth1 % 2 >0) {
x1 = (x/2) + 1
}
k = k.bilinearresize(iWidth1,iHeight1).addborders(x1,0,x/2,0,$101010)
}
RT_Debug("NewWidth =",String(k.Width),"NewHeight =",String(k.height))
picclip = (IsClip(picclip)) ? picclip ++ k : k
RT_Debug("")
}
""")
PicClip
assumefps(fps)

[StainlessS]

From ImageReader/ImageSource Docs

file: template for the image file(s), where frame number substitution can be specified using sprintf syntax. For example, the files written by ImageWriter's default parameters can be referenced with "c:\%06d.ebmp". As of v2.56 if the template points to a single file then that file is read once and subsequently returned for all requested frames.

So, file is a template, not a filename, it is swallowing your '%' char then stumbling when it cannot find the rest of
the expected sequence eg "06d" and has a last ditch attempt at opening the file without the '%' char that it has already skipped over, hence:

ImageReader: error 'Could not open file' in DevIL library
reading file D:\ImageTest\myfile.jpg


EDIT: Think this escape might fix it (replace '%' by double '%%' before call to imagereader, untested but should work)

FN=StrReplace(FN,"%","%%")

Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Original:- http://forum.doom9.org/showthread.php?t=147846&highlight=gscript By Vampiredom, Gavino, IanB
#{i=s.FindStr(find)return(i==0?s: s.LeftStr(i-1)+replace+s.MidStr(Strlen(find)+i).StrReplace(find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{i=s.FindStr(find) return(i==0?s:RT_StrAddStr(s.LeftStr(i-1),replace,s.MidStr(Strlen(find)+i).StrReplace(find,replace)))}

[martin53]

Here tis, given a moniker of YBetwixt(), (EDIT: "betwixt", usage archaic, "in the interval", ie "between")

You are so fast, and really helpful! :thanks: looking at the time of your post.
Hope you don't mind that I can only test tomorrow. I'll sure give you feedback soon.

[StainlessS]

No sweat, take your time.

The function was really quite easy to implement, about 10-15 mins at most.
I just hacked out a piece of RT_Stats code which is multi functional, ie is used by about 5 or 6
functions with the only difference being after the entire frame has been sampled into a 256 element count array,
fn specific stuff done on the array.
Only required about a dozen (if that) extra lines of code to implement the additional fn specific code.

[martin53]

I tested it, works fine! (Unfortunately, does not help with my problem as expected, but that's a different story)

Though not being a native english speaker, I'd propose the name RT_YFractionInRange().

For the scaling, I'd like to add these considerations

Because the result is quite likely to be used as a factor or a denominator, a scaling to 1.0 avoids additional scaling at script level. E.g. AverageLuma/(1.0001-RT_YFractionInRange(lo=128,hi=129)) is simple and efficient.

Because for computed ranges as in GScript("for (i=0,255,1) {RT_YFractionInRange(lo=i,hi=i) ... }") it is simpler to script, when the hi limit is included rather than excluded. This of course does not apply if hi defaults to lo, so hi can be omitted in the script if the range has only the range of 1.

I had a problem with clip size because I needed to evaluate the U plane: UToY().YBetwixt(lo=128, hi=129) produced an error about an uneven width (although I was quite sure the original's width and height were multiples of 4). It would be easier to script, and more important avoid the intermediate clip, if also RT_UFractionInRange() and RT_VFractionInRange() existed.
And yes, I think the functions are worth to be included in RT_Stats.

[StainlessS]

RT_YFractionInRange(), happy to accept although might a more succinct RT_YInRange() be OK ?

Would actually prefer to keep in range 0.0 - 255.0 as all others (except RT_LumaCorrelation(-1.0 - 1.0)), but your idea so OK, accepted.
(AverageLuma etc might also have been better if in range 0.0 - 1.0)

Lo and hi inclusive with hi defaulting to lo, Accepted. (unless you require otherwise, will default lo to 128).

I presume that the 'uneven width' error was produced in UToY, dont think can happen in the RT stuff, can you check.

As for the chroma equivalent routines, I will implement these when v2.6 is the norm, right now, luma is safe to use
whether 2.58 or 2.6 colorspaces.
[I dont use v2.6 plugins as they tend to break scripts and are perhaps a bit clunky because of kludges.
I may drop support for 2.58 altogether when v2.6 plugin support is concrete, as far as I'm, concerned, v2.6 is
considerably more stable and bug free than 2.58, we just have to wait for plug support to be equally stable.]

By the way, did you checkout the script in Developer Call() thread, it should be faster not using the RTE, also nice trick
forcing the initial ImageWrite during compile time. Your Exist() line also needs attention as noted there.

[martin53]

Thanks for reply, please do as you proposed.
I'll check the Call() thread in near future. When you first indicated it, I thought I knew the most actual state, but what you write now, contains details I don't remember.

I am in the middle of making a motion compensated background for deshaking, so I want to finish that first. Just have to find again a post where padding with PointResize() was shown - can't get it working (for the initial frame of my background).

[StainlessS]

RT_Stats() New Version v1.05, Added RT_YInRange(). See 1st post.


@Martin53,

Tried this without any problems:


avisource("D:\avs\avi\1.avi").ConvertToYV12()

u=UToy() # get u outside of runtime environment
v=VToy() # get v outside of runtime environment

scriptclip("""
zu=u.RT_YInRange(lo=128)
zv=v.RT_YInRange(lo=128)
subtitle(String(zu),align=7).subtitle(String(zv),align=9)
""",args="u,v") # Pass in u & v as args, needs Grunt()

return last

[martin53]

You document RT_ReadTxtFromFile() as a non-clip function.

As I learned from your explanation on the Call() plugin: can I use this function to retrieve data during the frame processing phase?

My situation is this: I need motion estimation data between, say, the previous frame and the current one.

MDepan() writes this information to a text file - after the graph is built, I think. I assume, that MDepan() closes the file immediately, although I don't know yet.

How can I get this information into script variables, if my script is running in the realtime environment?
First I need grep or something to extract the line - solved.
Now I need RT_ReadTextFromFile or something else to give me access to these informations :confused:

The following clip filter in the graph shall transform the frame (yep, the quad plugin :) ), i.e. I need the motion numbers from the file to serve them to quad()!

I figured out this code. But I suspect what I need is impossible in one pass, except if MDepan() uses the part of the graph that already exists during graph build, and I can gain access to the file in that time slot.

c.MDepan(v, log=logfile, zoom=true, rot=true, range=1, thSCD1=960)
#assumption: motion informations are now in logfile.

Command = """cmd /c find /N " """+string(frame)+""" " """+logfile+">C:\Temp\MotionData.log"
#"""find /N " 123 " CIMG6437_.avi.deshaker>C:\Temp\MotionData.log""" writes:
#
#---------- CIMG6437_.AVI.DESHAKER
#[124] 123 10.26 1.45 -0.049 0.99615
#
#(first is number of line, then line content from MDepan. This starts with frame number, then x motion, y motion, rotation angle, zoom factor.
#File ends with return char.

CallCmd(Command, "*", Hide=true, Debug=true)

motionData = "Select(i, "+StrReplace(StrReplace(RT_ReadTxtFromFile(C:\Temp\MotionData.log" ,Lines=1 , Start=-1)," "," ")," ",",")+")"
# make single spaces and convert them into commas for Select()

#motionData to be used like: "i=2 xShift=Eval(motionData)"


EDIT: MDepan() also writes the motion data into the topmost line of the output clip. With RT_AverageLuma(w=1,h=1) it is already possible to do "pixel peeking", but I fear that the reconstruction of float values by this means is not reasonable. Still wanted to add that aspect to the description.

[StainlessS]

@Martin53,
Sorry about not responding earlier, got a bit waylaid.
About to be sidetracked again (just decided to make a plugin to both auto crop and/or auto level) so decided
to answer briefly before going offline for perhaps a while.

Are you still dealing with the above problem, I've never used the Mdepan func with logfile (dont think).
Dont know whether log file creating on fly, with each frame or at destruction.
Either way, probably not available (or open file) and maybe requires 2 pass.
But perhaps, if you step through the MDepan frames using GScript, as a prescan stage (long timeout) and then
IF log file is complete, might be able to use RT_ReadTextFromFile to retrieve it, and throw away the original result clip.

Stepping through frames examples here:
http://forum.doom9.org/showthread.php?t=165694

RT_ReadTextFromFile reads possibly multiline [Chr(10) separated] string from a text file.
RT_FileQueryLines, gives number of lines in the multiline string (EDIT In file).
RT_TxtGetLine, extracts a single line from a multiline string.


Multiline string input and parse example:
http://forum.doom9.org/showthread.php?p=1588523#post1588523

EDIT: MDepan() also writes the motion data into the topmost line of the output clip. With RT_AverageLuma(w=1,h=1) it is already possible to do "pixel peeking

RT_AverageLuma(w=1,h=1) would return a value via this code:

return (float)((double)acc / Pixels)
as
return (float)((double)the_pixel_value / 1.0) # as Pixels (1) is coerced to double, division done, and then rounded to float.

Should return float representation of the int pixel value, although rounded conversion to int might be a good idea (probably NOT necessary).

EDIT: Rough guess, untested.

VideoFileName ="D:\AVS\AVI\1.d2v"
c=MPEG2Source(VideoFileName)
c
MDepan(...)
GScript("""
for (n=0,Framecount-1) {
RT_AverageLuma(n,w=1,h=1) # frame force read
}
""")
c # Hopefully destructor of temp last clip called here and log file available after here



As I learned from your explanation on the Call() plugin:
Dont place too much faith in that, mostly based on guesswork.

[StainlessS]

@Martin53, the below extracts the MDepan data as required, I'll leave the rest to you.


c=MPEG2Source("D:\AVS\AVI\1.d2v").Trim(0,5)
c
LOGFILE = "LogFile.Log"
LINEFILE = "C:\Temp\MotionData.log"

COMMAND = """cmd /c find /N " # " """ + LOGFILE + " >" + LINEFILE


v=MSuper().MAnalyse(isb = false)
gm=MDepan(v, log=LOGFILE, zoom=true, rot=true, range=1, thSCD1=960)

GScript(""" # Force MDepan analysis
for (n=0,Framecount-1) {
gm.RT_AverageLuma(n,w=1,h=1) # Force frame read of Global Motion clip, and MDepan testing.
}
""")

gm=0 # Destructor of temp Global Motion clip called here and log file available after here (created in Destructor)
c
CallCmd(COMMAND,"0,0",Insert="#",digits=1,Hide=true,Debug=true) # Extract line of motion data from log file prior each frame to scriptclip.
ScriptClip("""
MotionData=GetMotionData(LINEFILE)
return last
""",After_Frame=true) # NOTE, After_Frame

return last

Function GetMotionData(String fn) {
LOGSTR=RT_ReadTxtFromFile(fn,Lines=1,Start=2) # Skip blank line and "---------- LOGFILE.LOG" line.
LOGSTR=StrReplace(LOGSTR,Chr(10),"") # Remove Newline ie Chr(10)
RT_DEBUG("LOGSTR ='"+LOGSTR+"'")
LOGSTR=StrReplace(StrReplace(LOGSTR,"[",""),"]","") # Remove Line markers
RT_DEBUG("UNLINEMARKED ='"+LOGSTR+"'")
LOGSTR=StrReplaceDeep(LOGSTR," "," ") # Convert SPACE pairs to single SPACE
RT_DEBUG("SPACECOMPACTED='"+LOGSTR+"'")
LOGSTR=StrReplace(LOGSTR," ",",") # Convert SPACE to COMMA
RT_DEBUG("COMMACONVERTED='"+LOGSTR+"'")
MotionData = "Select(i, " + LOGSTR +")"
RT_DEBUG("MotionData ='"+MotionData+"'")
return MotionData
}

Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Original:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
#{i=s.FindStr(find)return(i==0?s: s.LeftStr(i-1)+replace+s.MidStr(Strlen(find)+i).StrReplace(find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{i=s.FindStr(find) return(i==0?s:RT_StrAddStr(s.LeftStr(i-1),replace,s.MidStr(Strlen(find)+i).StrReplace(find,replace)))}

Function StrReplaceDeep(string s,string find,string replace) # DEEP Repeated, string replacements
# When replacing eg pairs of SPACE characters by single SPACE, StrReplace can leave some pairs non replaced.
# Assumes that you want to rescan where replaced string could constitute part of a new find substring,
# Length of replace must be shorter than length of find, else will call StrReplace instead.
# Based On:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
# {i=s.FindStr(find)return i==0?s:s.LeftStr(i-1)+( strlen(replace)<strlen(find)? \
# StrReplaceDeep(replace+s.MidStr(Strlen(find)+i),find,replace) : replace+StrReplace(s.MidStr(Strlen(find)+i),find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{i=s.FindStr(find)return i==0?s:RT_StrAddStr(s.LeftStr(i-1),( strlen(replace)<strlen(find)? \
StrReplaceDeep(RT_StrAddStr(replace,s.MidStr(Strlen(find)+i)),find,replace) : \
RT_StrAddStr(replace,StrReplace(s.MidStr(Strlen(find)+i),find,replace))))}


Debug stuff

00000002 1.81485665 [2956] CallCmd: Constructor IN
00000003 1.81489933 [2956] CallCmd: CallCmd: v1.00 - 08 Oct 2012 - by StainlessS
00000004 1.81494594 [2956] CallCmd: Command for Frames = 'cmd /c find /N " # " LogFile.Log >C:\Temp\MotionData.log'
00000005 1.81499493 [2956] CallCmd: FRAMES: About to Parse Frames String
00000006 1.81504369 [2956] CallCmd: FRAMES: Line 1:1 0,5 $0,0
00000007 1.81509042 [2956] CallCmd: Doing command on 6 Frames
00000008 1.81513464 [2956] CallCmd: Constructor OUT
00000009 2.16997981 [2956] CallCmd: GetFrame(0) on command 'cmd /c find /N " 0 " LogFile.Log >C:\Temp\MotionData.log'
00000010 2.37153673 [2956] CallCmd: SUCCESS Creating Process
00000011 2.37663078 [2956] RT_Debug: LOGSTR ='[1] 0 0.00 0.00 0.000 1.00000'
00000012 2.38259101 [2956] RT_Debug: UNLINEMARKED ='1 0 0.00 0.00 0.000 1.00000'
00000013 2.41031384 [2956] RT_Debug: SPACECOMPACTED='1 0 0.00 0.00 0.000 1.00000'
00000014 2.42110348 [2956] RT_Debug: COMMACONVERTED='1,0,0.00,0.00,0.000,1.00000'
00000015 2.42123628 [2956] RT_Debug: MotionData ='Select(i, 1,0,0.00,0.00,0.000,1.00000)'
00000016 2.47051644 [2956] CallCmd: GetFrame(1) on command 'cmd /c find /N " 1 " LogFile.Log >C:\Temp\MotionData.log'
00000017 2.59828448 [2956] CallCmd: SUCCESS Creating Process
00000018 2.60344887 [2956] RT_Debug: LOGSTR ='[2] 1 -0.01 0.00 0.000 1.00000'
00000019 2.60861468 [2956] RT_Debug: UNLINEMARKED ='2 1 -0.01 0.00 0.000 1.00000'
00000020 2.63412428 [2956] RT_Debug: SPACECOMPACTED='2 1 -0.01 0.00 0.000 1.00000'
00000021 2.64520335 [2956] RT_Debug: COMMACONVERTED='2,1,-0.01,0.00,0.000,1.00000'
00000022 2.64533710 [2956] RT_Debug: MotionData ='Select(i, 2,1,-0.01,0.00,0.000,1.00000)'
00000023 2.67647266 [2956] CallCmd: GetFrame(2) on command 'cmd /c find /N " 2 " LogFile.Log >C:\Temp\MotionData.log'
00000024 2.80067348 [2956] CallCmd: SUCCESS Creating Process
00000025 2.80581069 [2956] RT_Debug: LOGSTR ='[3] 2 0.01 0.00 0.000 1.00000'
00000026 2.81096816 [2956] RT_Debug: UNLINEMARKED ='3 2 0.01 0.00 0.000 1.00000'
00000027 2.83819222 [2956] RT_Debug: SPACECOMPACTED='3 2 0.01 0.00 0.000 1.00000'
00000028 2.84876084 [2956] RT_Debug: COMMACONVERTED='3,2,0.01,0.00,0.000,1.00000'
00000029 2.84887767 [2956] RT_Debug: MotionData ='Select(i, 3,2,0.01,0.00,0.000,1.00000)'
00000030 2.84958386 [2956] CallCmd: GetFrame(3) on command 'cmd /c find /N " 3 " LogFile.Log >C:\Temp\MotionData.log'
00000031 2.97924876 [2956] CallCmd: SUCCESS Creating Process
00000032 2.98432541 [2956] RT_Debug: LOGSTR ='[4] 3 -0.01 0.28 0.033 1.00011'
00000033 2.98958492 [2956] RT_Debug: UNLINEMARKED ='4 3 -0.01 0.28 0.033 1.00011'
00000034 3.01530290 [2956] RT_Debug: SPACECOMPACTED='4 3 -0.01 0.28 0.033 1.00011'
00000035 3.02687788 [2956] RT_Debug: COMMACONVERTED='4,3,-0.01,0.28,0.033,1.00011'
00000036 3.02699447 [2956] RT_Debug: MotionData ='Select(i, 4,3,-0.01,0.28,0.033,1.00011)'
00000037 3.02782226 [2956] CallCmd: GetFrame(4) on command 'cmd /c find /N " 4 " LogFile.Log >C:\Temp\MotionData.log'
00000038 3.15263963 [2956] CallCmd: SUCCESS Creating Process
00000039 3.15752482 [2956] RT_Debug: LOGSTR ='[5] 4 -0.10 0.43 0.004 0.99982'
00000040 3.16313744 [2956] RT_Debug: UNLINEMARKED ='5 4 -0.10 0.43 0.004 0.99982'
00000041 3.18898296 [2956] RT_Debug: SPACECOMPACTED='5 4 -0.10 0.43 0.004 0.99982'
00000042 3.19961834 [2956] RT_Debug: COMMACONVERTED='5,4,-0.10,0.43,0.004,0.99982'
00000043 3.19973421 [2956] RT_Debug: MotionData ='Select(i, 5,4,-0.10,0.43,0.004,0.99982)'
00000044 3.20040274 [2956] CallCmd: GetFrame(5) on command 'cmd /c find /N " 5 " LogFile.Log >C:\Temp\MotionData.log'
00000045 3.35947132 [2956] CallCmd: SUCCESS Creating Process
00000046 3.36437130 [2956] RT_Debug: LOGSTR ='[6] 5 0.30 0.35 -0.009 0.99970'
00000047 3.37089896 [2956] RT_Debug: UNLINEMARKED ='6 5 0.30 0.35 -0.009 0.99970'
00000048 3.40061140 [2956] RT_Debug: SPACECOMPACTED='6 5 0.30 0.35 -0.009 0.99970'
00000049 3.41689062 [2956] RT_Debug: COMMACONVERTED='6,5,0.30,0.35,-0.009,0.99970'
00000050 3.41767979 [2956] RT_Debug: MotionData ='Select(i, 6,5,0.30,0.35,-0.009,0.99970)'
00000051 6.74900341 [2956] CallCmd: Destructor IN
00000052 6.75118876 [2956] CallCmd: Destructor OUT


EDITED: Moved contents of scriptclip text extraction/converting into a script function (precompiled).

EDIT: We could have loaded entire LogFile.Log into (EDIT: big) string and extracted single line at a time from it using
LINESTR=RT_TxtGetLine(LOGSTRING,current_frame) in ScriptClip, and avoid the CallCmd call to FIND the string.

[travolter]

@StainlessS
Im replying you from this thread
http://forum.doom9.org/showthread.php?p=1597871#post1597871

QueryBorderCrop() freezed image, and QueryLumaMinMax() not..
When I use robolevels(QBC=false) it does not freeze image but there is a problem. The image only show 5 colors: pink/orange/green/blue/white (remember me CGA) ;)

Im using your RT_Stats 20121003 and I use potplayer but also verified with mplayer. Using YV12 color space and overlay mixer as render.
No message appear when I write robolevels(QBC=false, Debug=true)

I only need to correct the levels.. maybe there is a workaround to solve the problem.

[StainlessS]

travolter,
Get this (MS DebugView):
http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx

Allows you to see messages coming out of scripts and other programs/filters.

And try


QueryLumaMinMax(debug=true) # for levels info

and

QueryBorderCrop(debug=true) # for test why fail


or

RoboLevels(QBC=false,debug=true) # for test why fail


A short sample that gives problems might be a good idea. Thanks.

EDIT:, you could also add this line just before the Levels() line near the end of RoboLevels function:

(DEBUG)? RT_Debug("Levels("+String(ALMin)+",1.0,"+String(AlMax)+","+String(CSMin)+","+String(CSMax)+",Coring=False)"):NOP

[StainlessS]

RT_TxtSort() error in docs, 'sort' should read 'mode'.

RT_TxtSort(String,int 'mode'=0)
Non-clip function.
Function to sort a Chr(10) separated multiline string.
The String arg is the name of the string, and 'mode' can be in range 0 to 7.
0 = Sort ascending, case insignificant
1 = Sort ascending, case significant
2 = Sort decending, case insignificant
3 = Sort decending, case significant
4 = Sort ascending, Filenames, digits sorted by value
5 = Sort ascending, Filenames, old style as for Windows 2000
6 = Sort decending, Filenames, digits sorted by value
7 = Sort decending, Filenames, old style as for Windows 2000

[martin53]

@Martin53, the below extracts the MDepan data as required, I'll leave the rest to you.


StainlessS,
thank you very much indeed for the work you put into this script!:thanks:
The task was part of my script FSubstitute - to get global pan/zoom/rotate information between two arbitrary frames of a clip.

I decided to also try a third option (besides of using MMask kind=5), because - according to what I found - the extraction script needs to be in a different RTE than MDepan, otherwise MDepan keeps the file handle locked and the file cannot be read from. In my case, MDepan must run in the RTE. Also, MDepan only delivers the values for two adjacent frames, so numerous extractions and further calculations would have been neccessary.

The third option is really cool: RT_LumaCorrelation between two frames typically has a maximum over varying x,y offset when the x,y is the global pan between these frames. I implemented a stepwise seek loop, and it is not even sooo slow. And what's best: with the same idea, but applied to the four quarters of the frame, you get data to calculate zoom and rotation. As if this were not enough, a real quadrilateral transformation that is possible with these data gives superior results than restriction to zoom & rotation. I.e. the third option excels the possibilities of MDepan, and it even opens the way for an idea of a deshaker based (almost) completely on AviSynth script level and existing plugins: That deshaker will maybe able to compensate tilt - something I don't know from any existing deshaker.

My apologies, if you feel disappointed that the extraction script is not used in my project. I hope you are compensated by the extraordinary value RT_LumaCorrelation has for my work.

[StainlessS]

My apologies, if you feel disappointed that the extraction script is not used in my project.

Not at all, it was an interesting exercise and resulted in the StrReplaceDeep mod which is a very useful alternative to StrReplace.
Good luck with your monster script. :)

[martin53]

StainlessS,
thanks to your suggestions, mainly because of the destructor tricks I think, I was able to integrate MDepan into my 'monster script'. It produces good motion estimation and runs faster than the RT_LumaCorr approximation.

[martin53]

StainlessS,
could you extend RT_TxtWriteFile(String,String,bool "append") to be able to append to an existing file?

[StainlessS]

Yo, I hear you but not well able to tie a shoelace right now, will try again when a little less unable.

[martin53]

StainlessS,
I wondered if RT_[R,G,B,A]Plane... functions would be useful for RGB32 clips.


Especially for this advanced challenge: BlankClip(color=...) allows to create a 1x1 RGB32 clip with color being a 32bit variable.
ColorBars(pixel_type="RGB32")
Layer(BlankClip(pixel_type="RGB32",color=$60101010))

I.e. the 4 [A,R,G,B] bytes could be used to store one frame specific value with a reasonable resolution for many integer or float applications.

As an alternative and opposed to a ConditionalReader or RT_ReadTxtFromFile file, the clip benefits from AviSynth's frame cache, does not suffer from file system overhead, and should to my expectation not slow down too much with very high frame numbers.

With a function like RT_AverageARGB() or RT_ARGBPlanesMin() or whatever, the full 32bit value could hopefully be retrieved and used.

With StackHorizontal(), multiple 1x1 clips could be assembled to one and be read out with RT_AverageRGBA(x=..., w=1,h=1).

Too weird? :)

[jmac698]

Hi,
I've finally had an opportunity to use your plugin! But I'm having problems.
I have an rgb clip and I want to autocrop all the way around, per frame. In other words, this is a kind of tracking object thing.
Even worse, I need to scan all frames, find the max size, then render all frames at this max size and center each frame inside it.
Think of it this way, it's a video of a planet that's moving across the frame. The entire frame is black except one object. I need to show only this object centered on each frame. The frame size has to be consistent.
Think you could come up with something like that?

[jmac698]

I haven't been able to get it to crop at all. If I set samples too high it takes several minutes. Maybe this can't be done without a plugin?
Here's my files:
http://www.sendspace.com/file/wlcoa8

And my script:

dir="C:\Users\StainlessS\Pictures\moon\"
#IMG_5245 IMG_5288
s=5245
e=5288
imagereader(dir+"IMG_%04d.jpg",s,e,24,true)
#~ Function QueryBorderCrop(clip c,int "Samples",Float "Thresh",bool "Laced",int "XMod",int "YMod",int "WMod",int "HMod", \
#~ bool "Relative", String "Prefix",int "RLBT",bool "DEBUG",float "Ignore",int "Matrix")
v=QueryBorderCrop(samples=1,laced=false,xmod=1,ymod=1,wmod=1,hmod=1)
#messageclip(v)
eval(v)
#QBCropX=8 QBCropY=8 QBCropW=640 QBCropH=480
Crop(QBCropX,QBCropY,QBCropW,QBCropH)

[StainlessS]

Jmac, see here: http://forum.doom9.org/showthread.php?p=1589022#post1589022

I modded QueryBorderCrop for that script and made it super sensitive for single frames. Give it a whirl.
(by the way, QueryBorderCrop is way better than AutoCrop).

StainlessS,
could you extend RT_TxtWriteFile(String,String,bool "append") to be able to append to an existing file?

Good as done. (thinking about other suggestions).

[jmac698]

It's sort of working, fails after frame 17 though. Maybe there's some bright pixel entering the frame. But for the frames that it works, it's jumping around a lot.

[StainlessS]

Try the update just posted, not any great improvement, just allows using exact crop coords (RGB).
I'll keep playing with it for a bit.

EDIT: The main problem is, the autocrop is intended to remove real borders, and there aint any.
You would need to find a good Thresh setting.

[StainlessS]

Jmac, you have white edges left and bottom, crop them off in scripts and perhaps try an explicit rather than auto Thresh.

EDIT: Forget above, I forgot there is protection against just a very thin line at edges.
We are getting somewhere, I shall post a slight amendment to the QueryBorderCrop
script soon that should ONLY be used on this as a one off.

[StainlessS]

Jmac, make this temporary mod to QueryBorderCrop


#Rgt=w/2 Bot=h/2 Lft=Rgt-1 Top=Bot-1
Rgt=Int(w*0.1) Bot=Int(h*0.1) Lft=Int(w*0.9) Top=Int(h*0.9) # Mod For JMAC698


Normally searches only left half of image for left border, mod allows search 90% of width instead for your ODD requirement,
same for other sides.

Make this mod

AUTOCROP_THRESH = -3.0 # Default -32
CROPMODE = 0


To this script: http://forum.doom9.org/showthread.php?p=1604043#post1604043

Not prefect BUT, works pretty damn good [better than I might have expected] :)

[StainlessS]

Jmac, Use This Version



# EDIT To Suit (Requires QueryBorderCrop v1.09 & QueryLumaMinMax v1.01 & RT_Stats v1.06)

VID = False # True == Video : False == Pics, see IN_FILES directories/wildcards

IN_FILES = (VID) ? "D:\VID\*.AVI|MPG|MPEG" : "D:\MOON\*.BMP|JPG|JPEG|PNG" # Added '|' Pipe separator for multiple Wildcard in Extensions ONLY.
AUTOCROP = True # Crop off black borders.
AUTOCROP_THRESH = 0.6 # Default 0.0 allows QueryBorderCop to auto set super sensitive mode for single images, use default 0.0 normally
SHOWBORDER = True # True Show replaced border in Red, else Black. (Test AUTOCROP)
AUTOLEVEL_STRENGTH = 1.0 # AutoLevel Strength 0.0 -> 1.0
SHOWAUTOLEVEL = False # Right Hand Side Only AutoLeveled
#
#FAR = 4/3.0 # Required Frame Aspect Ratio
FAR = 1.0 # Required Frame Aspect Ratio
SzLimit = 0 # Limit shorter size (usually height, see FAR). 0 Does NOT Resize.
DEBUG = True # Debug
CROPMODE = 0 # 0=CropExact(RGB Only), 1=CropLess, 2=CropMore, 3=CropPlus
WMOD = 2
HMOD = 2
# ---------------------
# Lesser changed settings
BORDER = (SHOWBORDER) ? $FF0000: (VID) ? $101010 : $000000 # Color for replaced borders.
SUBS = False # Subtitles on clips/images
DEBUG_QBC = False # Debug QueryBorderCrop
DEBUG_QLMM = False # Debug QueryLumaMinMax
FPS = (VID) ? 30000.0/1001.0 : 1.0 # Whatever.
OUT_FILE = "FILE.List" # Dont need to change really, just a name.
SCANPERC = 90.0 # Percentage of each side to scan in AutoCrop, see below link for reason.
# # http://forum.doom9.org/showthread.php?p=1604004#post1604004
#--------------------------------------------------------------------------------
GScript("""
picclip = 0 # Dummy for now
result=RT_WriteFileList(IN_FILES,OUT_FILE) # Create a listing file of Pic files
Assert((result!=0), "No files found")
FILELIST=RT_ReadTxtFromFile(OUT_FILE) # Get list of files
FILES=RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
(DEBUG) ? RT_Debug("Found files = " + String(FILES)) :NOP
# Prescan to ascertain max wid/hit of all files, also Autocrop & Auto Levels
MAXH=0 MAXW=0 # init max wid and hit
CROPCOORDS="" # Clear results array (string)
AUTOLEVELS="" # Clear AutoLevels array (string)
AUTOLEVEL=(AUTOLEVEL_STRENGTH>0.0) # Adjust using Levels.
QBCTrailer=Select(CROPMODE,"=","L=","M=","P=")
For(i=0,FILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic/vid file i
(DEBUG)?RT_Debug(String(i+1)+"/"+String(FILES)+") Getting File dimensions",FN):NOP
if(VID) {
k=DirectshowSource(FN)
k=k.ConvertToYUY2().Trim(0,0)
} else {
k=ImageReader(FN,end=0).ConvertToRGB32() # With Bmap, some come up as RGB24 bit some RGB32.
}
OrgW=k.width OrgH=k.height
W=OrgW H=OrgH # Pre-Duplicate in case of no AUTOCROP
if (AUTOCROP) {
if(AUTOCROP_THRESH == 0.0) { # Call without Thresh arg, allows use of auto super sensitive setting for single frame images
QBC = k.QueryBorderCrop(samples=24,laced=VID,debug=(DEBUG&&DEBUG_QBC),ScanPerc=ScanPerc)
} else { # Will prevent auto super sensitive from working, use on images with real borders
QBC = k.QueryBorderCrop(samples=24,Thresh=AUTOCROP_THRESH,laced=VID,debug=(DEBUG&&DEBUG_QBC),ScanPerc=ScanPerc)
}
Eval(QBC) # Set Avisynth vars
k=k.Crop(QBCropXM,QBCropYM,QBCropWM,QBCropHM) # OVERCROP using CropMore
QBC = RT_TxtGetLine(QBC,CROPMODE) # Extract REQUIRED mode, CropLess, CropMore etc
QBC1=StrReplace(QBC,QBCTrailer,"=") # Change all to style without a trailer eg QBCropXM to QBCropX.
QBC2=StrReplace(QBC1,"QBC","C") # Change all style QBCropX to CropX
Eval(QBC2) # Get the required cropmode into CropX,CropY,CropW,CropH
W=CropW H=CropH # The actual required crop mode dimensions.
CROPCOORDS=RT_TxtAddStr(CROPCOORDS,QBC2) # Avisynth Bugfix, same as CROPCOORDS = CROPCOORDS + QBC + Chr(10)
}
if (AUTOLEVEL) {
QLMM = k.QueryLumaMinMax(debug=(DEBUG&&DEBUG_QLMM)) # Query luma levels (on overcropped clip if AUTOCROP)
Eval(QLMM) # Set Avisynth vars
AUTOLEVELS=RT_TxtAddStr(AUTOLEVELS,QLMM) # Avisynth Bugfix, same as AUTOLEVELS = AUTOLEVELS + QLMM + Chr(10)
}
If(MAXW<W) {MAXW=W}
If(MAXH<H) {MAXH=H}
if (DEBUG) { # KEEP main debug messages together
RT_Debug ("Orig Width =",String(orgW),"Orig Height =",String(OrgH))
if((W != OrgW) || (H != OrgH)) { # Was Caused by AUTOCROP
RT_Debug("AutoCrop Width =",String(W),"AutoCrop Height =",String(H))
(AUTOLEVEL) ? RT_Debug("AutoLevel On Cropped : Min=",String(QLMMMin)," Max =",String(QLMMMax)) : NOP
} else {
(AUTOCROP) ? RT_Debug("AutoCrop Found no borders") : NOP
(AUTOLEVEL)? RT_Debug("AutoLevel NonCropped : MinLuma=",String(QLMMMin)," MaxLuma =",String(QLMMMax)) : NOP
}
RT_Debug("MAX Width So Far =",String(MAXW),"MAX Height So Far =",String(MAXH))
RT_Debug(" ----------------------------- ") # line separator
}
}
maxFAR = Float(MAXW) / MAXH
if(FAR >= maxFAR) {
# Required Frame Aspect Ratio greater or same as maximum accumulated dimensions. Need to pad horizontal
if(SzLimit == 0) { # NO SIZE LIMITING, Pad Only
canvasheight = Int((MAXH + (HMOD/2)) / HMOD) * HMOD
canvaswidth = Int((canvasheight * FAR + (WMOD/2)) / WMOD) * WMOD
} else { # Resize with SIZE LIMITING
canvasheight = (MAXH < SzLimit) ? Int((MAXH + (HMOD/2)) / HMOD) * HMOD : MAXH
canvasheight = (canvasheight > SzLimit) ? SzLimit : canvasheight # enforce SzLimit max
canvaswidth = Int((canvasheight * FAR + (WMOD/2)) / WMOD) * WMOD
}
} else { # Need to pad vertical
if(SzLimit == 0) { # NO SIZE LIMITING, Pad Only
canvaswidth = Int((MAXW + (WMOD/2)) / WMOD) * WMOD
canvasheight = Int((canvaswidth / FAR + (HMOD/2)) / HMOD) * HMOD
} else { # Resize with SIZE LIMITING
canvaswidth = (MAXW < SzLimit) ? Int((MAXW + (WMOD/2)) / WMOD) * WMOD : MAXW
canvaswidth = (canvaswidth > SzLimit) ? SzLimit : canvaswidth # enforce SzLimit max
canvasheight = Int((canvaswidth / FAR + (HMOD/2)) / HMOD) * HMOD
}
}
if(DEBUG){RT_Debug("SETTING CanvasWidth =",String(canvaswidth)," CanvasHeight =",String(canvasheight))}
if(DEBUG){RT_Debug(" ----------------------------- ")} # line separator
For(i=0,FILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic/vid file i
if(DEBUG){RT_Debug(string(i+1)+"/"+String(FILES)+")","Processing File",FN)}
if(VID) {
k=DirectshowSource(FN).ConvertToYUY2().Trim(0,0)
} else {
k=ImageReader(FN,end=0).ConvertToRGB32() # RGB24 problems with addborders
}
OrgH=k.height OrgW=k.width

if (AUTOLEVEL) {
# We do AUTOLEVELS before crop, because of SHOWAUTOLEVEL
QAT = RT_TxtGetLine(AUTOLEVELS,i)
Eval(QAT)
if(k.IsYUV()) {
CSMin = 16
CSMax = 235
} else {
CSMin = 0
CSMax = 255
}
TMP_K=k
ALMin = Int(CSMin - ((CSMin - QLMMMin) * AUTOLEVEL_STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * AUTOLEVEL_STRENGTH)) # Round down
k = k.Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DONT EVER USE Coring
(DEBUG) ? RT_Debug("Levels("+String(ALMin)+",1.0,"+String(AlMax)+","+String(CSMin)+","+String(CSMax)+",Coring=False)") : NOP

if(SHOWAUTOLEVEL) {
tmpw=int((k.width/4)*2)
Left = TMP_K.Crop(0,0,tmpw,-0)
Right= k.Crop(tmpw,0,-0,-0)
k=StackHorizontal(Left,Right)
}
}
if (AUTOCROP) {
QBC = RT_TxtGetLine(CROPCOORDS,i) # Get required coords from first pass
Eval(QBC) # Set variables in Avisynth, ie CropX etc
k=k.Crop(CropX,CropY,CropW,CropH)
(DEBUG)? RT_Debug("Crop("+String(CropX)+","+String(CropY)+","+String(CropW)+","+String(CropH)+")") : NOP
}
iWidth=k.Width() iHeight=k.Height()
cFAR = Float(canvaswidth) / canvasheight
iFar = Float(iWidth) / iHeight
Left=0 Top=0 Right=0 Bot=0

if(SzLimit==0) {
Left = (canvaswidth-iWidth) / 2
Right = canvaswidth-iWidth - Left
Top = (canvasheight-iHeight) / 2
Bot = canvasheight-iHeight-Top
owid = iWidth
oHit = iHeight
(DEBUG) ? RT_Debug("AddBorders("+String(Left)+","+String(Top)+","+String(Right)+","+String(Bot)+")") : NOP
k = k.Addborders(Left,Top,Right,Bot,BORDER)
} else {
if(cFAR >= iFAR) { # Canvas Aspect ratio greater (proportionally wider), resize to vertical and pad horizontal
oHit = canvasheight
oWid = Int(oHit*iFAR/4)*4
Left = (canvaswidth-oWid) / 2
Right = canvaswidth - oWid - Left
} else { # Canvas Aspect ratio smaller (proportionally taller), resize to horizontal and pad vertical
oWid = canvaswidth
oHit = Int(oWid/iFAR/4)*4
Top = (canvasheight-oHit) / 2
Bot = canvasheight - oHit - Top
}
if(DEBUG) {
RT_Debug("Spline36Resize("+String(oWid)+","+String(oHit)+")")
RT_Debug("AddBorders("+String(Left)+","+String(Top)+","+String(Right)+","+String(Bot)+")")
}
k = k.Spline36Resize(oWid,oHit).Addborders(Left,Top,Right,Bot,BORDER)
}
k=k.assumefps(fps)
if(SUBS) {
k=k.subtitle(String(i) + "/"+String(FILES-1) +" " + FN)
k=k.subtitle("Orig width=" + string(orgW) +" " + "Orig height=" + string(orgH) +" FAR="+String(Float(orgW) /orgH, "%.2f"),20,20)
if (AUTOCROP) {k=k.subtitle("Crop width=" + string(iWidth) +" " + "Crop height=" + string(iHeight) +" FAR="+String(Float(iWidth) /iHeight, "%.2f"),20,40)
} else {k=k.subtitle("NO AUTOCROP",20,40)}
k=k.subtitle("New width=" + string(oWid) +" " + "New height=" + string(oHit)+" FAR="+String(Float(oWid)/oHit,"%.2f"),20,60)
k=k.subtitle("Out width=" + string(k.Width) +" " + "Out height=" + string(k.Height)+" FAR="+String(Float(k.Width)/k.Height,"%.2f"),20,80)
if(AUTOLEVEL) {
k=k.subtitle("QLMMMin = " + String(QLMMMin) + " QLMMMax = " +String(QLMMMAx),20,120)
k=k.subtitle("AUTOLEVEL @ " + String(AUTOLEVEL_STRENGTH*100.0,"%.0f") + "%",20,140)
k=k.subtitle("Auto Levels(" + String(ALMin) + ",1.0," + String(ALMAx) + "," + String(CSMin) + "," + String(CSMax) + ")",20,100)
} else {k=k.subtitle("NO AUTOLEVEL",20,100)}
}
picclip = (IsClip(picclip)) ? picclip ++ k : k # Append to clip so far
if(!VID){k=k.ConvertToRGB24()}
if(DEBUG){RT_Debug(" ") RT_Debug(" ----------------------------- ")} # line separator

}
""")

PicClip
return Last

Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Original:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
#{i=s.FindStr(find)return(i==0?s: s.LeftStr(i-1)+replace+s.MidStr(Strlen(find)+i).StrReplace(find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{i=s.FindStr(find) return(i==0?s:RT_StrAddStr(s.LeftStr(i-1),replace,s.MidStr(Strlen(find)+i).StrReplace(find,replace)))}

Function StrReplaceDeep(string s,string find,string replace) # DEEP Repeated, string replacements
# When replacing eg pairs of SPACE characters by single SPACE, StrReplace can leave some pairs non replaced.
# Assumes that you want to rescan where replaced string could constitute part of a new find substring,
# Length of replace must be shorter than length of find, else will call StrReplace instead.
# Based On:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
# {i=s.FindStr(find)return i==0?s:s.LeftStr(i-1)+( strlen(replace)<strlen(find)? \
# StrReplaceDeep(replace+s.MidStr(Strlen(find)+i),find,replace) : replace+StrReplace(s.MidStr(Strlen(find)+i),find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{i=s.FindStr(find)return i==0?s:RT_StrAddStr(s.LeftStr(i-1),( strlen(replace)<strlen(find)? \
StrReplaceDeep(RT_StrAddStr(replace,s.MidStr(Strlen(find)+i)),find,replace) : \
RT_StrAddStr(replace,StrReplace(s.MidStr(Strlen(find)+i),find,replace))))}



Updated, needs RT_Stats 1.06

EDIT: See PlanetCrop() available in RT_Stats zip (much better for this purpose).

[StainlessS]

Modified previous post.

[jmac698]

You're amazing, thanks for the effort! I really think that is a useful script though, there is a need in astrophotography sometimes to crop out and track planets. That's because our software, like any other, has to search the entire image to align frames, and it's very slow, and can be beyond the search radius for matching. So this type of "pre-alignment" really speeds up the programs.

I can also see me making use of this in moviemaking, to keep the frame centered on an object just by mounting a light on it. It can be used to deshake in certain situations.

[StainlessS]

The Video Joining capability of the script is pretty handy too.

I'll implement the QueryBorderCrop mod today and do the Martin53 requested mod to RT_Stats and up within a couple of hours.

(And you said that QueryBorderCrop() would never work, hehehehe) :)

[jmac698]

Ok, I've tried it now, and it fails on frame 13. It also seems like it's being resized.

[martin53]

... and do the Martin53 requested mod to RT_Stats ...

don't hurry! Btw., I was able to write a 'proof of concept':

#==========================================
# Luma2argb puts the average luma of clip 'cMeasured' into the ARGB bytes of container clip
#
function Luma2argb(clip cMeasured) {
cContainer = blankclip(cMeasured, pixel_type="RGB32", width=1, height=1)
return cContainer.GScriptClip("""
longVal = round($1000000*cMeasured.RT_AverageLuma(Matrix=2)) #longVal is supposed to be 4 bytes long: $aa,rr,gg,bb.
return Blankclip(last, color=longVal) #Put high byte to alpha, next bytes to red,gren,blue: easy because color parameter of function BlankClip does just this!)
""", args="cMeasured")
}

#==============================================
# Argb2float returns a float value from ordered alpha, red, green and blue channels in the range 0...255.99999999
# inputs: clip, frame delta and current frame numbers
# output: quality
#
function Argb2float(clip c, int delta, int currentframe) {
a = c.ShowAlpha().RT_AverageLuma(n=currentframe, delta=delta, Matrix=2)
r = c.ShowRed().RT_AverageLuma(n=currentframe, delta=delta, Matrix=2)
g = c.ShowGreen().RT_AverageLuma(n=currentframe, delta=delta, Matrix=2)
b = c.ShowBlue().RT_AverageLuma(n=currentframe, delta=delta, Matrix=2)
return a + r/$100 + g/$10000 + b/$1000000
}

c = something...
cAL = c.Luma2argb() #cAL holds the AverageLuma of C with 32bit res.
c.GScriptClip("""
subtitle(string(cAL.Argb2float(1, current_frame))) #displays AverageLuma of following frame :) :) :)
""", args="cAL")

[StainlessS]

RT_Stats v1.06, see 1st post.

Jmac, Post #70 updated, still playing with it, converting to CropExact has some other probs that dont quite work.

I'll try to mod for your purpose in addition to current usage.

The original script is intended to do resizing, its function is to join dissimilar pics/vids together and resizing/addborders to a required
aspect ratio and size limit. Going to the pub now, and as I have been up since about 2:00AM, probably not do anything else today.

Martin53, have updated RT_stats with append update, the RGB stuff was what I was pondering.

Dont think a good idea as suggested, AverageLuma is float return and your suggested func would not be precise enough for
uses that it may be put to. Anyways, take a look at Showchannels updated today, easily used with scriptclip.

Cheers. :)

EDIT: The pubs shut way too early.

[StainlessS]

Jmac, post #70 updated, added No Resize functionality by setting SzLimit=0

notes here (too big for post 70, overflows limit) :

# Single Frame Super Sensitive, switched on when samples == 1 or single frame, intention to make as sensitive as possible hopefully without overcrop.
# Single frame Super Sensitive AutoThresh (arg Thresh = -ve), internally sets AutoThresh to -0.5, ie relative to YPlaneMin, == YPlaneMin + 0.5.
# Single frame Super Sensitive AutoThresh would have worked fine, was the halfway scan (ie SCANPERC==50.0%) that inhibited detection.
# YPlaneMin=0 All images, so AutoThresh of Little Use, Equivalent to Thresh = 0.5 on all frames.
# With SzLimit == 0 (NO RESIZING, Crop & AddBorders Only, and FAR == 1.0 [Frame Aspect Ratio 1:1])
# 0.5 keeps good margin all around without coming close to chopping off any Moon (954x954).
# 0.6 Close but clear margin Top, rest, bigger margin (922x922)
# 0.75 Just about touches top of moon, other sides good margin (862x862)



Still working on it but is pretty good right now, try with AUTOCROP_THRESH = 0.6

Going to try to implement keeping some of the margins instead of adding synthetic borders, to fit the
required aspect ratio (ie partially ignoring the crop coords where possible) to keep as much of the original
image as we can.

There is a problem (only with your type task). You will see that it detects very close to top/bottom of
the moon, but quite a margin left/right. This is due to the fact that we scan top and bot to find borders,
and then left and right, BUT, left/right scan only scans between the already established top and bottom
borders. With the top/bot scan a small number of lighter pixels in a vast expanse of black line requires
a very low threshold, but when we scan a much reduced vertical line length in the lft/rgt scan
(due to limiting to top/bot borders), so any approx same number or lighter pixels in the much reduced
sample line length does not require such a small thresh.
I shall try to rectify this, perhaps by adding some kind of threshold scaler based on scanline length
compared with perhaps full image width to try and achieve a comparable thresh.

Give it a whirl, works great and getting better.

[martin53]

the RGB stuff ... Dont think a good idea as suggested

StainlessS,
never mind. I think similar - it would be not generic as suggested. The 'proof of concept' works excellent. We could still get back to it and maybe handle exponent and signs of mantissa and exponent in a way that would really allow a floating point number, and maybe use 2 pixels to store 64 bits - if it fulfills a certain need better than other solutions.

:thanks: for the 'append'. Cheers - enjoy the pub!

[StainlessS]

Can always implement an eg RGB_AverageR etc, returning all three at once possible in string but would not really want to encourage scanning every pixel of a clip especially as strings are never released until close. Guess RGB24 could be returned in int, but not RGB32 which I'm guessing might produce some kind of overflow error when trying to extract alpha in script (maybe not, vb would overflow).

Might even be possible to take some args and manufacture a float.
(assume for your hinting thing).

Cheers - enjoy the pub!

and Cheers to you sir, I always enjoy the pub, on my way presently. :)

[StainlessS]

@Martin53,
Hows about this. Func to return combined rgb data (as for single clip Luma funcs, excluding comparison funcs), for all channels,
eg ave, max, min, YInRange, YPlaneMedian, YPlaneMinMaxDifference, all as Global Variables, with a user supplied Prefix.

Should be reasonably fast and no string accumulation probs. A Similar luma func could also be of use, and avoid multiple
func calls in some situations. Implemented as for RT_ luma funcs with eg laced arg and x,y,w,h.

EDIT:
@Jmac, got both "keep margin" and "thresh scaling" done, works brilliant, just a bit of tidying up to do.
Here's something to try.
EDIT: Link Removed.

[jmac698]

Tried it, working great! Wonder why it jumps a little though, did you see something in the background to mess it up?

[jmac698]

Ok,
I've tried another example and now I'm having a lot of trouble. I can't get it to work at all, and when changing around some parameters, sometimes it makes a letterbox on the video, I don't know why.
You've done so much, but it would be great if you could help me figure out this one:
http://www.sendspace.com/file/f4qh1v

[StainlessS]

Wonder why it jumps a little though

Who knows, maybe starlight making minor differences.
The moon is a little dark at the top and hence the close cropping there, and the bottom has
a bit of a halo around it hence the not quite so close cropping.
It's not a de-jitterer, it just crops when the scaled thresh is broken for 3 consecutive scanlines (h/v).

Shall download the 2nd sample and have a play.

PS, the "Insert Link" button works fine in Quick Reply.

[martin53]

@Martin53,
Hows about this. Func to return combined rgb data (as for single clip Luma funcs, excluding comparison funcs), for all channels,
eg ave, max, min, YInRange, YPlaneMedian, YPlaneMinMaxDifference, all as Global Variables, with a user supplied Prefix.

If you don't mind, please have a look at the 'proof of concept' code.
The intention and use case I had in mind is this:
You need a number that is frame specific. Today, you have the options

- Store it in a text file in a prior pass and read it in, e.g. with ConditionalReader() or with RT_Stats.
____Not so nice since you need multiple passes and have to create (unique) file names.
- Evaluate it directly from where it comes from.
____Slow and memory consuming if the evaluation is complicated.
- Use a veeerrry long string as comma separated list and Eval("Select(..." it.
____Not applicable to my experience with really long clips. Slow, I suspect.

The 'new' idea is, to store a floating point variable in a clip, because the clip has the neccessary property to be an 'array', the frame number being the index into the array, and AviSynth's frame cache will provide fast access to several elements of the array, as long as they are accessed frequently before being kicked out of the cache.

I started with 32 bits resolution and fixed point (range 0...256) for the variable. Of course, a real floating point variable would rather have bits for the signs of mantissa end exponent and fields of some width to store mantissa and exponent themselves.

What I really would like to point out, is, that one argb pixel can really be regarded as one variable. With two pixels of rgb24 etc, finer resolutions are easy to think of. I dont know what the AviSynth resolution of type float is. But the typical 6 digit display does not need more than 20 bits for the mantissa, so another 10 bits would be free for the exponent, thus allowing -1,048,576e-1024...1,048,575e1023 with an accurracy of 6 digits to be stored in the 1x1argb clip. A pair of functions that just stored the bits from a float variable into the - say - 8 bytes of a 1x2 argb clip and retrieved it to a float variable again would make a fine interface to use the kind of 'array of float' that I decribed above.

[jmac698]

Yeah, the same idea is used in avsLib where this is all done for you. Or you could move to Vapoursynth.

[StainlessS]

Martin53,


Function Luma2argb(clip cMeasured) {
cContainer = blankclip(cMeasured, pixel_type="RGB24", width=1, height=1)
return cContainer.GScriptClip("""
FV=cMeasured.RT_AverageLuma(Matrix=2)
longVal = round($10000*FV) #longVal is supposed to be 3 bytes : $rr,gg,bb.
RT_Debug("FV="+String(FV),"LONGVAL="+String(LongVal))
return Blankclip(Last, color=longVal) #Put high byte to red,then green,blue
""", args="cMeasured")
}

Function Argb2float(clip c, int delta, int currentframe) {
r = c.RGB_ChanAve(n=currentframe, delta=delta, chan=0)
g = c.RGB_ChanAve(n=currentframe, delta=delta, chan=1)
b = c.RGB_ChanAve(n=currentframe, delta=delta, chan=2)
RT_Debug("R="+String(R),"G="+String(G),"B="+String(B))
return r + g/$100 + b/$10000
}

c = ColorBars().Trim(0,-2) # With Matrix=0(rec601)==97.900780 : Matrix=2(pc601)=95.287369
cAL = c.Luma2argb() #cAL holds the AverageLuma of C with 24bit res.
c.GScriptClip("""
subtitle(string(cAL.Argb2float(1, current_frame))) #displays AverageLuma of following frame :) :) :)
""", args="cAL")
return last


Above uses only RGB24, 24 bits, same as mantissa in Avisynth float.
Note, Max value return by AverageLuma & RT_AverageLuma is 255.0 not 255.99999

I knocked up a temp func, RGB_ChanAve() as for RT_AverageLuma but without the matrix arg but additional chan arg (0-3==RGBA [0==R]).
EDIT: Link Removed.

In original script, dont think I noticed any value in the blue channel at all (other than 0).

Here a Jmac thread on float/int conversion:
http://forum.doom9.org/showthread.php?t=163038&highlight=mantissa

EDIT: You might want to edit the func names (remove the "A"), may or may not want to pack into RGB32.

[martin53]

StainlessS,
I appreciate RGB_ChanAve()! Thank you!

If I asked you if you can make a function that just 'typecasts' float-->RGBA and another one that 'typecasts' RGBA-->float, i.e the first takes a float and returns a valid 'color' in range 0 ...$ff.ff.ff.ff without further calculation, just the 4 bytes, the 2nd one is like RGB_ChanAve, say 'RGB_2float()' and directly returns type float from the 4 channels of the pixel, would you like to help me with that?

Edit: assume to only allow RGB_2float(w=1, h=1)

[martin53]

Yeah, the same idea is used in avsLib
... with that, you mean the very long string, right? Then, as only one matter, the string concatenation issue of AviSynth 2.6 affects AVSLib too, and makes the whole lib practically useless.

Or you could move to Vapoursynth.
Vapoursynth interests me very much, but I didn't take time to read more thoroughly through the doc. Can you please say a word about runtime environment in Vapoursynth ('ScriptClip()' etc). Is that supported?

[StainlessS]

('ScriptClip()' etc). Is that supported?

So far as I know, Avisynth script is not supported and so neither would ScriptClip be. (leastwise not yet)
But perhaps a Vapoursynth similar func is available.

(gonna sleep a few days and come back and read other posts).

[StainlessS]

@Martin53,
Done what you require but adding it and other stuff to RT_Stats (eg RT_BitAND, RT_BitSet, RT_Hex).
I've implemented as RT_RgbChanAve, RT_RGB32AsFloat, and RT_FloatAsRGBA, if you dont like names,
then suggest others.
RT_RGB32AsFloat() given args similar to eg RT_AverageLuma excluding w,h interlaced and matrix, so you could
store a two dimensional array of float using StackHorizontal/Vertical (presume thats what you wanted).

@Jmac,

Gonna have another crack at your moon prob soon as I get this RT_Stats stuff out of the way, need to
attack from a different direction.

EDIT:
so you could store a two dimensional array of float using StackHorizontal/Vertical.


Three dimensional if you count frame number.

[martin53]

RT_RgbChanAve, RT_RGB32AsFloat, and RT_FloatAsRGBA
magnificent! :thanks:
I am going to test it tomorrow or Thursday.

[StainlessS]

Temporary version RT_Stats 1.07beta.

New funcs noted here:

RT_Stats Avisynth Plugin by StainlessS

RT_RgbChanAve(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0, int "h"=0,bool "interlaced"=false,int "chan"=0)
Return the average value for an RGB channel (RGB 24/32). (default = 0 = R [RGBA])


RT_FloatAsRGBA(float)
Given a float arg, returns an int formatted as for use as a color to eg BlankClip.
Can later be recovered from that clip using RT_RGB32AsFloat().


RT_RGB32AsFloat(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0)
Given an RGB32 clip that had a pixel value created via RT_FloatAsRGBA(), gets that pixel and returns as the
original float given to RT_FloatAsRGBA. By creating single pixel clips and stacking them horizontally/vertically,
you can use frames as a two dimensional array of float, perhaps use the frame number for a third dimension.


RT_GraphLink(clip source, clip c1, ... , clip cn, bool b1, ... , bool bn)
Standard filter function.
This function is a standard filter graph function, not a runtime/compile time function.
It takes a compulsory source clip which it will return unchanged.
The clips, c1 to cn (one or more, at least one of them) are by default forcibly linked into
the filter graph. The bools b1 to bn are optional (zero or more) and would default to True
if not supplied. These bools will if false, switch OFF the forcible linking into the filter graph
for the corresponding clip c1 to cn.
Avisynth does not normally process any filter chains that do not contribute to the output clip,
this filter allows you to select graph chains that you wish to forcibly process.
The usual way to force process an unused chain is to do a stackHorizontal/Vertical and a crop.
All args un-named.
Example: [will only render the TestA" and TestC images]
a=ImageSource("d:\avs\1.jpg",end=0) # Single frame
TmpA = a.subtitle("TestA").ImageWriter("d:\avs\TestA_", type="png") # Written
TmpB = a.subtitle("TestB").ImageWriter("d:\avs\TestB_", type="png") # Skipped
TmpC = a.subtitle("TestC").ImageWriter("d:\avs\TestC_", type="png") # Written
RT_GraphLink(a,TmpA,TmpB,TmpC,True,false) # 3rd bool defaults true
return Last


RT_Hex(int , int "width"=0)
First arg is an integer to convert to a hexadecimal string.
'width' is the minimum width of the returned string.
eg RT_Hex(255,4) returns "00FF".


RT_HexValue(String)
Returns an int conversion of the supplied hexadecimal string.
Fixes a HexValue bug in 2.58 & 2.6a3.


RT_BitNOT(int)
Returns the integer result of a bitwise NOT operation on it's int argument.


RT_BitAND(int, int)
Returns the integer result of a bitwise AND operation on the two int arguments.


RT_BitOR(int, int)
Returns the integer result of a bitwise OR operation on the two int arguments.


RT_BitXOR(int, int)
Returns the integer result of a bitwise XOR operation on the two int arguments.


RT_BitLSL(int, int)
Returns the integer result of a Logical Shift Left on the first integer, by a shift count of the second int.
An Arithmetic Shift Left and a Logical Shift Left, are essentially the same.


RT_BitLSR(int, int)
Returns the integer result of a Logical Shift Right on the first integer, by a shift count of the second int.
Logical shift right will shift zeros into the high bits.


RT_BitASL(int, int)
Returns the integer result of an Arithmetic Shift Left on the first integer, by a shift count of the second int.
An Arithmetic Shift Left and a Logical Shift Left, are essentially the same.


RT_BitASR(int, int)
Returns the integer result of an Arithmetic Shift Right on the first integer, by a shift count of the second int.
Arithemtic shift right will shift into the high bits, zeros for a +ve number and ones for a -ve number, so -ve
numbers will stay -ve and +ve stay +ve.


RT_BitTST(int, int)
Returns the bool result of a bit test on first integer, the bit number is given in the second int.
A zero in the bit returns False and a 1 returns True.


RT_BitCLR(int, int)
Returns the integer result of a bit clear (make 0) on first integer, the bit number is given in the second int.


RT_BitSET(int, int)
Returns the integer result of a bit Set (make 1) on first integer, the bit number is given in the second int.


RT_BitCHG(int, int)
Returns the integer result of a bit change (0->1, 1->0) on first integer, the bit number is given in the second int.


RT_BitROR(int, int)
Returns the integer result of a bit Rotate Right on first integer, the bit rotate count is given in the second int.


RT_BitROL(int, int)
Returns the integer result of a bit Rotate Left on first integer, the bit rotate count is given in the second int.

http://www.mediafire.com/?lejdpznk631hdnd


Martin53, Testem all out if you can, Not had much testing here, done all of the funcs bar RT_RgbChanAve()
today, and I'm a bit whacked out.

EDIT: You interested in storing ints in a clip too ?

EDIT:

so you could store a two dimensional array of float using StackHorizontal/Vertical.
Three dimensional if you count frame number.

With a little thought you could have 4 dimensions using n and delta.

EDIT: Have now implemented RT_RGB32AsInt(), no need for a sister func.

[StainlessS]

Jmac, see below PlanetCrop() works great, even with your 2nd moon image set.


#import("D:\AVS\BANK\QueryLumaMinMax.avsi")

Fn="D:\MOON\*.JPG|BMP"
BAFFLE = 12 # Number of consecutive scanlines that need to pass, below considered noise.
BLK = 32 # Where black sky can vary in brightness
GROW_W = 256 # Enlarge canvas width by GROW_W pixels (keep surround area, Use with KEEP_MARGIN)
GROW_H = 256 # Enlarge canvas height by GROW_H pixels
FAR = 1.0 # Frame Aspect Ratio
STRENGTH = 1.0 # AutoLevel Strength 0.0 -> 1.0
SHOWAL = True # Right Hand Side Only AutoLeveled
KEEP = true # True tries to keep some of the cropped area instead of AddBorders
SHOWKEEP = true # Show Kept margin
SHOWBORDER = true # True Show replaced border in Red, else Black. (FAR affects) (Test CROP)
DEBUG = True


Return PlanetCrop(Fn,BAFFLE,BLK,GROW_W,GROW_H,FAR,STRENGTH,SHOWAL,KEEP,SHOWKEEP,SHOWBORDER,DEBUG)

#--------------------------------------------------------------------------------
Function PlanetCrop(String "FileName",int "Baffle",Int "BlackRange",int "Grow_W",int "Grow_H",Float "FAR", \
Float "AutoLevelStrength",bool "ShowAutoLevel",bool "KeepMargin",bool "ShowMargin",bool "ShowBorder",bool "Debug") {
# PlanetCrop v1.0 - 16 Dec 2012
#
# Requires GScript, RT_Stats and QueryLumaMinMax.avs
#
# Given a series of time lapse planetoid images, locates those images in frame, crops out and joins them all together into
# a video clip of required Frame Aspect Ratio, 1 FPS, with optional Auto Luma Levelling.
# First scans each image establishing maximum dimensions for whole sequence (after cropping out the planets). Optionally 'grows' those
# dimensions to maintain some of the area around the cropped planets. A 'canvas' size is then established to contain the now maximum
# dimensions and respecting the desired Frame Aspect Ratio, synthetic black borders are added where necessary.
# Borders are the areas added to maintain aspect ratio and can be hi-lited in red. Margins are areas that use the
# original image content instead of synthetic black borders (where possible), they replace synthetic borders, and can be hi-lited
# in green. Margins are the areas between the established crop coords and synthetic borders.
# --------------------------------------
# Args:-
# Filename: Default "*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF". Image Path and wildcard with mulitple pipe separated extensions.
# Baffle: Default 12. Number of scanlines that must break a threshold to be accepted as 'planet'. Avoids noise.
# BlackRange: Default 32. Where background (sky) can vary in luminance by a significant amount.
# Grow_W,Grow_H: Default 128. Size in pixels by which to 'grow' the accumulated maximum dimensions once all planets have been identified.
# FAR: Default 4.0/3.0. Frame Aspect Ratio.
# AutoLevelStrength: Default 0.0. Auto luma leveling, range 0.0 -> 1.0. 0.0 is OFF.
# ShowAutoLevel: Default False. If true then shows Auto leveling on right hande side of image ONLY.
# KeepMargin: Default True. When padding to maintain Frame Aspect Ratio, tries to keep image instead of adding borders.
# ShowMargin: Default False. If True, shows kept margins in Green.
# ShowBorder: Default False. When padding to maintain Frame Aspect Ratio, borders are shown in red.
# Debug: Default False. Show debug info via DebugView. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
#
VERS="PlanetCrop v1.0 - 16 Dec 2012"
FileName= Default(FileName,"*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF")
Baffle=Default(Baffle,12)
BlackRange=Default(BlackRange,32)
Grow_W=Default(Grow_W,128)
Grow_H=Default(Grow_H,128)
FAR=Float(Default(FAR,4.0/3.0))
AutoLevelStrength=Float(Default(AutoLevelStrength,0.0))
ShowAutoLevel=Default(ShowAutoLevel,false)
KeepMargin=Default(KeepMargin,True)
ShowMargin=Default(ShowMargin,False)
ShowBorder=Default(ShowBorder,False)
DEBUG=Default(DEBUG,False)
# Lesser changed stuff Hardcoded
BORDER = (ShowBorder) ? $FF0000 : $000000 # Color for replaced borders.
MARGINBORDER= $00FF00 # Color for KEEP_MARGIN borders.
DEBUG_QLMM = False # Debug QueryLumaMinMax
FPS = 1.0 # Whatever.
WMOD = 1
HMOD = 1
OUT_FILE = "PLANET.List"
Assert(Baffle>0,"Baffle must be 1 or more")
Assert(BlackRange>=0,"BlackRange must be 1 or more")
Assert(Grow_W>=0,"Grow_W Cannot be -ve")
Assert(Grow_H>=0,"Grow_H Cannot be -ve")
Assert(FAR>0.0,"FAR must be greater than 0.0")
Assert(AutoLevelStrength>=0.0 && AutoLevelStrength <= 1.0,"AutoLevelStrength must be 0.0 -> 1.0")
#
GScript("""
Function QPC(clip c,int Baffle) {
c
Thresh=0.04 # 4%, Baffle should remove need to change
w=Width h=height
got = 0 x1 = 0 x2 = w-1 Cnt=0
For(x=0,w-1) {
th=RT_YInRange(0,x=x,y=4,w=1,h=-4,lo=2,hi=255,Matrix=2) # PC levels and skip end 4 pixels (crud around frame edge)
if(got==0) {
if(th > Thresh) {
Cnt=Cnt+1
if(Cnt >= Baffle) {
x1=x-Baffle+1
Cnt=0
got=1
}
} else {
Cnt = 0
}
} else {
if(x2 == w-1 && th <= Thresh) {
Cnt=Cnt+1
if(Cnt >= 2) { # Weak BAFFLE on transition to black
x2=x-2
x=w
}
}
}
}
got=0 y1 = 0 y2 = h-1 Cnt=0
For(y=0,h-1) {
th=RT_YInRange(0,x=4,y=y,w=-4,h=1,lo=2,hi=255,Matrix=2)
if(got==0) {
if(th > Thresh) {
Cnt=Cnt+1
if(Cnt >= Baffle) {
y1=y-Baffle+1
Cnt=0
got=1
}
} else {
Cnt = 0
}
} else {
if(y2 == h-1 && th <= Thresh) {
Cnt=Cnt+1
if(Cnt >= 2) { # Weak BAFFLE on transition to black
y2=y-2
y=h
}
}
}
}
Return "PC_X="+String(x1)+" PC_Y="+String(y1)+" PC_W="+String(x2-x1+1)+" PC_H="+String(y2-y1+1)
}
#
if(DEBUG){ RT_Debug(" ") RT_Debug(VERS,"- By StainlessS") RT_Debug(" ") }
picclip = 0 # Dummy for now
result=RT_WriteFileList(FileName,OUT_FILE) # Create a listing file of Pic files
Assert((result!=0), "No files found")
FILELIST=RT_ReadTxtFromFile(OUT_FILE) # Get list of files
NFILES=RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
(DEBUG) ? RT_Debug("Found files = " + String(NFILES)) :NOP
# Prescan to ascertain max wid/hit of all files, also Autocrop & Auto Levels
MAXH=0 MAXW=0 # init max wid and hit
CROPCOORDS="" # Clear results array (string)
AUTOLEVELS="" # Clear AutoLevels array (string)
AUTOLEVEL=(AutoLevelStrength>0.0) # Adjust using Levels.
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic/vid file i
(DEBUG)?RT_Debug(String(i+1)+"/"+String(NFILES)+") Getting File dimensions",FN):NOP
k=ImageReader(FN,end=0).ConvertToRGB32()
OrgW=k.width OrgH=k.height
# For QPC
Grey=k.GreyScale()
MinLum=Grey.RT_YPlaneMin(0,threshold=0.2,Matrix=2)
Grey=Grey.Levels(MinLum+BlackRange,1.0,MinLum+BlackRange+1,0,255,Coring=false)
#
PCS=Grey.QPC(Baffle)
Eval(PCS)
k=k.Crop(PC_X,PC_Y,PC_W,PC_H)
CROPCOORDS=RT_TxtAddStr(CROPCOORDS,PCS)
if (AUTOLEVEL) {
QLMM = k.QueryLumaMinMax(debug=(DEBUG&&DEBUG_QLMM)) # Query luma levels (on cropped clip)
Eval(QLMM) # Set Avisynth vars
AUTOLEVELS=RT_TxtAddStr(AUTOLEVELS,QLMM) # Avisynth Bugfix, same as AUTOLEVELS = AUTOLEVELS + QLMM + Chr(10)
}
If(MAXW<PC_W) {MAXW=PC_W}
If(MAXH<PC_H) {MAXH=PC_H}
if (DEBUG) { # KEEP main debug messages together
RT_Debug ("Orig Width =",String(orgW),"Orig Height =",String(OrgH))
if((PC_W != OrgW) || (PC_H != OrgH)) { # Was Caused by AUTOCROP
RT_Debug("Crop Width =",String(PC_W),"Crop Height =",String(PC_H))
}
RT_Debug("MAX Width So Far =",String(MAXW),"MAX Height So Far =",String(MAXH))
(AUTOLEVEL)? RT_Debug("AutoLevel MinLuma=",String(QLMMMin)," MaxLuma =",String(QLMMMax)) : NOP
RT_Debug(" ----------------------------- ") # line separator
}
}
if(Grow_W>0 || Grow_H > 0) {
(DEBUG)?RT_Debug("Enlarging Max Width,Height to enlarge canvas"):NOP
MAXW=MAXW+Grow_W
MAXH=MAXH+Grow_H
}
maxFAR = Float(MAXW) / MAXH
if(FAR >= maxFAR) {
# Required Frame Aspect Ratio greater or same as maximum accumulated dimensions. Need to pad horizontal
canvasheight = Int((MAXH + (HMOD/2)) / HMOD) * HMOD
canvaswidth = Int((canvasheight * FAR + (WMOD/2)) / WMOD) * WMOD
} else { # Need to pad vertical
canvaswidth = Int((MAXW + (WMOD/2)) / WMOD) * WMOD
canvasheight = Int((canvaswidth / FAR + (HMOD/2)) / HMOD) * HMOD
}
if(DEBUG){
RT_Debug("SETTING CanvasWidth =",String(canvaswidth)," CanvasHeight =",String(canvasheight))
RT_Debug(" ----------------------------- ") # line separator
}
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic/vid file i
if(DEBUG){RT_Debug(string(i+1)+"/"+String(NFILES)+")","Processing File",FN)}
k=ImageReader(FN,end=0).ConvertToRGB32()
OrgW=k.width OrgH=k.height
Eval(RT_TxtGetLine(CROPCOORDS,i))
Left = (canvaswidth-PC_W) / 2 # Borders
Right = canvaswidth-PC_W - Left
Top = (canvasheight-PC_H) / 2
Bot = canvasheight-PC_H-Top
Left2=0 Right2=0 Top2=0 Bot2=0 # Init keep margin borders
if(KeepMargin) {
Left2 = ((PC_X<Left) ? PC_X : Left) / WMOD *WMOD
if(Left2>0) {
Left=Left-Left2
}
Right2 = (((OrgW-PC_X-PC_W)<Right) ? (OrgW-PC_X-PC_W) : Right) / WMOD * WMOD
if(Right2>0) {
Right=Right-Right2
}
Top2 = ((PC_Y<Top) ? PC_Y : Top) / HMOD *HMOD
if(Top2>0) {
Top=Top-Top2
}
Bot2 = (((OrgH-PC_Y-PC_H)<Bot) ? (OrgH-PC_Y-PC_H) : Bot) / HMOD * HMOD
if(Bot2>0) {
Bot=Bot-Bot2
}
if(!ShowMargin) {
PC_X=PC_X-Left2
PC_W=PC_W+Left2+Right2
PC_Y=PC_Y-Top2
PC_H=PC_H+Top2+Bot2
}
}
(DEBUG)? RT_Debug("Crop("+String(PC_X)+","+String(PC_Y)+","+String(PC_W)+","+String(PC_H)+")") : NOP
k=k.Crop(PC_X,PC_Y,PC_W,PC_H)
if (AUTOLEVEL) {
QAT = RT_TxtGetLine(AUTOLEVELS,i)
Eval(QAT)
if(k.IsYUV()) {
CSMin = 16
CSMax = 235
} else {
CSMin = 0
CSMax = 255
}
TMP_K=k
ALMin = Int(CSMin - ((CSMin - QLMMMin) * AutoLevelStrength) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * AutoLevelStrength)) # Round down
(DEBUG) ? RT_Debug("Levels("+String(ALMin)+",1.0,"+String(AlMax)+","+String(CSMin)+","+String(CSMax)+",Coring=False)") : NOP
k = k.Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DONT EVER USE Coring
if(ShowAutoLevel) {
tmpw=k.width/4*2
LeftClip = TMP_K.Crop(0,0,tmpw,-0)
RightClip= k.Crop(tmpw,0,-0,-0)
k=StackHorizontal(LeftClip,RightClip)
}
}
if(KeepMargin && ShowMargin) {
(DEBUG) ? RT_Debug("KeepMargin AddBorders("+String(Left2)+","+String(Top2)+","+String(Right2)+","+String(Bot2)+")") : NOP
k = k.Addborders(Left2,Top2,Right2,Bot2,MARGINBORDER)
}
(DEBUG) ? RT_Debug("AddBorders("+String(Left)+","+String(Top)+","+String(Right)+","+String(Bot)+")") : NOP
k = k.Addborders(Left,Top,Right,Bot,BORDER).assumefps(fps)
picclip = (IsClip(picclip)) ? picclip ++ k : k # Append to clip so far
k=k.ConvertToRGB24() # Addborders done (bug in AddBorders rgb24)
if(DEBUG){RT_Debug(" ") RT_Debug(" ----------------------------- ")} # line separator
}
""")
return PicClip
}


Given args are just to demo the border/margin stuff.

Updated PlanetCrop() in RT_Stats zip.

[martin53]

StainlessS,
:thanks: - How could you know that I just needed RT_BitTST() and RT_BitSET() ! It was exactly what I intended to use to replace some string operations.

RT_FloatAsRGBA() and RT_RGB32AsFloat() seem to work flawless.
I'm sure the hexfunctions will come in handy at least for subtitles.

[StainlessS]

Thank you for the report. As you probably know, 2.6a3 has bit type operations, however few will use in
general scripts because they are not available in 2.58, the RT_ versions overcome that as available to all.
Also several of the bit ops are not implemented in 2.6 (rotates and bitchg, I think). I did not use identical names to
2.6, I'm more comfortable with the Motorola MC68000 assembler mnemonics for the bit ops.
In QueryBorderCrop, I ended up splitting a flags int into multiple separate bools to be able to test,
bit ops make some things so much easier.

Hope you noticed that I used the RT_YInRange func as suggested by you, in previous post script, would not
have been able to implement that script without it, thank you again. :)

[Forensic]

StainlessS, this is brilliant. I haven't done much coding since my 6502 days and have yet to develop the skills to write my own DLLs. Implementing your tools just gave me a huge speed improvement over using the internal runtime functions. My next task is extracting Mediainfo using your RT_Call. Thank you.

[StainlessS]

You're very welcome kind sir. :)
The built in's are clearly faster than RT_stats raster funcs for full frame, but RT is intended for partial frame/scanline/pixel
reading, and are probably faster than repeatedly chopping out individual scanlines/pixels and testing them.

I did a tiny-weeny bit of 6502/6809/8080 in 81 but not much, later did Z80A for about 3/4 years, would hate to do that again.
M68K spoilt me for assy progging and I just hate little endian, did not really know any different when I did Z80.

Will up a new version RT within a few days, with a few additional funcs to those previously posted for v1.07beta.
(By the way, no real problems found in v1.07 beta although I changed some parts a little.)


RT_GetFrame(clip,int "n"=current_frame,int "delta"=0) # Name might change : Changing to RT_YankChain()
Compile/runtime clip function.
n (default = current_frame) frame number.
delta (default = 0), frame number offset.
Process frame (n + delta) of a clip ie forcibly process the filter graph chain for frame n + delta.
Replaces need for eg RT_AverageLuma(n=i,w=1,h=1) to sample a single pixel from frame n to do the same.


RT_NumberString(int ,int "base"=10, int "width"=0)
First arg is an integer to convert to a number base/radix string.
Base, (10, 2 -> 36), is the number base or radix, eg 2 == Binary, 8 == Octal, 10 == Denary/Decimal, 16 == Hexadecimal.
The default of 10 (decimal) will just convert a number to its decimal string equivalent possibly with a '-' minus sign.
All number bases with the exception of decimal, will be unsigned form, ie -1 to hexadecimal will produce "FFFFFFFF",
(the sign is in the digits rather than as separate 'sign and magnitude' used in decimal representation).
Here the digits used for the base, "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ". Binary uses first two; decimal, the first 10;
Hexadecimal the first 16; base 36 all 36 digits.
Width, (0, 0 -> 32) is the minimum width of the returned string.
eg RT_NumberString(255,16,4) returns "00FF".


RT_NumberValue(String,int "base"=10)
Returns an int conversion of the supplied number base string.
Base, (10, 2 -> 36) is the number base that the string uses.
Conversion will cease at the first non legal number base digit, without producing an error.
eg RT_NumberValue("1100",2) returns 12 from the binary string.

RT_RGB32AsInt(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0)
Compile/Runtime clip function.
Given an RGB32 clip that had a pixel value created from an Int, gets that pixel and returns as the
original Int. By creating single pixel clips and stacking them horizontally/vertically,
you can use a frame as a two dimensional array of Int, perhaps use the frame number for a third dimension.



Peace Bro :)

EDIT: The built in's are clearly faster than RT_stats raster funcs
As it turns out, actually they are not faster.

[StainlessS]

@mastrboy,

Could I possibly ask you to delete your post #2 and repost again. I shall then repost my reply to you.
This is so that I can use my post #3 as an extension to the first post.
That might well break some links, but the limit in the Usage forum is 16K (20K I think in Developer),
but the current text description for RT is now at 23kb. Thank you.

(Unless a moderator has an alternative solution)

Happy New Year to one and all, and may your God smile upon you.

[Forensic]

@StainlessS

DOS BAT file (works great)
set var=test.avi
MediaInfo_CLI\MediaInfo.exe "%CD%\%var%" > %var%.txt

RT_Call in AVS file to get the same output (fails)
a="test.avi"
Cmd="MediaInfo_CLI\MediaInfo.exe "+chr(34)+"%CD%\"+a+chr(34)+" > "+a+".txt"
RT_Call(Cmd,Hide=True)

I can't figure out what I am doing wrong. Any suggestions?

[StainlessS]

At a guess I'de say that it has no idea where MediaInfo_CLI\ is nor Test.avi,
Try using full path to both. DOS knows where it is, and so can use paths relative to itself,
RT_Call does not have a clue where DOS is/was.

I'll add a debug option to the func, to give a better clue as to what went wrong.

EDIT: I'll give it try myself, and check it out.

[StainlessS]

@Forensic, (sorry about the delay in getting back to you)

The main problem in you usage of RT_Call is the mixing of MediaInfo commands and DOS commands.

You are calling MediaInfo and supplying it with %CD% and redirection ">", neither of which means anything to MediaInfo.

This is more like what you want


a="test.avi"

#SetWorkingDir("D:\Whatever")
Dir=_GetWorkingDir()

Cmd_1=QuoteStr(Dir+"MediaInfo_CLI\MediaInfo.exe") + " " + QuoteStr(Dir+a) + " --LogFile=" + QuoteStr(Dir+a+".1.txt")
Cmd_2=QuoteStr("MediaInfo_CLI\MediaInfo.exe") + " " + QuoteStr(a) + " --LogFile=" + QuoteStr(a+".2.txt")

RT_Call(Cmd_1,Hide=true)
RT_Call(Cmd_2,Hide=true)

return colorbars

# _GetWorkingDir(), Get Current Working Directory as used by Avisynth for easy importing of scripts:- By StainlessS
Function _GetWorkingDir(){Try{Import("?\")}Catch(_){s=FindStr(_,Chr(34))_=MidStr(_,s+1,FindStr(_,"?")-s-1)}_}

Function QuoteStr(string s) {return Chr(34)+s+Chr(34)}


Might though be better to give an explicit directory for MediaInfo rather than scriptdir relative.

EDIT: Oops, removed the "debug=true" arg which is not as yet available in public releases.

ALSO NOTE, below previously given MediaInfo usage in RT_Stats doc


Example to write Aspect Ratio in form "1.333" to file D:\OUT\MEDINFO.TXT.
RT_Call("D:\TEST\MediaInfo.Exe" + " --LogFile=" + "D:\OUT\MEDINFO.TXT" + " --Output=Video;%AspectRatio% " + "D:\VID\1.mpg")


I think I got the above from the MediaInfo included doc, but, --Output and %AspectRatio% seem to be deprecated (missing from
MediaInfo built in help) but still work ok.

Below the currently non deprecated usage, (using '--Inform' and '%DisplayAspectRatio%')

RT_Call(QuoteStr("D:\TEST\MediaInfo.Exe") + " --LogFile=" + Quotestr("D:\OUT\MEDINFO.TXT") + \
" --Inform=Video;%DisplayAspectRatio% " + Quotestr("D:\VID\1.mpg"))

[Forensic]

@StainlessS

I marvel at your awesome skill. It worked like a charm. Thank you.

[StainlessS]

@Forensic

Ooooo, you really are a sycophant, but dont stop, me likee. :)

[StainlessS]

RT_Stats new version v1.08, see 1st post.

Also contains new script function in zip:


Function ImageSplicer(String "FileName",int "SzLimit",Float "FAR",Bool "AutoCrop",Float "AutoLevelStrength",bool "ShowAutoLevel",bool "ShowBorder", \
bool "Subs", Float "CropThresh",Float "FPS",int "WMod",int "HMod",int "Baffle",Float "ScanPerc", \
bool "Debug", bool "Debug_QBC",bool "Debug_QLMM",String "Resizer") {
##########################################
# ImageSplicer v1.00 - 08 Jan 2013
#
# Requires GScript, RT_Stats v1.08 and QueryBorderCrop.avs, QueryLumaMinMax.avs
#
# ImageSplicer, joins together images of different sizes into a sort of slideshow, auto cropping borders, auto levelling, resizing or original size,
# border padding to chosen aspect ratio.
# Gets max width/height of all images (after optional auto cropping) and creates a background 'canvas' based on SzLimit & FAR.
# SzLimit==0: Images kept at original dimensions (post cropping), canvas made to encompass all cropped images.
# SzLimit>0: The shorter canvas dimension (FAR) limited to AT MOST SzLimit, where images will be resized to fit canvas.
# SzLimit<0: The shorter canvas dimension (FAR) set at abs(SzLimit) and images resized to fit canvas.
#
# Args:-
# Filename: Default "*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF". Image Path and wildcard with mulitple pipe separated extensions.
# SzLimit: Default=0, 0=No resize. Greater than 0 is limit on shorter dimension (see FAR), -ve resize to abs(SzLimit) shorter dimension.
# FAR: Default 4.0/3.0. Output Frame Aspect Ratio, if FAR>=1.0 then SzLimit affects height, else affects width.
# AutoCrop: Default=True. Crops borders off images.
# AutoLevelStrength: Default 0.0, Auto luma leveling, range 0.0 -> 1.0. 0.0 is OFF.
# ShowAutoLevel: Default False. If true then shows Auto leveling on right hande side of image ONLY.
# ShowBorder: Default False. When padding to maintain Frame Aspect Ratio, borders are shown in red.
# Subs: Default False. Shows Subtitled info on frames.
# CropThresh: Default -32.0. Leave at default -32.0 to allow auto adjustment (EDIT by QueryBorderCrop).
# CropThresh <=0.0 is auto thresh where thresh is set to (YPlaneMin + abs(CropThresh))
# CropThresh >0.0 is explicit threshold @ PC Levels, probably best avoided.
# Default -32.0 will for single frames auto adjust cropthresh to -0.58219178 (-0.5 * 255.0/(235-16)), ie -0.5 scaled to PC levels.
# CropThresh is then set to (YPlaneMin + abs(CropThresh)). This very sensitive threshold for single
# frame clips is to avoid possibility of overcropping. User specified cropthresh of eg -1.0 would scale to YPlaneMin + (1.0*255/219)
# RGB clips are sampled as YUV-Y @ PCLevels in QueryBorderCrop. See QueryBorderCrop.
# FPS: Default 1.0. Frames Per Second of output clip.
# WMod: Default 2. Output clip width is a multiple of WMod.
# HMod: Default 2. Output clip height is a multiple of HMod.
# Baffle: Default 3. Number of scanlines that must break a CropThresh to be accepted as Image. Avoids noise. Not usually altered.
# ScanPerc: Default 49.0. Percentage of image dimensions to scan per side. Not usually altered. See QueryBorderCrop.
# Debug: Default False. Show debug info via DebugView. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
# Debug_QBC: Default False. Debug QueryBorderCrop.
# Debug_QLMM: Default False. Debug QueryLumaMinMax.
# Resizer: Default "Spline36Resize(_W_,_H_)", the resizer to use. "_W_" and "_H_" are replaced by the resize width & height.
##########################################


PlanetCrop.avs v1.01 updated in zip.

[StainlessS]

v1.09 Bugfix, Fixed RT_RgbChanAve w==1 bug (always blue channel).

[StainlessS]

Coming, bit pissed at the moment but thought someone may like the below adds's (what It've got sooooo far)..

RT_GetWorkingDir()
Return current Working directory, directory of script, or dir set by SetWorkingDir().
The returned directory ends with a backslash.


RT_GetFullPathName(string "name")
Creates an absolute or full path name for the specified relative path 'name'.
eg 'RT_GetFullPathName("test.avs")' might return "D:\avs\test.avs".
Throws and error if cannot return path. (PS, pretty much there).


RT_FileDelete(string fn)
Delete file fn. Use with care.
Returns 0 = OK. -1 = access denied / read only. -2 = Path not found OR is Directory. -3 = Unknown error.


RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the
parts joined together.
'Get' (default 15, 1 -> 15), If set, Bit 0=DRIVE, 1=Dir, 2=Name, 4=Extension.
Add 1 for Drive (bit 0), add 2 for Dir (bit 1), add 4 for Name (bit 2), add 8 for Extension (bit 3).
Some combinations do not make sense, eg Drive + Extension (1+8=9). Below sensible options.
1 = Drive (includes trailing ':')
2 = Dir (includes trailing '\')
3 = Drive + Dir
4 = Name
6 = Dir + Name
7 = Drive + Dir + Name
8 = Extension (includes leading '.')
12 = Name + Extension
14 = Dir + Name + Extension
15 = Drive + Dir + Name + Extension
Assuming a current working directory of eg "D:\avs\avi", 'filename'="test.avi" and 'get'=15, returns "D:\avs\avi\test.avi",
so given a relative filename and default 'get'=15, is equivalent to GetFullPathName(filename).

Also, gonna add a func to get all Y(luma) stats info from a single call that is little more expensive than a single call.
ie, Ave luma, min / max / max-min / Median / Yinrange / stddev, all in a single call, and all costing little more than
a single RT call (single RT call about 30% slower than a built in Avisynth call (I thiink).
Need some sleep. (wish I had some beer) [EDIT: pretty much there,.]

[StainlessS]

This script to extract security camera footage with movment only, removing static scenes.
In response to this thread here:
http://forum.doom9.org/showthread.php?t=166974

Requires GScript, Grunt, RT_Stats v1.09+, FrameSelect and DegrainMedian Plugins.


AVISource("D:\AVS\AVI\CAP.avi")

# DO NOT USE DirectShowSource OR DENOISER THAT NEEDS LINEAR ACCESS eg HQDN3D
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN ShowMetrics OR MakeFramesCmd.

Function ShowMetrics(clip c,float th,int "x",int "y",int "w",int "h",bool "dim",bool "fast") {
# SUBTITLES, 1st number Frame number, 2nd Anchor frame number, Action, Diff between Frame and Anchor.
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) dim=Default(dim,true) Fast=Default(Fast,False)
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
cropped=Crop(x,y,w,h)
(dim && (w!=width || h!=height)) ? levels(0,1.0,255,0,128).Overlay(cropped,x=X,y=Y,opacity=1.0) : Last
Global ShowMetrics_Start=0 # Frame to start comparing with (Anchor)
Global ShowMetrics_KEPT=0 # Count of kept frames
Global ShowMetrics_PREV=0 # The 2nd most recent kept frame
ScriptClip("""
d = RT_LumaDifference(Last,Last,n2=ShowMetrics_Start,x=x,y=y,w=w,h=h)
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th)
Global ShowMetrics_KEPT = (keep) ? ShowMetrics_KEPT + 1 : ShowMetrics_KEPT
SS=String(current_frame) + " ] (" + String(ShowMetrics_Start) + ") " + ((keep)?" KEEP":" DROP") + " Diff="+String(d,"%03.2f") +\
" KEPT=" + String(ShowMetrics_KEPT) + " " + String(ShowMetrics_KEPT*100.0 / (current_frame+1.0),"%03.2f") + "%"
Global ShowMetrics_Start=(keep)?ShowMetrics_Prev:ShowMetrics_Start # Follow but NOT too closely (delayed by 1)
Global ShowMetrics_Prev=(keep)?current_frame:ShowMetrics_Prev # 1 kept frame delay
Subtitle(SS)
""",After_Frame=True,args="th,x,y,w,h") # Needs Grunt for args
SubTitle("Thresh = "+String(th,"%02.2f"),Align=1)
return ((Fast)?AssumeFPS(250.0):Last) .KillAudio()
}

Function MakeFramesCmd(clip c,float th,int "x",int "y",int "w",int "h",String "FileName") {
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) FileName=Default(FileName,"FrameSelect_Cmd.Txt")
(Exist(FileName)) ? RT_Call("cmd /c DEL /Q "+RT_QuoteStr(FileName),hide=true) : NOP # Delete existing
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
Global MakeFramesCmd_Start=0 # Frame to start comparing with (Anchor)
Global MakeFramesCmd_PREV=0
ScriptClip("""
# We DONT compare with previous frame, we compare with previous KEEP frame to avoid slow (and sneaky) burglars.
d = RT_LumaDifference(Last,Last,n2=MakeFramesCmd_Start,x=x,y=y,w=w,h=h) # 2 clip func, use 2 frame nos, not delta
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th) # Keep BOTH FIRST and LAST frame ALWAYS
Global MakeFramesCmd_Start=(keep)?MakeFramesCmd_Prev:MakeFramesCmd_Start # Follow but NOT too closely
Global MakeFramesCmd_Prev=(keep)?current_frame:MakeFramesCmd_Prev
WriteFileIf(FileName, "(keep)", "current_frame", append=True)
""",After_Frame=True,args="th,x,y,w,h,FileName") # Needs Grunt for args
return AssumeFPS(250.0).KillAudio()
}

Function SelectFrames(clip c,String "FileName",bool "Show") {
c
FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Show=Default(Show,False)
(Show) ? ShowTime() : NOP
FrameSelect(CMD=FileName) # Returns NO audio
}

Function GetFrames(clip c,float th,int "x",int "y",int "w",int "h",String "FileName",clip "dclip",bool "Show") {
Default(dclip,c) # Use denoised clip for scanning
MakeFramesCmd(th,x,y,w,h,FileName) # Make the command file for FrameSelect()
GSCript("""
For(i=0,framecount-1) {
RT_YankChain(n=i) # Force Process, MakeFrames
}
c.SelectFrames(FileName,Show=Show) # Extract frames and return to client
""")
}

###################

X=128 Y=172 W=-128 H=-0 # Area Of Interest, avoid eg passing traffic or timecode.

THRESH =1.3 # Whatever, use ShowMetrics, need adjust, suggest 0.5 -> 1.5, need check against source.

DENOISE=7 # DegrainMedian LimitY, 0 = OFF, higher stronger, Suggest 4 -> 8
DN_MODE=0 # DegrainMedian mode, see DegrainMedian docs, suggest 0 -> 2, higher weaker
SHOW=True # Show original Time on final output

# Optional Denoise, perhaps there are better ones, Produces identical results in below functions. (OK for non-linear access).
dclip = (DENOISE>0) ? ConvertToYV12().DeGrainMedian(limitY=DENOISE,limitUV=0,mode=DN_MODE).GreyScale() : Last

###################

ShowMetrics(dclip,THRESH,X,Y,W,H) # Show metrics to get best Threshold 'th'. Denoised clip shown in GrayScale.
#MakeFramesCmd(dclip,THRESH,X,Y,W,H) # Make Frames Cmd file for SelectFrames()
#SelectFrames(show=SHOW) # Return Frames with EXISTING frames Command file (after MakeFramesCmd)
#GetFrames(THRESH,X,Y,W,H,dclip=dclip,show=SHOW) # Make Frames command file and then extract frames automatically in 1 pass.


UPDATED:

[StainlessS]

Ref previous post:
I stuck a webcam out my window and captured about 2750 frames at about midnight.
Running through previous script with Thresh set to 1.0, it reduced frame count to 145 fames
with 1 bike and 3 cars passing, lights behind curtains going off and a fox passing close by.
So, it does not seem too bad as it is, but might be happier if more sensitive to small localized movement.
Would probably do better with LumaDifference (SAD, Sum Of Absolute Differences) being replaced with
Sum Of Squared Differences. I'll add a func to RT_Stats to do that.
Ooooh, there's that fox making a weird racket.

EDIT: Old Webcam 640x480@15FPS Codec Indeo IV50, murky window glass with street lamp shining upon it.

[StainlessS]

Updated post #106.
Added dclip denoised detection clip + other mods.

[StainlessS]

New Version RT_Stats v1.10 17 Mar 2013. See first post.

Updates in brief:

RT_Debug(string s1, ... , string sn,bool "name"=true)
v1.10, Added name arg, setting false inhibits title "RT_Debug:" output, set s1 to your own title instead.

RT_GetLastError()
Return a error number describing previous system error (if available). Perhaps use after eg RT_Call.

RT_GetLastErrorString()
Return a string describing previous system error (if available).

RT_TxtSort(String,int 'mode'=0)
Function to sort a Chr(10) separated multiline string.
The String arg is the multiline string, and 'mode' can be in range 0 to 11.
v1.10, added modes 8 -> 11.
8 = Sort ascending, Integer number Strings
9 = Sort ascending, float number Strings
10= Sort decending, Integer number Strings
11= Sort decending, float number Strings

RT_StrPad(String,Int n,string "c"=" ")
Pads an unnamed string to at least int n (unnamed) length with first character of string c (Default SPACE).

RT_GetWorkingDir()
Return current Working directory, directory of script, or dir set by SetWorkingDir().
The returned directory ends with a backslash.

RT_GetFullPathName(string "name")
Creates an absolute or full path name for the specified relative path 'name'.
eg 'RT_GetFullPathName("test.avs")' might return "D:\avs\test.avs".
Throws and error if cannot return path.

RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the
parts joined together.
'Get' (default 15, 1 -> 15), If set, Bit 0=DRIVE, 1=Dir, 2=Name, 4=Extension.
Add 1 for Drive (bit 0), add 2 for Dir (bit 1), add 4 for Name (bit 2), add 8 for Extension (bit 3).
Some combinations do not make sense, eg Drive + Extension (1+8=9). Below sensible options.
1 = Drive (includes trailing ':')
2 = Dir (includes trailing '\')
3 = Drive + Dir
4 = Name
6 = Dir + Name
7 = Drive + Dir + Name
8 = Extension (includes leading '.')
12 = Name + Extension
14 = Dir + Name + Extension
15 = Drive + Dir + Name + Extension
Assuming a current working directory of eg "D:\avs\avi\", 'filename'="test.avi" and 'get'=15, returns "D:\avs\avi\test.avi",
so given a relative filename and default 'get'=15, is equivalent to RT_GetFullPathName(filename).

RT_GetFileExtension(string s)
Returns filename extension from the supplied filename, including '.' prefix.


RT_FileDelete(string fn)
Delete unnamed file fn. Use with care.
Returns 0 = OK. -1 = access denied / read only. -2 = Path not found OR is Directory. -3 = Unknown error.

RT_Undefined()
Returns same as undefined() in v2.6. Defined(RT_Undefined()) would return false.

RT_VarExist(string)
Given the name (string) of the variable that you want to test for existence, returns true if exists, else false.
Eg, #a=32
RT_Debug(string(RT_VarExist("a"))) # would output false to debugview unless '#a=32' uncommented. {Defined(a) would fail with error}.
return colorbars()

RT_Ord(string)
Returns same as 'Ord' in v2.6, the ASCII code for the 1st letter in string (0 if empty string "").

RT_Timer()
Returns time in seconds since start of process.

RT_Ystats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=(width<=720?2:3),int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="YS_")
Returns multiple results as for above single frame Luma sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", and "Matrix", are as for all other clip functions, "threshold" used only for "RT_YPlaneMin", "RT_YPlaneMax" and
"RT_YPlaneMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_YInRange" equivalent routine with same functionality.
The only new args are int "Flgs" which selects which results you want returned and the string "Prefix" that is prepended
to the returned global variables.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.

Flgs_Bit_Number Add_To_Flgs Equivalent_Funcion Global_Var_Set_Excluding_Prefix
0 1($01) RT_YPlaneMin() "yMin" (0->255)
1 2($02) RT_YPlaneMax() "yMax" (0->255)
2 4($04) RT_YPlaneMinMaxDifference() "yMinMaxDiff" (0->255)
3 8($08) RT_YPlaneMedian() "yMed" (0->255)
4 16($10) RT_AverageLuma() "yAve" (0.0->255.0)
5 32($20) RT_YPlaneStdev() "yStdev" (0.0->255.0)
6 64($40) RT_YInRange() "yInRng" (0.0->1.0)
RT_Ystats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=127($7F) are all bits set and so sets ALL global vars at once.
RT_Ystats(flgs=1+2+16) would set global vars "YS_yMin", "YS_yMax" and "YS_yAve" for full frame current_frame.


****** SCRIPT CONVERSION TO PLUGIN FUNCTIONS ********

Below fns converted from previously existing script functions which will also be supplied so that you can see the logic,
any changes after v1.10 plugin versions may not be reflected in the later script versions.

RT_GetSAR(clip,dar float)
Gets Sample (pixel) Aspect Ratio from DAR Display Aspect Ratio.

RT_GetDAR(clip,sar float)
Gets DAR Display Aspect Ratio from SAR, Sample (pixel) Aspect Ratio.

RT_SignalDAR(float) # See Megui wiki (name change from SetDAR)
Signal DAR to MEGUI. Sets global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

RT_GetCropDAR(clip,float DAR,float "X"=0,float "Y"=0,Float"W"=0,float "H"=0)
Call prior to Crop/Resize with (possibly fractional) cropping to calc resultant DAR, X,Y,W,H are cropping coords
DAR = FAR * SAR ::: FAR = DAR / SAR ::: SAR = DAR / FAR
Cropping affects FAR & usually DAR, resizing does not affect DAR (unless also cropped). Resizing affects FAR and maybe SAR.
We dont allow eg -ve X

RT_QueryLumaMinMax(clip c,int "Samples"=24,float "Ignore"=0.2,String "Prefix"="QLMM",bool "DEBUG"=false,int "X"=0,int "Y"=0,int "W"=0,int "H"=0, \
int "Matrix"=(width<=720?2:3))
Prescan function to get Luma Min,Max for clip scanning Samples frames area x,y,w,h.
Returns luma min/max as string Default eg "QLMMMin=25 QLMMMax=244", use eg "Eval(RT_QueryLumaMinMax())" to set variables for use in script.
Samples:=24 = frames in clip to sample.
Ignore:=0.2=Percentage of extreme pixels to ignore (noise) when getting luma min/max.
Prefix:="QLMM"=Prefix for return string values.
X=Y=W=H=0=Full frame, as crop, area to examine.
DEBUG:=false= dont show. True=debug info. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
Matrix:, For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
Default for is 2(PC601) if width <= 720 else 3(PC709) : YUV not used

Function RT_QueryBorderCrop(clip c,int "Samples"=24,Float "Thresh"=-32,bool "Laced"=true,int "XMod",int "YMod",int "WMod",int "HMod", \
bool "Relative"=false, String "Prefix"="QBCrop",int "RLBT"=15,bool "DEBUG"=false,float "Ignore"=0.2,int "Matrix"=(width<=720?2:3), \
Float "ScanPerc"=49.0,int "Baffle"=4, bool "ScaleAutoThreshRGB"=true,bool "ScaleAutoThreshYUV"=false)

Prescan function to get coords for eg cropping black borders by sampling Samples frames. Planar, YUY2, RGB.
Borders are detected by sampling at Samples frames, at scanlines (h/v) using AverageLuma (RGB is converted to Luma Y at either TV or
PC levels, See Matrix). This sampling is done on all 4 sides of the frame.
If a scanline Average luma is below or equal to Thresh, then is considered possible black border, above considered possible image,
if Baffle [default=4] adjacent scanlines above Thresh, then it IS image.

Simultaneously returns 4 sets of strings holding crop coords, here is one set: "QBCropX=8 QBCropY=8 QBCropW=640 QBCropH=480".
String sets are Chr(10) separated, the 'exact found' coords set as above, not rounded at all and possibly throwing an error if
you try to crop, although could use in eg resize.
Second set using eg "QBCropXL=8", which is CropLess("L" trailer), ie when rounding for Xmod,WMod etc may leave some black border.
Third set using eg "QBCropXM=8", which is CropMore, ie when rounding for Xmod,WMod etc may crop some image.
Forth set using eg "QBCropXP=8", which is CropPlus, moves border positions an extra 2 pixels inwards on each edge and then as CropMore.
The non-exact coords try to center the coords when cropping for larger WMod/HMod's so as to evenly crop from eg both left and right instead of
just one side. Also returned in the return string is the used Threshold, perhaps set by AutoThresh, as eg "QBCropThresh=32.0"
You can use eg Eval(QueryBorderCrop()) to set values into QBCropX, QBCropY,QBCropW,QBCropH etc variables.



Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(width<=720?2:3),String "Prefix"="YIRL_",bool "Debug"=false)

Clip function to scan frame n+delta for an area where percentage of pixels in luma range Lo to Hi (inclusive) breaks a threshold over at least Baffle
consecutive number of scanlines (all four sides). Returns True on Successful location else false. On success set Global var Coords.
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold. Default is any pixel in the range Lo to Hi will break threshold.
Lo: Default 128, lower bound of pixel values to search for.
Hi: Default 255, upper bound of pixel values to search for.
Matrix: Default for RGB is 2(PC601) if width <= 720 else 3(PC709) : YUV not used
Prefix: Default "YIRL_", prefix for Global Var coords set on successful location of object, eg YIRL_X, YIRL_Y, YIRL_W, YIRL_H.
Debug: Default false. If true outputs info to debugView.

[StainlessS]

See also previous post:

Here result of script included in RT_Stats v1.10 (EDIT: Required GScript)


00000024 20:06:29 RT_Debug: RT_Stats Timing Test
00000025 20:06:29 RT_Debug: Timing 14x5(70) runs on 2500 Frames (Total Frames=175000)
00000026 20:07:06 RT_Debug: RT_YankChain 7.19Secs : 347.56FPS :: Basic GetFrame() [ADJUSTMENT]
00000027 20:07:58 RT_Debug: YPlaneMin 10.55Secs : 237.03FPS :: ADJUSTED 3.35Secs 745.38FPS
00000028 20:08:47 RT_Debug: RT_YPlaneMin 9.66Secs : 258.75FPS :: ADJUSTED 2.47Secs 1012.69FPS
00000029 20:09:39 RT_Debug: YPlaneMax 10.55Secs : 237.03FPS :: ADJUSTED 3.35Secs 745.38FPS
00000030 20:10:28 RT_Debug: RT_YPlaneMax 9.67Secs : 258.48FPS :: ADJUSTED 2.48Secs 1008.47FPS
00000031 20:11:23 RT_Debug: YPlaneMinMaxDif 11.03Secs : 226.74FPS :: ADJUSTED 3.83Secs 652.23FPS
00000032 20:12:13 RT_Debug: RT_YPlaneMinMaxDif 10.00Secs : 250.00FPS :: ADJUSTED 2.81Secs 890.63FPS
00000033 20:13:06 RT_Debug: YPlaneMedian 10.59Secs : 235.99FPS :: ADJUSTED 3.40Secs 735.15FPS
00000034 20:13:55 RT_Debug: RT_YPlaneMedian 9.73Secs : 256.82FPS :: ADJUSTED 2.54Secs 983.73FPS
00000035 20:14:38 RT_Debug: AverageLuma 8.63Secs : 289.86FPS :: ADJUSTED 1.43Secs 1745.81FPS
00000036 20:15:20 RT_Debug: RT_AverageLuma 8.44Secs : 296.30FPS :: ADJUSTED 1.24Secs 2009.12FPS
00000037 20:17:57 RT_Debug: Avisynth_ALL 31.44Secs : 79.52FPS :: ADJUSTED 24.24Secs 103.12FPS
00000038 20:18:56 RT_Debug: RT_Avisynth_ALL 11.83Secs : 211.35FPS :: ADJUSTED 4.64Secs 539.30FPS
00000039 20:19:56 RT_Debug: RT_ALL 11.87Secs : 210.62FPS :: ADJUSTED 4.68Secs 534.57FPS
00000040 20:19:56 RT_Debug: Total Runtime = 13.45 Mins


720x576@25 UT Video Codec. Avisynth v2.6a4 Standard. (RT Compiled TK3, VS6 not too much different)

Does 5 runs on each func, throws away fastest and slowest times and gets average of remaining three.

Adjusted times have the timing for reading frames (eg HardDrive, codec, sourcefilter) subtracted from them.
Avisynth_ALL samples each frame via all the Avisynth Native Luma Sampling runtime funcs,
RT_Avisynth_ALL() [RT_YStats] does same but in 1 function. RT_ALL includes two additional Runtime funcs that
are not native to Avisynth, ie 7 sampling funcs altogether at once.

EDIT: If your results contradict above, post them here (above surprises the hell out of me, RT_ just standard CPP).
Above results on Core 2 Duo duel core. Similar results speed ratios on single core Prescot and another P4 machine.

EDIT: Below is the RT_Ystats() doc which is repeated here from previous post (is a bit hidden there):

RT_Ystats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=(width<=720?2:3),int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="YS_")
Returns multiple results as for above single frame Luma sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", and "Matrix", are as for all other clip functions, "threshold" used only for "RT_YPlaneMin", "RT_YPlaneMax" and
"RT_YPlaneMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_YInRange" equivalent routine with same functionality.
The only new args are int "Flgs" which selects which results you want returned and the string "Prefix" that is prepended
to the returned global variables.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.

Flgs_Bit_Number Add_To_Flgs Equivalent_Funcion Global_Var_Set_Excluding_Prefix
0 1($01) RT_YPlaneMin() "yMin" (0->255)
1 2($02) RT_YPlaneMax() "yMax" (0->255)
2 4($04) RT_YPlaneMinMaxDifference() "yMinMaxDiff" (0->255)
3 8($08) RT_YPlaneMedian() "yMed" (0->255)
4 16($10) RT_AverageLuma() "yAve" (0.0->255.0)
5 32($20) RT_YPlaneStdev() "yStdev" (0.0->255.0)
6 64($40) RT_YInRange() "yInRng" (0.0->1.0)
RT_Ystats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=127($7F) are all bits set and so sets ALL global vars at once.
RT_Ystats(flgs=1+2+16) would set global vars "YS_yMin", "YS_yMax" and "YS_yAve" for full frame current_frame.

[StainlessS]

OOPS, mismatch in docs, script YInRangeLocate.avs and RT_YInRangeLocate default Prefix,
docs say "YIRL" avs script "YIRL" and "YIRL_" in RT_YInRangeLocate. Have changed online docs to "YIRL_"
and will update doc and avs script in next version to "YIRL_".


Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(width<=720?2:3),String "Prefix"="YIRL_",bool "Debug"=false)


Affects none of the included scripts (PlanetCrop.avs) as they use an explicit Prefix.

[StainlessS]

Oh dear!
Another update, v1.11 19 Mar 2013.
Last minute change to v1.10, I missed out a '!' in check on clips in RT_LumaCorrelation and RT_LumaDifference, so they will
give a "Must Have Two Clips" type error. Fixed here in v1.11. See 1st post or in sig.

Also posted a request for a name of new func here (in developers forum):-
LINK REMOVED, THINK I MADE A BOOBOO

If you have a better name then please submit.

As a temp measure, v1.11 includes func as RT_LumaSNSSD() # Scaled Normalized Sum Of Squared Differences.


RT_LumaSNSSD(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value Scaled Normalized Sum of Squared Differences. (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h,
and clip2 frame (n2+delta2) area x2,y2,w,h. Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Calculates Sum Of Squared Differences, takes the Square root of it and then scales in range 0.0 -> 255.0.


EDIT: Will also add RT_LumaSSSD() {or other name} for Scaled Sum of Squared Differences.

[kypec]

Another update, v1.11 19 Mar 2013.

yet you updated the thread title too much ahead in time "19 May 2013" :D

[StainlessS]

Thank you kind sir, making lots of mistakes lately, must be getting senile.

Think I may also have dropped a B@ll@ck with the RT_LumaSNSSD() thing,
it seems to be graphing linearly (not much different to SAD LumaDifference).
Perhaps an early April fool thing. :)

I was going to experiment with the RT_LumaSNSSD thing and had just put back the
source into RT_Stats source when I found the "!" error in v1.10, as I was reluctant
to remove it again for the v1.11 update I just pushed it out, but not sure that it is terribly
useful.
Have just now implemented the RT_LumaSSSD and that seems to be working as expected.

[StainlessS]

And again. New Version v1.12.


# PROBABLY REMOVED IN NEXT VERSION, I THINK I MADE A BOOBOO
RT_LumaSNSSD(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value Scaled Normalized Sum of Squared Differences. (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h,
and clip2 frame (n2+delta2) area x2,y2,w,h. Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Calculates Sum Of Squared Differences, takes the Square root of it and then scales in range 0.0 -> 255.0.



# CONSIDER IT EXPERIMENTAL
RT_LumaSSSD(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value Scaled Sum of Squared Differences. (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h,
and clip2 frame (n2+delta2) area x2,y2,w,h. Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Calculates Sum Of Squared Differences, then scales in range 0.0 -> 255.0.


EDIT: @ Jmac, I think the Moon2 sample took about 6.5 minutes in old version of PlanetCrop, knocked it down to about
18.5 seconds in current version, :) :) :) :) :) :) :) :)

[StainlessS]

New version, RT_Stats v1.13, see first post.
RT_QueryBorderCrop CropLess/CropMore modified (did not quite work as it should have).

[StainlessS]

New Version, RT_Stats v1.14, see 1st post.

Added RT_FindStr. Mods:- RT_Debug, StrReplace fns, RT_NumberValue and RT_HexValue.

Newly Added:


RT_FindStr(String S,string Substr,bool "Sig"=True,int "Pos"=1)
Finds unnamed string Substr in unnamed string S.
Optional bool Sig: Default=True, does case significant comparison, insignificant if false.
Optional int Pos: Default=1, is start position in string S to find string Substr. If Pos smaller than 1 or greater than length of string s,
then will return 0, Not Found.
Returns position (1 relative) of found string within the FULL LENGTH string S, or 0 if not found.
Differs from Avisynth FindStr in Sig and Pos args and in that if either string S or string Substr is "" then will ALWAYS return 0, 'Not Found',
Avisynth FindStr always returns 1 if SubStr="" even if S is also "".
v1.14, New.


Mods:


RT_Debug(string s1, ... , string sn,bool "name"=true)
Non-clip function. Takes one or more strings, joins them together and then outputs the combined
string to OutputDebugString(), if strings are NOT space separated, it inserts a single space between them.
Useful during development Compile-time/Run-time scripts to monitor status in real time.
Can also output multiline string obtained via eg RT_ReadTxtFromFile.
Name: bool Default=True. If False, then does NOT output "RT_Debug: " at the beginning of each line, you
might want to replace with your own title in string s1.
Use DebugView utility to view and capture output. http://technet.microsoft.com/en-gb/sysinternals/bb545027
Usage eg:- RT_Debug("["+String(42)+"]","Going pretty good so far @","Line "+String(19))
Would Produce this "RT_Debug: [42] Going pretty good so far @ Line 19"
The above "RT_Debug:" title is present in all RT_Debug() output to the debugview window and you can (in DebugView),
filter out all lines that do not contain "RT_Debug:" so as to skip stuff that is of no interest (eg AVSPMod generates
some output, not sure but I think Avisynth v2.58 also generates lots of debug stuff).
NOTE, Any program can send output to DebugView window, if no debug viewer is present, it normally just disappears
down the plughole.
In DebugView, filter for eg "RT_Debug:" to skip all the extraneous stuff generated by other plugins/programs.
(In DebugView "Menu:Edit-Filter/Highlight", and enter "RT_Debug:".
v1.14, No longer converts characters 127->255 to SPACE, only chr(2) converted to chr(1).
Characters displayed via debugview are same as those via SubTitle, but those above 127 are dissimilar to the ones
produced via Avisynth source Info.h, used during eg plugin metrics display.
Example to show characterset with both RT_Debug & Subtitle:
# ShowCharacterset.avs # Requires GScript & RT_Stats v1.14+
S=""
GSCript("""
for(y=0,15) {
s=s+"$"+RT_Hex(y*16,2)+ " "
for(x=0,15) {
n=y*16+x
c=(n==0 || n==9 || n==10 || n==13) ? Chr(1) : Chr(n)
# Convert Odd Effect chars to NO GRAPHIC {Square} Chr(1)
# NOTE, DebugView
# Chr(0) {NUL} [Null or End of String], converted here to NO GRAPHIC Chr(1).
# Chr(9) {TAB} [Horizontal TAB], produces SPACE, converted here to NO GRAPHIC Chr(1).
# Chr(10) {LF} [Line Feed or NewLine], converted here to NO GRAPHIC Chr(1).
# Chr(13) {CR} [Carriage Return], converted here to NO GRAPHIC Chr(1).
#
# Chr(2) {SOH} [Start Of Heading ], Produces ODD effect , converted internally by RT_Debug to a NO GRAPHIC Chr(1).
s=s+c
}
s=s+Chr(10)
}
""")
colorbars(width=320,height=320).Trim(0,-1)
RT_Debug(s)
s=RT_StrReplace(s,Chr(10),"\n")
Subtitle(s,font="Courier New",lsp=0)
return Last

RT_StrReplace(string source,string find,string replace,bool "sig"=True) # Based on Algorithm by Vampiredom, Gavino & IanB.
String args 'source', 'find' and 'replace' unnamed and compulsory.
Takes a source string, searches for all occurences of find string and replaces the found strings with the replace string.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Newlines are treated no differently to other characters, and could be replaced/deleted.
v1.14, added 'sig' arg,default true is Case Significant (as previous version). Set false for case insignificant find string.

RT_StrReplaceDeep(string source,string find,string replace,bool "sig"=True)
String args 'source', 'find' and 'replace' unnamed and compulsory.
Takes a source string, searches for all occurences of find string and replaces the found strings with the replace string.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Newlines are treated no differently to other characters, and could be replaced/deleted.
Differs from RT_StringReplace in that if length of replace < length of find string will rescan after a replacement to see if
the replaced string combines with currently existing string to create another find string, If length replace >= find, then calls
RT_StringReplace instead. Useful to replace eg " " with " ", (two spaces with 1 space), where more than two consecutive
spaces would end up with more than one consecutive resulting space if using plain RT_StrReplace.
v1.14, added 'sig' arg,default true is Case Significant (as previous version). Set false for case insignificant find string.


RT_StrReplaceMulti(string source,string find,string replace,bool "sig"=True)
String args 'source', 'find' and 'replace' unnamed and compulsory.
Takes a source string, searches for all occurences of find substrings and replaces the found substrings with the replace substrings.
Find string and Replace string are both multiline [Chr(10) separated] strings, both of n lines where find[i] and replace[i]
are matched pairs, ie the first line of a multiline find is replaced by the first line of a multiline replace, etc.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Differs from RT_StrReplace, in that both find and replace strings are multiline strings and MUST match in number of lines.
Cannot use either carriage return [Chr(13)] or newline [Chr(10)] in find or replace as these are interpreted as end of current line,
you would need to use RT_StrReplace() instead.
eg S=RT_StrReplaceMulti("THE cat SAT.","SAT"+Chr(10)+"THE","sat"+Chr(10)+"The") produces "The cat sat.".
v1.14, added 'sig' arg,default true is Case Significant (as previous version). Set false for case insignificant find string.


RT_HexValue(String,"pos"=1)
Returns an int conversion of the supplied hexadecimal string.
Conversion will cease at the first non legal number base digit, without producing an error
Fixes HexValue bug in 2.58 & 2.6a3. eg "FFFFFFFF" returns 2147483647 (0x7FFFFFFF) instead of the correct -1 (As most/all calculators)
Bug is fixed in Avisynth v2.6a4.
v1.14, Added optional pos arg default=1, start position in string of the HexString, 1 denotes the string beginning. Will return 0
if error in 'pos' ie if pos is less than 1 or greater than string length.


RT_NumberValue(String,int "base"=10,int "pos"=1)
Returns an int conversion of the supplied number base string.
Base, (10, 2 -> 36) is the number base that the string uses.
To convert a Decimal string to a Float, use Avisynth native "Value()" func.
Conversion will cease at the first non legal number base digit, without producing an error.
eg RT_NumberValue("1100",2) returns 12 from the binary string.
v1.14, Added optional pos arg default=1, start position in string of the number string, 1 denotes the string beginning. Will return 0
if error in 'pos' ie if pos is less than 1 or greater than string length.




Result of the ShowCharacterSet.avs Script (doom9 swallows some of the chars, shows space,should look like NO GRAPHIC char $7F, square):


00000004 04:48:40 RT_Debug: $00
00000005 04:48:40 RT_Debug: $10
00000006 04:48:40 RT_Debug: $20 !"#$%&'()*+,-./
00000007 04:48:40 RT_Debug: $30 0123456789:;<=>?
00000008 04:48:40 RT_Debug: $40 @ABCDEFGHIJKLMNO
00000009 04:48:40 RT_Debug: $50 PQRSTUVWXYZ[\]^_
00000010 04:48:40 RT_Debug: $60 `abcdefghijklmno
00000011 04:48:40 RT_Debug: $70 pqrstuvwxyz{|}~
00000012 04:48:40 RT_Debug: $80 €�‚ƒ„…†‡ˆ‰Š‹Œ�Ž�
00000013 04:48:40 RT_Debug: $90 �‘’“”•–—˜™š›œ�žŸ
00000014 04:48:40 RT_Debug: $A0 *¡¢£¤¥¦§¨©ª«¬*®¯
00000015 04:48:40 RT_Debug: $B0 °±²³´µ¶·¸¹º»¼½¾¿
00000016 04:48:40 RT_Debug: $C0 ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏ
00000017 04:48:40 RT_Debug: $D0 ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß
00000018 04:48:40 RT_Debug: $E0 àáâãäåæçèéêëìíîï
00000019 04:48:40 RT_Debug: $F0 ðñòóôõö÷øùúûüýþÿ

[StainlessS]

New Version v1.15, see 1st post.
v1.15 pretty much same as v1.14, made small alteration to RT_YInRangeLocate, Internally, Baffle separately limited to dimensions on both axis.

Also added below script to v1.15 zip.


AVISource("D:\avs\test.avi")

# Pick a test below

#Return TestClip($FFFFFF).TweenFrames(LimitLo=200.0,LimitHi=255.0,Show=true) # Fix White frames PROGRESSIVE
#Return TestClip($000000).TweenFrames(LimitLo=0.0,LimitHi=32.0,Show=true) # Fix Black frames PROGRESSIVE
#Return TestClipI($FFFFFF).TweenFramesI(LimitLo=200.0,LimitHi=255.0,Show=true) # Fix White frames INTERLACED
#Return TestClipI($000000).TweenFramesI(LimitLo=0.0,LimitHi=32.0,Show=true) # Fix Black frames INTERLACED

Function TweenFrames(clip c,float "LimitLo",float "LimitHi",bool "show") {
# TweenFrames() by StainlessS
# PROGRESSIVE. Requires RT_Stats & MvTools 2
# Replace isolated bad frames eg Black or White with a frame tweened from those either side using MvTools, and RT_Stats.
# LimitLo: is minimum luma value of frame to replace. Allows select, eg black/White.
# LimitHi: is maximum luma value of frame to replace.
# Show: Puts indicator on fixed frames.
c
LimitLo=Float(Default(LimitLo,200.0)) # Average Luma greater or equal to this eligible for replacement
LimitHi=Float(Default(LimitHi,255.0)) # Average Luma lesser or equal to this eligible for replacement
show=Default(show,False)
Prev=DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("FRAME FIXED") : inter
CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>=LimitLo && ave<=LimitHi && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"]","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""
CondS=RT_StrReplaceMulti(CondS,"LimitLo"+Chr(10)+"LimitHi",String(LimitLo)+Chr(10)+String(LimitHi))
ConditionalSelect(Last,CondS,Last,inter) # Fix bad frames
return Last
}

Function TweenFramesI(clip c,float "LimitLo",float "LimitHi",bool "show") {
# TweenFramesI() by StainlessS
# INTERLACED.
# Replace isolated bad fields eg Black or White with a field tweened from those either side using MvTools, and RT_Stats.
# LimitLo: is minimum luma value of field to replace. Allows select, eg black/White.
# LimitHi: is maximum luma value of field to replace.
# Show: Puts indicator on fixed fields.
c
LimitLo=Float(Default(LimitLo,200.0)) # Average Luma greater or equal to this eligible for replacement
LimitHi=Float(Default(LimitHi,255.0)) # Average Luma lesser or equal to this eligible for replacement
show=Default(show,False)
SEP=SeparateFields()
Even=SEP.SelectEven()
Prev=Even.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("EVEN FIELD FIXED") : inter
CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>=LimitLo && ave<=LimitHi && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"] EVEN","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""
CondS=RT_StrReplaceMulti(CondS,"LimitLo"+Chr(10)+"LimitHi",String(LimitLo)+Chr(10)+String(LimitHi)) # Import explicit into script
EvenFixed=ConditionalSelect(Even,CondS,Even,inter) # Fix bad even fields
Odd=SEP.SelectOdd()
Prev=Odd.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("ODD FIELD FIXED",align=1) : inter
CondS=RT_StrReplace(CondS,"EVEN","ODD")
OddFixed=ConditionalSelect(Odd,CondS,Odd,inter) # Fix bad Odd fields
Interleave(EvenFixed,OddFixed)
Weave()
return Last
}

Function TestClip(clip c,int "Color") {
# Replace every 8th frame with blank.
c Color=Default(Color,$FFFFFF) KillAudio() BAD=Last.BlankClip(color=Color)
A=SelectEvery(8,0) B=SelectEvery(8,1) C=SelectEvery(8,2) D=SelectEvery(8,3)
E=SelectEvery(8,4) F=SelectEvery(8,5) G=SelectEvery(8,6)
Interleave(A,B,C,D,E,F,G,BAD) # every 8th frame replaced with BAD
Return Trim(0,1000)
}

Function TestClipI(clip c,int "Color") {
# Replace every 8th EVEN field and every 12th ODD field with a blank.
Color=Default(Color,$FFFFFF) c=c.KillAudio() SEP=c.SeparateFields BAD=SEP.BlankClip(color=Color) EVEN=SEP.SelectEven()
A=EVEN.SelectEvery(8,0) B=EVEN.SelectEvery(8,1) C=EVEN.SelectEvery(8,2) D=EVEN.SelectEvery(8,3)
E=EVEN.SelectEvery(8,4) F=EVEN.SelectEvery(8,5) G=EVEN.SelectEvery(8,6) EVEN=Interleave(A,B,C,D,E,F,G,BAD)
ODD=SEP.SelectODD() A=ODD.SelectEvery(12,0) B=ODD.SelectEvery(12,1) C=ODD.SelectEvery(12,2)
D=ODD.SelectEvery(12,3) E=ODD.SelectEvery(12,4) F=ODD.SelectEvery(12,5) G=ODD.SelectEvery(12,6)
H=ODD.SelectEvery(12,7) I=ODD.SelectEvery(12,8) J=ODD.SelectEvery(12,9) K=ODD.SelectEvery(12,10)
ODD=Interleave(A,B,C,D,E,F,G,H,I,J,K,BAD) Interleave(EVEN,ODD) Weave()
Return Trim(0,1000)
}


Mod of script posted here:-
http://forum.doom9.org/showthread.php?t=167632

EDIT: Maybe TweenFramesI should be called TweenFields.

[StainlessS]

New version RT_Stats v1.16. See 1st post.

Function added:

RT_LumaSceneChange(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value (0.0 -> 255.0) scene change detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Is implemented sort of like:- mt_lutxy(mt_polish("((x-y)/4)^2")).AverageLuma() where x=clip(n+delta) and y=clip2(n2+delta2).
Scene change metric between current frame and next frame, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,delta2=1,h=-32)
and exactly the same thing again:
SC = RT_LumaSceneChange(Last,Last,n2=current_frame+1,h=-32)
There are two arg clips for flexibility, might usually both be same, but might want to measure against two separate clips and so
implemented as above.
Scene change detection between current frame top field and bottom field, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,y2=1,interlaced=true,h=-32) # EDIT: Expect higher metrics here
A threshold of about 80.0 is pretty good for a hi/lo motion clip, above is scene change, below not scene change although metrics
before and after scene change would best be used as well.


See this post for usage example.
http://forum.doom9.org/showthread.php?p=1625975#post1625975

[StainlessS]

New Version RT_Stats v1.17, see 1st post.

v1.17, Added RT_LumaMovement. RT_Ord mod.


RT_LumaMovement(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3),
int "Mode"=0,int "BlkW"=32,int "BlkH"=32)
Returns FLOAT value (0.0 -> 255.0) movement detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
There are two modes available, 0 (Default) = RT_LumaDifference() : 1 = RT_LumaSceneChange().
BLKW/BLKH (default 32), Block size. Minimum 8 (silently limited to frame dimensions).
RT_LumaMovement() splits the frames into block sized lumps and calls the desired mode RT_LumaDifference or RT_LumaSceneChange
function to get a metric for each of those blocks. The metric returned is the metric with the highest value, and so is more
sensitive to localized movement.
If Interlaced is true, then only every other horizontal scanline in BLKH will be scanned.
v1.17, New.

RT_Ord(string,int "pos"=1)
Returns same as 'Ord' in v2.6, the ASCII code for the 1st letter in string (0 if empty string "").
v1.17, Added pos arg, default 1, ie 1st character in string. Returns 0 if pos < 1 or greater than string length.
Could possibly use a string as a sort of BYTE sized int array containing non zero values.


New Addition SecurityCamExtractMotion.AVS


AVISource("D:\AVS\AVI\CAP4.avi") #.Trim(32,0)

############################################
# SecurityCamExtractMotion.AVS, v1.0, By StainlessS.
# Requires RT_Stats v1.17, FrameSelect v1.03, GScript, Grunt, Maybe DeGrainMedian if required.
#
# DO NOT USE DirectShowSource OR DENOISER THAT NEEDS LINEAR ACCESS eg HQDN3D
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN ShowMetrics OR MakeFramesCmd.
############################################

X=32 Y=32 W=-32 H=-0 # Area Of Interest, avoid eg passing traffic or timecode.

MODE =1 # For RT_LumaMovement. 0=RT_LumaDifference, 1=RT_LumaSceneChange, Metrics will change with Mode.
BLKW =32 # Metrics will change with block size. For RT_LumaMovement().
BLKH =32 #
THRESH =2.25 # Below KEEP. Use ShowMetrics, need check against source. About 3.0 for Mode 1.

DENOISE=0 # DegrainMedian LimitY, 0 = OFF, higher stronger, Suggest 4 -> 8
DN_MODE=0 # DegrainMedian mode, see DegrainMedian docs, suggest 0 -> 2, higher weaker
SHOW=True # Frame Info
SHOWTIME=True # Show original Time on output
ROBOLEVELS=False # AutoLevel, need differnt Thresh
PC=False # PC Levels for ROBOLEVELS

dclip = (ROBOLEVELS) ? RoboLevels(x=X,y=Y,W=W,h=H,pc=PC) : Last
# Optional Denoise, perhaps there are better ones, (OK for non-linear access).
dclip = (DENOISE>0) ? dclip.ConvertToYV12().DeGrainMedian(limitY=DENOISE,limitUV=0,mode=DN_MODE).GreyScale() : dclip

##########################################
# Choose 1 of below options:- (uncomment)
##########################################

# Show metrics to get best Threshold 'th'. Denoised clip shown in GrayScale.
ShowMetrics(dclip,THRESH,X,Y,W,H,fast=true,showTime=SHOWTIME,mode=MODE,blkw=BLKW,blkh=BLKH)

# Make Frames Cmd file for SelectFrames(), Must Play all way through.
#MakeFramesCmd(dclip,THRESH,X,Y,W,H,mode=MODE,blkw=BLKW,blkh=BLKH)

# Return Frames with EXISTING frames Command file (after MakeFramesCmd)
#SelectFrames(show=SHOW,showTime=SHOWTIME)

# Return NON-selected frames, for scrutiny, with EXISTING frames Command file (after MakeFramesCmd)
#RejectedFrames(show=SHOW,showTime=SHOWTIME)

# Make Frames command file and then extract frames automatically in 1 pass.
#AutoGetFrames(THRESH,X,Y,W,H,dclip=dclip,show=SHOW,showTime=SHOWTIME,mode=MODE,blkw=BLKW,blkh=BLKH)

AssumeFPS(250.0)
return Last
##########################################

Function RoboLevels(clip c,float "Strength",int "Samples",float "Ignore",Bool "QBC",Float "QBC_Thresh",int "X",int "Y",int "W",int "H", \
int "Matrix",bool "PC",bool "DEBUG",bool "Interlaced",bool "DoCrop",Bool "AutoGain") {
c
Frames=FrameCount
Strength=Float(Default(Strength,0.95))
Samples=(!Defined(Samples)) ? 24 : (Samples > Frames) ? Frames : (Samples < 1) ? 1 : Samples
Ignore=Float(Default(Ignore,0.2))
QBC=Default(QBC,False)
QBC_Thresh=Float(Default(QBC_Thresh,-32.0))
QBCropXM = Default(X,0)
QBCropYM = Default(Y,0)
QBCropWM = Default(W,-0)
QBCropHM = Default(H,-0)
DoCrop = Default(DoCrop,True)
AutoGain = Default(AutoGain,false)
QBCropWM = (QBCropWM<=0) ? QBCropWM : Width -(QBCropXM+QBCropWM)
QBCropHM = (QBCropHM<=0) ? QBCropHM : Height-(QBCropYM+QBCropHM)
QBCropXP = QBCropXM+2
QBCropYP = QBCropYM+2
QBCropWP = QBCropWM-2
QBCropHP = QBCropHM-2
Matrix = Default(Matrix,(width <= 720) ? 2 : 3)
PC=Default(PC,False)
DEBUG=Default(DEBUG,False)
Interlaced=Default(Interlaced,true)
Assert(Strength>=0.0 && Strength <=1.0, "RoboLevels: 0.0 <= Strength <= 1.0")
GScript("""
if(QBC) {
QBCS=RT_QueryBorderCrop(samples=Samples,thresh=QBC_Thresh,debug=DEBUG,ignore=Ignore,laced=Interlaced,matrix=Matrix)
Eval(QBCS)
}
if(Strength>0.0||AutoGain) {
if(AutoGain) {
(DEBUG)?RT_Debug("AutoGain"):NOP
TMPW=QBCropWP/4*4 # Mod 4 for AutoGain
TMPH=QBCropHP/4*4 # Mod 4 for AutoGain
TMP=Crop(QBCropXP,QBCropYP,TMPW,TMPH)
AutoGain(Last,TMP)
} else {
QLMMS=RT_QueryLumaMinMax(samples=Samples,ignore=Ignore,debug=DEBUG,X=QBCropXP,Y=QBCropYP,W=QBCropWP,H=QBCropHP,matrix=Matrix)
(DEBUG)?RT_Debug(QLMMS):NOP
Eval(QLMMS)
if(IsRGB() || PC) {
CSMin = 0
CSMax = 255
} else {
CSMin = 16
CSMax = 235
}
ALMin = Int(CSMin - ((CSMin - QLMMMin) * Strength) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * Strength)) # Round down
(DEBUG)? RT_Debug("Levels("+String(ALMin)+",1.0,"+String(AlMax)+","+String(CSMin)+","+String(CSMax)+",Coring=False)"):NOP
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring
}
} else {
(DEBUG) ? RT_Debug("No Auto Levels") : NOP
}
if(QBC && DoCrop) {
RT_Debug("RoboLevels",String(QBCropXM),String(QBCropYM),String(QBCropWM),String(QBCropHM))
Crop(QBCropXM,QBCropYM,QBCropWM,QBCropHM)
}
""")
return Last
}



Function ShowMetrics(clip c,float th,int "x",int "y",int "w",int "h",bool "dim",bool "fast",bool "Showtime",int "Mode",int "BlkW",int "BlkH") {
# SUBTITLES,
# LINE 1, 1st number Frame number, 2nd Anchor frame number (current comparison frame), Remainder, delayed KEPT frames (avoid dups).
# LINE 2, Diff between Frame and Anchor, Action, KEPT Frame count, % of frames KEPT.
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) dim=Default(dim,true) Fast=Default(Fast,False)
ShowTime=Default(ShowTime,True) Mode=Default(Mode,0) BlkW=Default(BlkW,32) BlkH=Default(BlkH,32)
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
cropped=Crop(x,y,w,h)
(dim && (w!=width || h!=height)) ? levels(0,1.0,255,0,128).Overlay(cropped,x=X,y=Y,opacity=1.0) : Last
Global ShowMetrics_KEPT=0 # Count of kept frames
Global ShowMetrics_DEL7=0 Global ShowMetrics_DEL6=0 Global ShowMetrics_DEL5=0 Global ShowMetrics_DEL4=0
Global ShowMetrics_DEL3=0 Global ShowMetrics_DEL2=0 Global ShowMetrics_DEL1=0 Global ShowMetrics_DEL0=0
ScriptClip("""
d = RT_LumaMovement(Last,Last,n2=ShowMetrics_DEL7,x=x,y=y,w=w,h=h,mode=Mode,blkw=BlkW,blkh=BlkH)
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th)
Global ShowMetrics_KEPT = (keep) ? ShowMetrics_KEPT + 1 : ShowMetrics_KEPT
SS=String(current_frame) + "]{" + \
String(ShowMetrics_DEL7) + "}("+ String(ShowMetrics_DEL6) + "," + \
String(ShowMetrics_DEL5) + "," + String(ShowMetrics_DEL4) + "," + \
String(ShowMetrics_DEL3) + "," + String(ShowMetrics_DEL2) + "," + \
String(ShowMetrics_DEL1) + "," + String(ShowMetrics_DEL0) + ")\n" + \
"Diff="+String(d,"%-6.2f ") + ((keep)?"KEEP":"DROP") + \
" KEPT=" + String(ShowMetrics_KEPT) + " " + String(ShowMetrics_KEPT*100.0 / (current_frame+1.0),"%03.2f") + "%\n" + \
"Thresh="+String(th,"%.2f"+" Mode="+String(Mode)+" BlkW="+String(BlkW)+" BlkH="+String(BlkH))
Global ShowMetrics_DEL7=(keep)?ShowMetrics_DEL6:ShowMetrics_DEL7 # Follow but NOT too closely (delayed by 7)
Global ShowMetrics_DEL6=(keep)?ShowMetrics_DEL5:ShowMetrics_DEL6
Global ShowMetrics_DEL5=(keep)?ShowMetrics_DEL4:ShowMetrics_DEL5
Global ShowMetrics_DEL4=(keep)?ShowMetrics_DEL3:ShowMetrics_DEL4
Global ShowMetrics_DEL3=(keep)?ShowMetrics_DEL2:ShowMetrics_DEL3
Global ShowMetrics_DEL2=(keep)?ShowMetrics_DEL1:ShowMetrics_DEL2
Global ShowMetrics_DEL1=(keep)?ShowMetrics_DEL0:ShowMetrics_DEL1
Global ShowMetrics_DEL0=(keep)?current_frame:ShowMetrics_DEL0
Subtitle(SS,lsp=0,font="Courier New",size=18)
""",After_Frame=True,args="th,x,y,w,h,Mode,BlkW,BlkH") # Needs Grunt for args
(ShowTime) ? ShowTime() : NOP
return ((Fast)?AssumeFPS(250.0):Last) .KillAudio()
}

Function MakeFramesCmd(clip c,float th,int "x",int "y",int "w",int "h",String "FileName",int "Mode",int "BlkW",int "BlkH") {
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Mode=Default(Mode,0) BlkW=Default(BlkW,32) BlkH=Default(BlkH,32)
(Exist(FileName)) ? RT_FileDelete(FileName) : NOP # Delete existing
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
Global MakeFramesCmd_DEL7=0 # Frame to start comparing with (Anchor)
Global MakeFramesCmd_DEL6=0 Global MakeFramesCmd_DEL5=0 Global MakeFramesCmd_DEL4=0 Global MakeFramesCmd_DEL3=0
Global MakeFramesCmd_DEL2=0 Global MakeFramesCmd_DEL1=0 Global MakeFramesCmd_DEL0=0
ScriptClip("""
# We DONT compare with previous frame, we compare with a previous KEPT frame to avoid slow (and sneaky) burglars.
d = RT_LumaMovement(Last,Last,n2=MakeFramesCmd_DEL7,x=x,y=y,w=w,h=h,mode=Mode,blkw=BlkW,blkh=BlkH)
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th) # Keep BOTH FIRST and LAST frame ALWAYS
Global MakeFramesCmd_DEL7=(keep)?MakeFramesCmd_DEL6:MakeFramesCmd_DEL7
Global MakeFramesCmd_DEL6=(keep)?MakeFramesCmd_DEL5:MakeFramesCmd_DEL6
Global MakeFramesCmd_DEL5=(keep)?MakeFramesCmd_DEL4:MakeFramesCmd_DEL5
Global MakeFramesCmd_DEL4=(keep)?MakeFramesCmd_DEL3:MakeFramesCmd_DEL4
Global MakeFramesCmd_DEL3=(keep)?MakeFramesCmd_DEL2:MakeFramesCmd_DEL3
Global MakeFramesCmd_DEL2=(keep)?MakeFramesCmd_DEL1:MakeFramesCmd_DEL2
Global MakeFramesCmd_DEL1=(keep)?MakeFramesCmd_DEL0:MakeFramesCmd_DEL1
Global MakeFramesCmd_DEL0=(keep)?current_frame:MakeFramesCmd_DEL0
WriteFileIf(FileName, "(keep)", "current_frame", append=True)
""",After_Frame=True,args="th,x,y,w,h,FileName,Mode,BlkW,BlkH") # Needs Grunt for args
return AssumeFPS(250.0).KillAudio()
}


Function SelectFrames(clip c,String "FileName",bool "Show",bool "Showtime") {
c
FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Show=Default(Show,False)
ShowTime=Default(ShowTime,False)
(ShowTime) ? ShowTime() : NOP
FrameSelect(CMD=FileName,show=Show) # Returns NO audio
}


Function RejectedFrames(clip c,String "FileName",bool "Show",bool "Showtime") {
c
FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Show=Default(Show,False)
ShowTime=Default(ShowTime,False)
(Showtime) ? ShowTime() : NOP
FrameSelect(CMD=FileName,show=Show,reject=True) # Returns NO audio
}


Function AutoGetFrames(clip c,float th,int "x",int "y",int "w",int "h",String "FileName",clip "dclip",bool "Show",bool "Showtime", \
int "Mode",int "BlkW",int "BlkH") {
Default(dclip,c) # Use denoised clip for scanning
Mode=Default(Mode,0) BlkW=Default(BlkW,32) BlkH=Default(BlkH,32)
MakeFramesCmd(th,x,y,w,h,FileName,mode=MODE,blkw=BLKW,blkh=BLKH) # Make the command file for FrameSelect()
GSCript("""
For(i=0,framecount-1) {
RT_YankChain(n=i) # Force Process, MakeFrames
}
c.SelectFrames(FileName,Show=Show,showtime=Showtime) # Extract source frames and return to client
""")
}


Works really well. :)

One clip about 11:00 PM, cut down to about 10% of frames, where about 25 vehicles, 3 cyclists, 2 pedestrians and one fox caused
motion detection. Original clip about 18 mins. Dark, rain shimmering under street lamps.

2nd clip, about 02:00 AM cut down to about 3.5% of frames, where about 9 vehicles caused motion detection.
Original clip about 18 mins. Dark, little activity.

[Forensic]

Stainless. I look forward to testing your motion extraction against DeDup, but I can't locate the download link for the current version.

[StainlessS]

5th link from bottom in sig or here:-
http://www.mediafire.com/StainlessS

however, it's not for removing duplicate frames, its for extraction of movement in eg security camera clips,
eg only having to watch the interesting bits where the burglar is paying your building site a visit.

Or for those of you that like watching the nightlife of the local fauna.

[StainlessS]

@Forensic
It would be nice if you could post findings in comparison with dedup, it did not really occur
to me to try dedup for this purpose.

EDIT: By the way, the delays in the script are intended (in part) to keep duplicate frames in areas of movement,
it perturbs me to see movement in the rejected frames.
I cant really do a good test during daylight hours as outside my window is like a ruddy motorway during the day.

[Forensic]

I can't get past a "FrameSelect: Cannot Open Cmd File FrameSelect_Cmd.Txt" error. So I uploaded my test file to (https://www.yousendit.com/download/WFJWd0VFQXAyWGR4Tk1UQw). The only motion in that video, other than minor real-world camera fluctuations, occur at frames 3390 and 3804.

Three passes with DeDup did a perfect job. I simply nest each resulting output pass into the next input so only the end results gets saved.

[StainlessS]

You need to use MakeFramesCmd to create the text file before the other functions, except
ShowMetrics() which does not need/use frame command file, or AutoGetFrames which makes it's own frame command file.

[StainlessS]

By the way, an https account seems to be private to you, ie cannot access it.

[Forensic]

@StainlessS. I marvel at your programming creations and RT_Stats is quickly becoming a "must have" tool among my fellow Forensic Video Analysts. However, I think I will stick with my DeDup solution to isolate motion frames. Thank you though. As for the test video, it is not a private link but it does require a secure socket connection. Try the unsecure alternative http://www.yousendit.com/download/WFJWd0VFQXAyWGR4Tk1UQw

[StainlessS]

Thankyou, I did try that but presume that failure was due to my crap connection, been having a terrible connection since about November.
Constantly disconnected, Three.co.uk SuperFast Speed Mobile Broadband, 2kb-4kb/s during the day is a regular occurence,
good job I dont ffffing swear much (my neighbour thinks I've got Tourette's).

[StainlessS]

For your clip, I think mode 1 is overkill (over sensitive), need mode 0 with a thresh of maybe ~24, cuts down to 29 frames.
Need to avoid them there flowers, waving around in the wind.

Can you post your dedup script for that clip (bit of a phoney clip by the way).

EDIT: Perhaps mode 1 is better for low light captures (lots of fluctuating dark noise), Mode 0 for daylight.
I've only really tested much with night caps.

[Forensic]

th=1.0 # I do not recall the exact value that I used
# uncomment each successive line per "video analysis pass"
DupMC(log="1.dup")
#DeDup(threshold=th,maxcopies=19,maxdrops=18,log="1.dup").DupMC(log="2.dup")
#DeDup(threshold=th,maxcopies=19,maxdrops=18,log="1.dup").DeDup(threshold=th,maxcopies=19,maxdrops=18,log="2.dup").DupMC(log="3.dup")
#DeDup(threshold=th,maxcopies=19,maxdrops=18,log="1.dup").DeDup(threshold=th,maxcopies=19,maxdrops=18,log="2.dup").DeDup(threshold=th,maxcopies=19,maxdrops=18,log="3.dup")

I am sure that your method is faster, but this works for the rarity that I need such a feature.

[Forensic]

#StainlessS
Is there any way to use RT_Stats to return the estimated resulting compressed file size of the video file and MPEG CoDec name passed to it (without actually saving the file)? If so, I will create a script to automatically test dozens of denoising/degraining filter combinations to determine which produces the clearest result, and thus the smallest file size. When dealing with each surveillance video, the best filter combination is rarely the one that I last used.

This need comes up weekly for me and thousands of times for my industry. Currently we have to encode using VirtualDub (2.5) to view the estimated resulting file size, or let it finish writing the file. This becomes quite time consuming and I am all about automation.

I do not mind, and actually prefer, if the solution runs through the entire encoding process (without writing anything to the hard drive) so it could return the most accurate final file size. If this ability is/can not be in RT-Stats, maybe it can be a portion of the solution. As an added bonus, such a feature would also allow the user to confirm that they have ample hard drive space before committing a save.

[StainlessS]

Sorry, I dont have a clue how to do that.

[StainlessS]

Forensic
You might be able to knock up something in AutoIt script (quite similar to basic), to start VD (or whatever), run encode and after maybe 5 mins, read
estimated output size, then abort encode and try again. You could call the compiled autoit exe from Avisynth script
and read data written by autoit exe into some file.
http://www.autoitscript.com/site/autoit/
Also get the editor (Scite4AutoIt or something like that, linked in downloads, better than the basic editor + tools).

You might find MegUIAutoEncode and DGIndex Batchers source useful especially the utility functions. (Mediafire in sig)

AutoIt has an unbelievable number of libraries to do all sorts of things including eg Post OS SysPrep setup stuff
to set Windows Defaults, install Software, configure registry etc.

EDIT: The DGIndex Batcher, waits for DGIndex to show the "FINISHED" text in info box so that it knows
that it has completed it's operation, similar could be used with VD to get estimated file size.

[Forensic]

I had never heard of autoit. It looks like fun to learn and has tremendous tutorial resources, including this website (Doom9 is amazing). I have also never worked with "DGIndex" or "MeGUI AutoEncode Batcher", the later of which was written by you (no surprise there). I can't fathom how you can maintain a paying career or personal life (let alone ever sleep) and still contribute as much as you do to help others. Your programming contributions have actually saved lives, including RT-Stats role in enhancing images of the Boston Marathon bombers when expediency mattered most. A debt of appreciation we can never repay. Thank you StainlessS.

[StainlessS]

DGIndex Batcher will probably be the better template to use as a starting point rather than MeGUI Auto Encode Batcher.
If you've programmed in BASIC before, it should be reasonably easy to pick up, took me about 1.5 days to feel relatively comfortable.
Not sure, I think I may have seen Selur post some AutoIt stuff.

Thank you for your nice words, glad to see it being of use.

EDIT: As far as coding is concerned, I normally (broke two fingers last night, I think, was a bit alcohol laden, typing has suffered) am a pretty
fast coder. When I was coding C and Assembler full time (until about 1996) I was convinced that I could think in C (sounds daft but true).
EDIT: Wish I could think in Machine Code, that would be real fun, dreaming in M/C. :eek:

[martin53]

StainlessS,
would you mind to spend some thoughts about a potential use case of the built in and RT_Stats statistic functions?

I currently check if it would be possible to make an automatic white balance script.
I think, although I don't yet have proof of it, that the first task is to find areas in the image that show white/grey objects, skin or maybe sky. Then their U/V values should be measured against a standard value and the bias reduced.
For the first task, I made a line of AviSynch code that selects the parts of the frame that are inside of a certain Y/U/V range and deliver the U-V bias (c is the clip)
ymin = "150"
ymax = "240"
uMin = "120"
uMax = "135"
vMin = "120"
vMax = "135"
c = c.ConvertToYV24()
# ( yMin<Y<yMax && uMin<U<uMax && vMin<V<vMax ) ? ( U - V + 128 ) : ( 0 )
mt_lutxyz(c, c.UToY(), c.VToY(), expr="x "+ymin+" > x "+ymax+" < & y "+uMin+" > & y "+uMax+" < & z "+vMin+" > & z "+vMax+" < & y z - 128 + 0 ?", chroma="128")
That function delivers grey values " U - V + 128 ", i.e. something around 128, and if the average is above 128, then I expect the frame to be too blue and vice versa.

The point is now, that it is difficult (ahem, almost impossible) to retrieve the statistics, because most of the frame is black and shall be left unconsidered. But no statistic function supports a mask, all evaluate the whole frame (x,y,w,h not of interest here).

The one solution I know of is: use RT_YInRange(lo=0,hi=0) to get the amount of black, and divide the RT_AverageLuma by (1.0 - YInRange) - no surprise since I asked you for the RT_YInRange() function for a similar problem. It works because I can be sure that black is outside the useful luma range and because it does not contribute to the average. Similar tweaks can be used to get other statistical numbers, e.g. return 255 for the invalid pixels when RT_YPlaneMin() is needed. But that needs repeated calls of the mt_lutxyz() function and slows down things.

Currently I feel it's obvious that all statistic functions can profit from an additional mask clip. With that option, I would probably prepare a clip "clip UV = c.U - C.V + 128" (with mt_lutxy() ), make the mt_lutxyz() function return the mask instead of U-V; and then query min,max,average and all that from the UV clip.

All comments appreciated!

[StainlessS]

Martin53, do I have this right,

You want funcs where only pixels taken into account are those where same coords in mask clip are non zero ?

EDIT: Or if not non zero, what ? I've actually just done the plug but as yet no testing whatever. Only supports Planar.
As for other RT stats Y funcs, n,delta,x,y,w,h etc.

EDIT: At the moment, I've got it returning -1 for all funcs except YStats equivalent where it returns flgs=0,
when there are no pixels to return anything for.

I could implement a range of mask values to process pixels for eg MaskMin, MaskMax inclusive, a single variable value, or a fixed
value eg 0.

What you want ?

[martin53]

I could implement a range of mask values to process pixels for eg MaskMin, MaskMax inclusive, a single variable value, or a fixed
value eg 0.

What you want ?

Hi StainlessS,
makes me happy that you like the idea and even work already on an implementation!

... I did not think all alternatives to the end so I'm also unsure what would be the best way to evaluate the mask for the stats functions. First, I thought that a multiplication with (mask luma value / 255) would be the most intuitive implementation. But I can imagine no real useful application for a weighted average, maximum etc. One could 'dim' the mask edges and give more weight on the center, for example. But what purpose would that be useful for?

Your proposal MaskMin MaskMax sounds much more useful: one could prepare a mask with several discrete values, and then re-use the same mask for a number of statistics queries. E.g. a mask value stands for 'about white', another one for 'sky blue' or 'skin'. That would support a much faster approach than repeated mt_lutxyz() mask creations.

Maybe there is someone else besides us who might want to share his opinion? If not, I think I favour MaskMin, MaskMax.

EDIT: only pixels taken into account where same coords in mask clip are valid. Yes, exactly.

EDIT: and I don't think it is useful to use chroma of the mask with chroma stats. Better always use luma, so luma and chroma query take the same pixels into account and no repeated mask preparation is needed. It's always possible to use UtoY() when someone really needs U of a clip as the mask.

[StainlessS]

OK, I'll proceed and give it a little testing and hand over to you.

Currently with names eg YStatsM, etc.
EDIT: Changed to MYStats

[StainlessS]

Martin53, Here, v2.6 dll + source
http://www.mediafire.com/download/cw93p3rqxp6g5t6/MyStats.zip

Script I used to create the maskclip

c=Avisource("D:\avs\test2.avi")

ymin = "150"
ymax = "240"
uMin = "120"
uMax = "135"
vMin = "120"
vMax = "135"

c = c.ConvertToYV24()
mt_lutxyz(c, c.UToY(), c.VToY(), expr="x "+ymin+" > x "+ymax+" < & y "+uMin+" > & y "+uMax+" < & z "+vMin+" > & z "+vMax+ \
" < & y z - 128 + 0 ?", chroma="128")
Return ConvertToYV12()


A basic test clip, not much other testing done

c=Avisource("D:\avs\test2.avi")
Mask=Avisource("D:\avs\Mask.avi")

LO=128
HI=LO+24
THRESH=0.0
PREFIX="MYS_"

c.ScriptClip("""
v = MYPlaneMin(Mask,threshold=THRESH)
S=(V<0) ? "NONE" : \
"\nMin="+String(v) + \
"\nMax="+String(MYPlaneMax(Mask,threshold=THRESH)) + \
"\nMnMxDiff="+String(MYPlaneMinMaxDifference(Mask,threshold=THRESH)) + \
"\nMed="+String(MYPlaneMedian(Mask)) + \
"\nAve="+String(MAverageLuma(Mask),"%.2f") + \
"\nStdev="+String(MYPlaneStdev(Mask),"%.2f") + \
"\nInRng="+String(MYInRange(Mask,lo=LO,hi=HI),"%.2f")
flgs=MYStats(Mask,threshold=THRESH,lo=LO,hi=HI,flgs=$7F,prefix=PREFIX)
S2="\nflgs="+String(flgs)
S3="\nMin="+(RT_BitTst(flgs,0) ? String(MYS_yMin) : " ---") + \
"\nMax="+(RT_BitTst(flgs,1) ? String(MYS_yMax) : " ---") + \
"\nMnMxDiff="+(RT_BitTst(flgs,2) ? String(MYS_yMinMaxDiff) : " ---") + \
"\nMed="+(RT_BitTst(flgs,3) ? String(MYS_yMed) : " ---") + \
"\nAve="+(RT_BitTst(flgs,4) ? String(MYS_yAve,"%.2f") : " ---") + \
"\nStdev="+(RT_BitTst(flgs,5) ? String(MYS_yStdev,"%.2f") : " ---") + \
"\nInRng="+(RT_BitTst(flgs,6) ? String(MYS_yInRng,"%.2f") : " ---")
Subtitle(String(current_frame)+" ]" + S+S2+S3,lsp=0)
return Last
""")

return StackHorizontal(Mask).convertToRGB32()


Watch out for using Subtitle over YV24, I think it uses OverLay and hence the YV24 Overlay bug will exhibit itself.

EDIT: Args (default Prefix="MYS_")


"MYPlaneMin", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[MaskMin]i[MaskMax]i"
"MYPlaneMax", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[MaskMin]i[MaskMax]i"
"MYPlaneMinMaxDifference", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[MaskMin]i[MaskMax]i"
"MYPlaneMedian", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[MaskMin]i[MaskMax]i"
"MAverageLuma", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[MaskMin]i[MaskMax]i"
"MYPlaneStdev", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[MaskMin]i[MaskMax]i"
"MYInRange", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[lo]i[hi]i[MaskMin]i[MaskMax]i"
"MYStats", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[lo]i[hi]i[flgs]i[prefix]s[MaskMin]i[MaskMax]i"



EDIT: Oops, MaskMin default = 128, MaskMax default = 255.

EDIT: I could also implement for both YUY2 and RGB/24/32 but with the Mask being Planar Only, wadya think.
(in which case the Matrix arg will be inserted at positions similar to original funcs).

[martin53]

StainlessS,
thank you!
I am going to test it with my footage and adapted mt_... clips.
Can't guarantee a feedback to you and the doom9 community tomorrow since I'm refurbishing the living room...

[StainlessS]

RT_Stats v1.18, New version.

v1.18, Fixed RT_QueryBorderCrop RGB "Matrix NOT @ PC levels - Check!" bug.
Added RT_ColorSpaceXMod/YMod. Added RT_RgbChan family.
Added v2.6 dll

Stuff added:

RT_ColorSpaceXMod(clip)
Return int, the natural cropping XMod for clip colorspace, eg YV411=4, YV12=2, YUY2=2, RGB=1
v2.5 plugin limited to v2.58 colorspaces.

RT_ColorSpaceYMod(clip,bool "Laced"=true)
Return int, the natural cropping YMod for clip colorspace, eg YV411=1, YV12=2, YUY2=1, RGB=1
Laced, bool. Default true. If Laced==true, returns doubled YMod.
v2.5 plugin limited to v2.58 colorspaces.

RT_RgbChanMin(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0)
Returns int minimum value (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA]). Threshold is a percentage, stating how many percent of the pixels are allowed below minimum.
The threshold is optional and defaults to 0.0


***
***
***

RT_RgbChanMax(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0)
Returns int maximum value (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA]). Threshold is a percentage, stating how many percent of the pixels are allowed below minimum.
The threshold is optional and defaults to 0.0

***
***
***

RT_RgbChanMinMaxDifference(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
bool "interlaced"=false,float "threshold"=0.0,int "chan"=0)
Returns int value range maximum - minimum difference (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA]). Threshold is a percentage, stating how many percent of the pixels are allowed below minimum.
The threshold is optional and defaults to 0.0

***
***
***

RT_RgbChanMedian(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "chan"=0)
Return the int channel median (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA])

***
***
***

RT_RgbChanAve(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0, int "h"=0,bool "interlaced"=false,int "chan"=0)
Return the float average value (0.0 -> 255.0) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA])

***
***
***

RT_RgbChanStdev(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "chan"=0)
Returns FLOAT value Standard Deviation (0.0 -> 255.0) for an RGB channel in frame(n+delta) for area x,y,w,h.
Sample standard deviation http://en.wikipedia.org/wiki/Standard_deviation
(chan default = 0 = R [RGBA])


***
***
***

RT_RgbChanInRange(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "chan"=0,
int "lo"=128,int "hi"=lo)
Returns float value (0.0 -> 1.0) being the amount of pixels with a channel value in the range "lo" to "hi" (inclusive), 1.0 is
equivalent to 100%. NOTE, differs from other funcs that return range 0.0 to 255.0. NOTE, lo defaults to 128, "hi" defaults to "lo".
(chan default = 0 = R [RGBA])

***
***
***

RT_RgbChanStats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0,int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="RCS_")
Returns multiple results as for above single frame RGB sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", and "chan", are as for all other RGB clip functions, "threshold" used only for "RT_RgbChanMin",
"RT_RgbChanMax" and "RT_RgbChanMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_RgbChanInRange" equivalent routine with same functionality.
The only new args are int "Flgs" which selects which results you want returned and the string "Prefix" that is prepended
to the returned global variable names.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.

Flgs_Bit_Number Add_To_Flgs Equivalent_Function Global_Var_Set_Excluding_Prefix(Where 'x' is channel number)
0 1($01) RT_RgbChanMin() "Min_x" (0->255)
1 2($02) RT_RgbChanMax() "Max_x" (0->255)
2 4($04) RT_RgbChanMinMaxDifference() "MinMaxDiff_x" (0->255)
3 8($08) RT_RgbChanMedian() "Med_x" (0->255)
4 16($10) RT_RgbChanAve() "Ave_x" (0.0->255.0)
5 32($20) RT_RgbChanStdev() "Stdev_x" (0.0->255.0)
6 64($40) RT_RgbChanInRange() "InRng_x" (0.0->1.0)
RT_RgbChanStats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=127($7F) are all bits set and so sets ALL global vars at once.
RT_RgbChanStats(chan=0,flgs=1+2+16) would set global vars "RCS_Min_0", "RCS_Max_0" and "RCS_Ave_0" for full frame current_frame.



See 1st post.

[StainlessS]

New version RT_Stats v1.19, see 1st post.


v1.19, RT_LumaSceneChange() moved cap @ 255.0 to just before return instead of on Lut, better metrics.
Originally like:- mt_lutxy(mt_polish("((x-y)/4)^2")).AverageLuma() where was capped @ 255 @ equivalent mt_lutxy step.
Changed RT_QueryBorderCrop() Thresh Massaging.

[Forensic]

How can I use RT-Stats to return the 6 character RGB (or 8 character RGBa) value of one user specified pixel on one specified frame in RGB-24 (or RGB-32) color space?
EDIT: RT_RgbChanMedian did the trick one channel at a time. :)

[StainlessS]

Forensic, glad you got it sorted.
Min, max, median and average should all return identical results for a single pixel.

EDIT: this should also be able to do it in a single call although for RGB32 could return a -ve int.

RT_RGB32AsInt(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0)
Compile/Runtime clip function.
Given an RGB32 clip that had a pixel value created from an Int, gets that pixel and returns as the
original Int. By creating single pixel clips and stacking them horizontally/vertically,
you can use a frame as a two dimensional array of Int, perhaps use the frame number for a third dimension.


EDIT

6 character RGB (or 8 character RGBa) suggests you want Hex, this would convert even -ve
int to hex

RT_Hex(int , width=8)

Or

RT_NumberString(int ,base=16, int width=8)

v2.58 has bug in Hex function for -ve numbers.

[StainlessS]

RT_Stats v1.20, new version.

Changed the way RT_LumaSceneChange works. Added Bias->Pord args. Added RT_Subtitle(), Added RT_LumaSceneChange_Graffer with vb source.

Mod:

RT_LumaSceneChange(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3), \
Float "Bias"=0.0,Float "Gain"=1.0,Float "Cont"=1.0,Float "Rpow"=1.0,Float "Spow"=1.0,Float "SPMid"=0.5,Bool "Pord"=false)

Returns FLOAT value (0.0 -> 255.0) scene change detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
v1.20, Modified how it works. Creates a look up table (lut) which is used for the pixel differences between same pixel in frames
(n+delta) and (n+delta2). The lut is created using args "Bias" through to "Pord" and a sort of levels type internal function
creates the lut. An absolute pixel difference is taken between corresponding pixels and then the contents of the lut for that difference is
added to the sum of differences, and finally the average is taken. Is sort of like mt_lutxy("x y - abs").Lut().AverageLuma()
Additional Args:
Bias, Default 0.0, (-512 -> 512). Equivalent to Bias in RGBAdjust(), ie brightness.
Gain, Default 1.0, (-8.0 -> 8.0). Equivalent to Gain in RGBAdjust(), varies slope.
Cont, Default 1.0, (-8.0 -> 8.0). Contrast. A bit like Cont in ColorYUV.
RPow, Default 1.0, (0.1 -> 4.0). Similar to Gamma.
SPow, Default 1.0, (0.1 -> 4.0). S shaped power curve.
SPMid,Default 0.5, (0.01 -> 0.99). Mid point control for SPow.
Pord, Default False (False -> True). Governs which power function is applied first. False (default) applies RPow 1st,
True Applies SPow 1st. Only applies if both power functions not at default 1.0.
All Defaults for lut creation are linear and will be equivalent to RT_LumaDifference(). As currently implemented, may provide
alternative functionality other than scene change detection, but no idea what those applications could be.
A graphing program with VB source is provided in the zip, so you can play with sliders to choose function args.
Func versions previous to v1.20, used args similar to:- gain=4.0,rpow=0.5 with remaining args at default.
The old default, gave less prominence to big differences so that a few pixels with massive difference were limited in
the overall effect on results. New args produce a linear LUT by default.

Scene change metric between current frame and next frame, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,delta2=1,h=-32)
and exactly the same thing again:
SC = RT_LumaSceneChange(Last,Last,n2=current_frame+1,h=-32)
There are two arg clips for flexibility, might usually both be same, but might want to measure against two separate clips and so
implemented as above.
Scene change detection between current frame top field and bottom field, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,y2=1,interlaced=true,h=-32) # Expect higher metrics here.


***
***
***

RT_LumaMovement(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3),
int "Mode"=0,int "BlkW"=64,int "BlkH"=64 \
Float "Bias"=0.0,Float "Gain"=1.0,Float "Cont"=1.0,Float "Rpow"=1.0,Float "Spow"=1.0,Float "SPMid",Bool "Pord"=false)
Returns FLOAT value (0.0 -> 255.0) movement detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
There are two modes available, 0 (Default) = RT_LumaDifference() : 1 = RT_LumaSceneChange().
BLKW/BLKH (default 64), Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
RT_LumaMovement() splits the frames into block sized lumps and calls the desired mode RT_LumaDifference or RT_LumaSceneChange
function to get a metric for each of those blocks. The metric returned is the metric with the highest value, and so is more
sensitive to localized movement. Best control of localized movement sensitivity is by changing blk size, lower is more sensitive.
If Interlaced is true, then only every other horizontal scanline in BLKH will be scanned.
v1.20, See RT_LumaSceneChange() for additional args when Mode=1 (ignored when Mode = 0).


Added

RT_Subtitle(clip source, string format, dat1,...,datn,int "align",int "x","y",bool "vcent"=false,bool "expx"=false,bool "expy"=false )
Standard filter function.
This function is a standard filter graph function, not a runtime/compile time function.
The v2.5 plugin limited to v2.58 colorspaces.
Prints formatted text to clip frame using Avisynth Info.h source font and DDigit font renderer, faster than using Subtitle
system fonts. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the text string that is
printed on frame, uses C/CPP printf() style formatting. The args 'align', 'x', and 'y', work in a similar fashion to
Avisynth Subtitle() eg
RT_Subtitle(c,"Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013,align=4,x=100,vcent=true)
would print

Hello there Fred and Ted.
Goodbye 2013.

at x pixel position 100 and y pixel position centered, the vcent=true will vertical center a block of text (align=4 or 5 or 6),
whereas Subtitle() starts at centred position and prints downwards from there (as default vcent=false).

Align, default = 7 ie top left (as on numeric keypad).
x and y allow modification of alignment, eg x= -1, centers x at mid point, x=width, would right align,
y=height would align at bottom of frame.
It is necessary to specify the arg names of x,y,align and vcent, as the dat args can take 0 or more variables
of any type, so Avisynth cannot tell when the list of dat args ends and the named optional args begins. I did
try to use the dat args last but Avisynth seems to get confused when you use dat= etc, it throws an error
saying something like "dat arg specified more than once", even if it was not.
printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned.
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

eg
FMT="%d ] \a%cAverageLuma=\a-%6.2f"
ALPHA="0123456789ABCDEFGHIJKLMNOPQRSTUV-!"
ScriptClip("""
RT_Subtitle(FMT,current_frame,RT_Ord(ALPHA,current_frame%16+1),AverageLuma(),align=5)
""")

In the above example you see the sequence "\a%c" where %c is replaced with a character from ALPHA, "\a0"
selects color 0, "\aV" selects the last color (31). "\a-" selects the default color, and "\a!" selects
the hilite color. You can in Scriptclip() select colors programmatically, perhap selecting red to show an
error or bad metrics condition. The initially selected color will always be the default color (white).
Ripped from DDigit source code:
// Color control codes as strings. ASCII Code of Final char
#define DDIGIT_CC_HILITE "\a!" // 33
#define DDIGIT_CC_DEFAULT "\a-" // 45
#define DDIGIT_CC_DARKGRAY "\a0" // 48
#define DDIGIT_CC_DODGERBLUE "\a1" // 49
#define DDIGIT_CC_ORANGERED "\a2" // 50
#define DDIGIT_CC_ORCHID "\a3" // 51
#define DDIGIT_CC_LIME "\a4" // 52
#define DDIGIT_CC_AQUAMARINE "\a5" // 53
#define DDIGIT_CC_YELLOW "\a6" // 54
#define DDIGIT_CC_WHITE "\a7" // 55
#define DDIGIT_CC_SILVER "\a8" // 56
#define DDIGIT_CC_CORNFLOWERBLUE "\a9" // 57
#define DDIGIT_CC_ORANGE "\aA" // 65
#define DDIGIT_CC_PLUM "\aB" // 66
#define DDIGIT_CC_CHARTREUSE "\aC" // 67
#define DDIGIT_CC_POWDERBLUE "\aD" // 68
#define DDIGIT_CC_GOLD "\aE" // 69
#define DDIGIT_CC_GAINSBORO "\aF" // 70
#define DDIGIT_CC_Y_0 "\aG" // 71
#define DDIGIT_CC_Y_1 "\aH" // 72
#define DDIGIT_CC_Y_2 "\aI" // 73
#define DDIGIT_CC_Y_3 "\aJ" // 74
#define DDIGIT_CC_Y_4 "\aK" // 75
#define DDIGIT_CC_Y_5 "\aL" // 76
#define DDIGIT_CC_Y_6 "\aM" // 77
#define DDIGIT_CC_Y_7 "\aN" // 78
#define DDIGIT_CC_Y_8 "\aO" // 79
#define DDIGIT_CC_Y_9 "\aP" // 80
#define DDIGIT_CC_Y_A "\aQ" // 81
#define DDIGIT_CC_Y_B "\aR" // 82
#define DDIGIT_CC_Y_C "\aS" // 83
#define DDIGIT_CC_Y_D "\aT" // 84
#define DDIGIT_CC_Y_E "\aU" // 85
#define DDIGIT_CC_Y_F "\aV" // 86
In addition to color control codes, the filter will accept "\n" or "\r" as newline, and "\f" as forward
space (move 1 right without printing anything), no other control codes are supported. "\t" and TAB control
codes are converted to a single space, I could think of no sensible way to handle tab when center or
right aligned, if there is a demand, I could maybe handle tab for left alignment only and replace with single space
for other alignment.
To embed a single BackSlash ("\") into the format string or even an argument dat string you must insert a double
BackSlash ie "\\". To embed a single percent character ("%") into a format string, you must embed a double percent
sequence ie "%%". Reason is that such characters are used for formatting eg "%f" flags a float arg to embed into
the string and eg "\n" would mean insert newline into string.
Note, the colored text is not great in YV12 and quite awful in YV411, try to avoid YV411 at all times,
tis truly nasty.
To Print contents of multiline text file on frame:

AVISource("D:\AVS\TEST.AVI")
S=RT_ReadTxtFromFile("D:\avs\avi\test.txt")
RT_Subtitle("%s",S,align=5,vcent=True)

As can be seen from above example, you can also use eg Chr(10) instead of "\n" or Chr(13), or Chr(13)+Chr(10)
instead of "\r\n" with identical results, CR, NL and CR/NL pairs are internally replaced by a single Chr(13).
To Print contents of multiline text file on frame, Scrolling upwards as for end credits:
AVISource("D:\AVS\TEST.AVI")
S=RT_ReadTxtFromFile("D:\avs\avi\test.txt")
DELAY=100
Scriptclip("""
RT_Subtitle("%s",S,align=5,y=height+DELAY-current_frame,expy=true)
""")
The above example uses the "expy" arg, which tells the filter to NOT interpret y == -1 as screen vertical
center align and to NOT interpret y=height as bottom aligned. ExpX does the same for the x arg. If you change
above to ExpY=false, as the text scrolls on screen and again when text scrolls off top of frame, there will be
a single frame where y==height OR y==-1, will align the text an cause a glitch in the video clip.
You could also embed eg color control codes into any above such scrolling text file for your amusement, eg
embedding "\a!" into the text file would switch on Hi-Lite-ed text.
If you want you can run above script with this text file as "test.txt" and because it has color control codes
embedded into it (eg the ripped DDigit source code earlier) so it will show the file, some lines of which will be
colored. This text file has NOT been prepared for that particular reason but may demo the embedded color codes.
RT_Subtitle() has the advantage of not requiring string memory as the string is created internally and
released on plugin destruction, whereas SubTitle() within ScriptClip will keep accumulating string memory
which is not released until Avisynth closes. Of course any eg concatenated strings supplied to RT_Subtitle
will be accumulated unless eg assigned to string variable external to Scriptclip.
The RT_Subtitle filter is intended to be used in providing on-frame metrics in a script, and not for final
output, use SubTitle() for that.
For any script using standard Subtitle() for metrics, it would not be at all surprising if that script were
to double (or more) in speed when replaced by RT_Subtitle, it really is quite light weight in comparison.


See 1st post.

[StainlessS]

New function contained in RT_Stats v1.20 DynaCrop.avs, equivalent but more robust version of SCSelect()


Function SC_SceneSelect(clip dclip,clip start,clip end,clip motion,
\ float "dfact",float "minim",
\ Float "Bias",Float "Gain",float "Cont",Float "RPow",Float "SPow",Float "SPMid",bool "Pord",bool "Show") {
# Selects frames depending upon scene change status. Similar to SCSelect()
# Dclip, detection clip. Start, Frames after scene change. End, Frames before scene change. Motion, not scene change.
# Start, End and Motion must be same size and colorspace, Dclip can be other.
myName="SC_SceneSelect: "
df=Float(Default(dfact,4.0)) #
minim=Float(Default(minim,4.0)) # Scene change Diff has to be at least this, avoids eg scene change in near static scene.
Bias=Float(Default(Bias,0.0)) # Arg to RT_LumaSceneChange(), -511.0 -> 511.0, Default 0.0
Gain=Float(Default(Gain,1.0)) # Arg to RT_LumaSceneChange(), -8.0 -> 8.0, Default 1.0
Cont=Float(Default(Cont,1.0)) # Arg to RT_LumaSceneChange(), -8.0 -> 8.0, Default 1.0
RPow=Float(Default(RPow,1.0)) # Arg to RT_LumaSceneChange(), 0.1 -> 4.0, Default 1.0
SPow=Float(Default(SPow,1.0)) # Arg to RT_LumaSceneChange(), 0.1 -> 4.0, Default 1.0
SPMid=Float(Default(SPMid,0.5)) # Arg to RT_LumaSceneChange(), 0.01 -> 0.99, Default 0.5
Pord=Default(Pord,False) # Arg to RT_LumaSceneChange(), Default False
Show=Default(Show,False) # True = Show a little extra Info (Default False)
Assert(df>1.0,myName+"dfact MUST be greater than 1.0")
Assert(minim>0.0,myName+"minim MUST be greater than 0.0")
dclip=dclip.RoboCrop(WMod=4,HMod=4) # Dont allow common border to 'water down' metrics
Global SCM_B=0.0 Global SCM_C=0.0 Global SCM_D=0.0 Global SCM_E=0.0
Global SCM_SC=False Global TrimN=0 Global SCM_Prev=-2 # Init vars, SCM_Prev=-2 forces initalize
Start =(Show)? Start.SubTitle("START OF SCENE",align=1,size=30) : Start
End =(Show)? End.SubTitle("END OF SCENE",align=3,size=30) : End
Motion=(Show)? Motion.SubTitle("MOTION",align=5,size=30) : Motion
# Make format string only once, not at every frame
Fmt=(Show)? "%d ] Trim=%d {\a%c%.1s\a-:\a%c%.1s\a-:\a%c%.1s\a-} L=%.2f : R=%.2f\n" +
\ "%6.2f \a%c%6.2f %6.2f %6.2f\a- %6.2f\n %6.2f %6.2f %6.2f %6.2f\n \a%c%6.2f %6.2f %6.2f\a-\n" +
\ "DFact=%6.2f : Minim=%6.2f\nBias =%6.2f : Gain =%6.2f : Cont =%6.2f\nRPow =%6.2f : SPow =%6.2f : SPMid=%6.2f : Pord=%.1s":""
Motion.ScriptClip("""
NotNext = (current_frame!=SCM_Prev+1)
# If jumped about, get values for Previous frame, else keep same.
# Below Eval avoids extra calcs when scanning forward but is slower going backwards.
psc=(NotNext) ? Eval("
Global SCM_A=RT_LumaSceneChange(dclip,dclip,n=current_frame-3,n2=current_frame-2,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_B=RT_LumaSceneChange(dclip,dclip,n=current_frame-2,n2=current_frame-1,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_C=RT_LumaSceneChange(dclip,dclip,n=current_frame-1,n2=current_frame ,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_D=RT_LumaSceneChange(dclip,dclip,n=current_frame ,n2=current_frame+1,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_E=RT_LumaSceneChange(dclip,dclip,n=current_frame+1,n2=current_frame+2,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
AB=SCM_B-SCM_A BC=SCM_C-SCM_B CD=SCM_D-SCM_C DE=SCM_E-SCM_D # Diff of Diff
ABC=BC-AB BCD=CD-BC CDE=DE-CD # Diff Of Diff Of Diff
#
# BELOW MUST BE SAME AS MAIN CODE.
# Big enough compared to movement immediately either side of scene change
T1 = minim<SCM_C # Not static scene ?
T2 = ABC>0.0 && BCD<0.0 && CDE>0.0 # Scene Change sign OK ?
T3 =(df*SCM_B<SCM_C && df*SCM_D<SCM_C) # sufficiently bigger than either side ?
# ABOVE MUST BE SAME AS MAIN CODE.
#
return (T1 && T2 && T3)
") : SCM_SC
Global SCM_A=SCM_B Global SCM_B=SCM_C Global SCM_C=SCM_D Global SCM_D=SCM_E # Pass the parcel
Global SCM_E=RT_LumaSceneChange(dclip,dclip,n=current_frame+2,n2=current_frame+3,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
AB=SCM_B-SCM_A BC=SCM_C-SCM_B CD=SCM_D-SCM_C DE=SCM_E-SCM_D # Diff of Diff
ABC=BC-AB BCD=CD-BC CDE=DE-CD # Diff Of Diff Of Diff

# Big enough compared to movement immediately either side of scene change
T1 = minim<SCM_C # Not static scene ?
T2 = ABC>0.0 && BCD<0.0 && CDE>0.0 # Scene Change sign OK ?
T3 =(df*SCM_B<SCM_C && df*SCM_D<SCM_C) # sufficiently bigger than either side ?

psc=(psc||current_frame==0)
Global SCM_SC=((T1 && T2 && T3) || current_frame==FrameCount()-1) # Last frame in scene ?
Last = (psc&&!SCM_SC)?start:(SCM_SC&&!psc)?end:Last
CC=(T1&&T2&&T3)?33:45 # Hilite or Default color ?
LFACT=(SCM_C<=SCM_B || !T1 || !T2)?0.0:(SCM_B<0.000001)?999999.9:(SCM_C-0.000001)/SCM_B
RFACT=(SCM_C<=SCM_D || !T1 || !T2)?0.0:(SCM_D<0.000001)?999999.9:(SCM_C-0.000001)/SCM_D
(Show) ? RT_Subtitle(Fmt,current_frame,TrimN,CC,T1,CC,T2,CC,T3,LFACT,RFACT,SCM_A,CC,SCM_B,SCM_C,SCM_D,SCM_E,AB,BC,CD,DE,CC,ABC,BCD,CDE,
\ df,minim,bias,gain,cont,rpow,spow,spmid,pord) : NOP
Global TrimN = (SCM_SC) ? TrimN+1 : TrimN # Incr for next time
Global SCM_Prev=current_frame
Return Last
""",args="start,end,dclip,myName,Show,df,minim,Bias,Gain,Cont,RPow,SPow,SPMid,Pord,Fmt") # Needs Grunt for args
return Last
}


Client script just to show subtitled results

Import("Dynacrop.avs")

Avisource("D:\avs\test.avi")

DClip=Last
Start=Last
End=Last
Motion=Last

SC_SceneSelect(dclip,start,end,motion,show=true,dfact=4.0,minim=4.0,Bias=0.0,gain=1.0,cont=1.0,rpow=1.0,spow=1.0,SPMid=0.5,pord=false)

[Forensic]

Increasing a video's gamma or histogram profile can brighten a seemingly black nighttime surveillance video. However, with only 256 shades of YUV illumination, the results can fail to be forensically useful. It seems to me that a your RT-Stats tool in RGB space could provide a much better alternative. My idea is to select the upper left most pixel and read its RGB values along with those of the preceding and postceding "N" frames. Average those temporal values and then subtract that average from the current frame's R G and B pixel values. If I now multiply that difference by "M", and add it to the original RGB pixel values, the resulting RGB values will be an exaggeration of the color deviation from average at that one pixel. If I repeat this process for every pixel of every frame, I should end up with greater detail of any changes that are occurring, even if the scene appears almost completely dark. I realize that the resulting scene will look weird, and poison noise will become amplified, but the goal is to detect movement, amplify signage (logos, license plates, etc...), and other forensic stuff.

I tried this myself using "RgbChanMedian" to read each pixel and "Overlay" to write the new pixel data. Not only was the process painfully slow (several minutes per frame) but it also caused out of memory issues due to all the looping required (thus I now have no solution). Although my work is mostly with low quality surveillance video, such a feature would encourage new uses. For example, the exaggerated color shifts would allow anyone to see the normally imperceptible temperature changes as machinery heated up (and thus red incremented by one in RGB space).

EDIT: This comment led to a new thread that contains my original code, and improvements by others, all located HERE (http://forum.doom9.org/showthread.php?t=168293)

[StainlessS]

RT_Stats v1.21 update. See 1st post.

New GUI Filselector functions.


RT_FSelOpen(string "title"="Open",string "dir"="",string "filt",string "fn="",bool "multi"=false,bool "debug"=false)

Function to select EXISTING filename using GUI FileSelector.

Title = Title bar text.
Dir = Directory, "" = Current
Filt = Lots, eg "All Files (*.*)|*.*"
[Displayed text | wildcard] [| more pairs of Displayed text and wildcard, in pairs ONLY].
first one is default.
fn = Initially presented filename (if any).
multi = Multiply Select filenames. Allows selection of more than one filename at once.
debug = Send error info to DebugView window.

Returns
int, 0, user CANCELLED.
int, non zero is error (error sent to DebugView window).
String, Filename selected, Chr(10) separated multiline string if MULTI==true (and multiple files selected).

Example, to prompt for an AVI file and play it.
avi=RT_FSelOpen("I MUST have an AVI",filt="Avi files|*.avi")
Assert(avi.IsString,"RT_FSelOpen: Error="+String(avi))
AviSourcE(avi)

***
***
***

Function RT_FSelSaveAs(string "title"="Open",string "dir"="",string "filt",string "fn="",bool "debug"=false)

Function to select filename for Save using GUI.

Title = Title bar text.
Dir = Directory, "" = Current
Filt = Lots, eg "All Files (*.*)|*.*"
[Displayed text | wildcard] [| more pairs of Displayed text and wildcard, in pairs ONLY].
first one is default.
fn = Initially presented filename (if any).
debug = send errors to DebugView window.

Returns
int, 0, user CANCELLED.
int, non zero is error (error sent to DebugView window).
String, Filename selected.

***
***
***

Function RT_FSelFolder(string "title"="",string "dir"=".",bool "debug"=false)

Function to select Folder using GUI.

Title = UNDERNEATH the title bar text, for instructions.
Dir = Directory, Default "." = Current, ""=Root.
debug = Send errors to DebugView window.

Returns
int, 0, user CANCELLED.
int, non zero is error (ie -1, error sent to DebugView window, usually selecting non Folder object eg 'My Computer').
String, Folder selected (minus trailing BackSlash).



Also one of the added FSel_Demo AVS files, for scrolling text demo (5 modes).

avi = RT_FSelOpen("Please select an AVI file",Filt="*.AVI|*.AVI")
Assert(avi.IsString,"RT_FSelOpen: Error="+String(avi))
AVISource(AVI)
txt = RT_FSelOpen("And now select a Text file",Filt="*.txt|*.txt")
Assert(txt.IsString,"RT_FSelOpen: Error="+String(txt))
Txt=RT_ReadTxtFromFile(txt)
Lines=RT_TxtQueryLines(Txt)
# config
DELAY=100
ALIGN=5 # As Numeric KeyPad
SCROLL=0 # 0 = Upwards : 1 = Downwards : 2 = Right to Left : 3 = Left to right : 4 = Karaoke
#
ORG=Last
Last=(SCROLL==4)? ORG.Blankclip(height=64) : Last
CMD_0 = """RT_Subtitle("%s",Txt,align=ALIGN,y=height+DELAY-current_frame,expx=true,expy=true)"""
CMD_1 = """RT_Subtitle("%s",Txt,align=ALIGN,y=-(Lines*20+DELAY) + current_frame,expx=true,expy=true)"""
CMD_2 = """RT_Subtitle("%s",Txt,align=ALIGN,x=width+DELAY-current_frame,expx=true,vcent=true)"""
CMD_3 = """RT_Subtitle("%s",Txt,align=ALIGN,x=-(width+DELAY)+current_frame,expx=true,vcent=true)"""
CMD_4 = """RT_Subtitle("%s",Txt,align=ALIGN,y=height+DELAY-current_frame,expx=true,expy=true)"""
CMD_5 = """RT_Subtitle("BAD SCROLL COMMAND (0->4)")"""
CMD = (SCROLL<0 || SCROLL>4) ? CMD_5 : Select(Scroll,CMD_0,CMD_1,CMD_2,CMD_3,CMD_4)
ScriptClip(CMD)
Return (SCROLL==4)? StackVertical(ORG,Last) : Last



EDIT: RT_FselOpen() prompts for EXISTING filename, RT_FselSAveAs() will prompt for overwrite if already existing.

[StainlessS]

RT_Stats v1.22, update.

v1.22, Fixed RT_RGBChanXXXX channels. Changed RT_YStdev() to Standard Deviation from Sample Standard Deviation.

See 1st post.

[StainlessS]

RT_Stats v1.23, update. See first post.

v1.23, Mod RT_Debug, added Repeat arg.
Mod RT_YInRangeLocate, added Baffle_W and Baffle_H args.
Added RT_RgbInRange func.
Added RT_RgbInRangeLocate func.


RT_Debug update in Blue:

RT_Debug(string s1, ... , string sn,bool "name"=true,bool "Repeat"=true)
Non-clip function. Takes one or more strings, joins them together and then outputs the combined
string to OutputDebugString(), if strings are NOT space separated, it inserts a single space between them.
Useful during development Compile-time/Run-time scripts to monitor status in real time.
Can also output multiline string obtained via eg RT_ReadTxtFromFile.
Name: bool Default=True. If False, then does NOT output "RT_Debug: " at the beginning of each line, you
might want to replace with your own title in string s1.
Use DebugView utility to view and capture output. http://technet.microsoft.com/en-gb/sysinternals/bb545027
Usage eg:- RT_Debug("["+String(42)+"]","Going pretty good so far @","Line "+String(19))
Would Produce this "RT_Debug: [42] Going pretty good so far @ Line 19"
The above "RT_Debug:" title is present in all RT_Debug() output to the debugview window and you can (in DebugView),
filter out all lines that do not contain "RT_Debug:" so as to skip stuff that is of no interest (eg AVSPMod generates
some output, not sure but I think Avisynth v2.58 also generates lots of debug stuff).
NOTE, Any program can send output to DebugView window, if no debug viewer is present, it normally just disappears
down the plughole.
In DebugView, filter for eg "RT_Debug:" to skip all the extraneous stuff generated by other plugins/programs.
(In DebugView "Menu:Edit-Filter/Highlight", and enter "RT_Debug:".
v1.14, No longer converts characters 127->255 to SPACE, only chars 0->31 (excluding n/l and CR) converted to space.
Characters displayed via debugview are same as those via SubTitle, but those above 127 are dissimilar to the ones
produced via Avisynth source Info.h, used during eg plugin metrics display.
v1.23, Added Repeat arg, only used if Name==False.
If name==False and Repeat==True then outputs first string at beginning of each line, ie shows
your own name on each line where input strings contain Chr(10) line breaks. False output first string on first
line only.


New func RT_RgbInRange

RT_RgbInRange(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false, \
int "RLo"=128,int "RHi"=RLo,int "GLo"=128,int "GHi"=GLo,int "BLo"=128,int "BHi"=BLo)
Returns float value (0.0 -> 1.0) being the amount of pixels with all channels in the range "cLo" to "cHi" (inclusive, c==R or G or B),
1.0 is equivalent to 100%.
NOTE, differs from other funcs that return range 0.0 to 255.0. NOTE, cLo defaults to 128, "cHi" defaults to "cLo".
RLo: Default 128, lower bound of pixel values of Red Channel to search for.
RHi: Default RLo, upper bound of pixel values to Red Channel search for.
GLo: Default 128, lower bound of pixel values of Green Channel to search for.
GHi: Default GLo, upper bound of pixel values to Green Channel search for.
BLo: Default 128, lower bound of pixel values of Blue Channel to search for.
BHi: Default BLo, upper bound of pixel values to Blue Channel search for.


RT_YInRangeLocate, Added Baffle_W, Baffle_H

Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(width<=720?2:3),String "Prefix"="YIRL_",bool "Debug"=false,int "Baffle_W"=Baffle,int "Baffle_H"=Baffle)

Clip function to scan frame n+delta for an area where percentage of pixels in luma range Lo to Hi (inclusive) breaks a threshold over at
least Baffle consecutive number of scanlines (all four sides).
Returns True on Successful location else false. On success sets Global var Coords.

RT_YInRangeLocate() tests scanlines from outer most scanlines (h and v), towards inner most, stops where it find Baffle consecutive scanlines
that contain more than Thresh percent of pixels in range Lo to Hi. Object interiors are not tested and so could be hollow.
Would also give +ve result for a disjoint shape like this
--
| |
--
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold. Default is any pixel in the range Lo to Hi will break threshold.
Lo: Default 128, lower bound of pixel values to search for.
Hi: Default 255, upper bound of pixel values to search for.
Matrix: Default for RGB is 2(PC601) if width <= 720 else 3(PC709) : YUV not used
Prefix: Default "YIRL_", prefix for Global Var coords set on successful location of object, eg YIRL_X, YIRL_Y, YIRL_W, YIRL_H.
Debug: Default false. If true outputs info to debugView.
Baffle_W, Default Baffle. Thickness of Left and Right side vertical edges, silently limited to width W of search area.
Baffle_H, Default Baffle. Thickness of Top and Bottom horizontal edges, silently limited to height H of search area.


RT_RgbInRangeLocate, new function.

Function RT_RgbInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "RLo"=128,int "RHi"=255,int "GLo"=128,int "GHi"=255,int "BLo"=128,int "BHi"=255, \
String "Prefix"="RGBIRL_",bool "Debug"=false,int "Baffle_W"=Baffle,int "Baffle_H"=Baffle)

Clip function to scan frame n+delta for an area where percentage of pixels in all 3 RGB channels range cLo to cHi (inclusive, c==R or G or B)
breaks a threshold over at least Baffle consecutive number of scanlines (all four sides).
Returns True on Successful location else false. On success sets Global var Coords.

RT_RgbInRangeLocate() tests scanlines from outer most scanlines (h and v), towards inner most, stops where it find Baffle consecutive scanlines
that contain more than Thresh percent of pixels in range cLo to cHi. Object interiors are not tested and so could be hollow.
Would also give +ve result for a disjoint shape like this
--
| |
--
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold.
Default is any pixel with all three channels in range cLo to cHi, will break threshold.
RLo: Default 128, lower bound of pixel values of Red Channel to search for.
RHi: Default 255, upper bound of pixel values to Red Channel search for.
GLo: Default 128, lower bound of pixel values of Green Channel to search for.
GHi: Default 255, upper bound of pixel values to Green Channel search for.
BLo: Default 128, lower bound of pixel values of Blue Channel to search for.
BHi: Default 255, upper bound of pixel values to Blue Channel search for.
Prefix: Default "RGBIRL_", prefix for Global Var coords set on successful location of object, eg RGBIRL_X, RGBIRL_Y, RGBIRL_W, RGBIRL_H.
Debug: Default false. If true outputs info to debugView.
Baffle_W, Default Baffle. Thickness of Left and Right side vertical edges, silently limited to width W of search area.
Baffle_H, Default Baffle. Thickness of Top and Bottom horizontal edges, silently limited to height H of search area.


Demo for RT_RgbInRangeLocate()

RED =$AA # Object Color to look for
GREEN =$CC
BLUE =$FF

OBJECT_W = 10 # Object Width
OBJECT_H = 10 # Object Height

AREA_X = 0 # Area of frame to search (0,0,0,0=full frame, As Crop)
AREA_Y = 0
AREA_W = -0
AREA_H = -0

R_MIN = RED R_MAX = RED
G_MIN = GREEN G_MAX = GREEN
B_MIN = BLUE B_MAX = BLUE

# Overlay Converts RGB->YUV->RGB (can set inexact color). Use Layer Instead, MUST set ALPHA
RGBA=((255 * 256 + RED)*256+GREEN)*256+BLUE

COORDINATES="coordinates.txt"

# Fake input clip, for testing # EDIT: Animated fake clip partly liberated from a Cretindesalpes script
a = BlankClip(length=480,width=480,height=480)
b = a.BlankClip(width=OBJECT_W, height=OBJECT_H, color=RGBA)
a.Animate (0, a.FrameCount () - 1, "Layer", b,"add",257, 0, 0, b,"add",257, 479, 479)

NOTFOUND="NOT FOUND"
sep = ", "
Bra = "] "
RT_FileDelete(COORDINATES) # Delete coordinates file
ScriptClip ("""
Bingo=RT_RgbInRangeLocate(x=AREA_X,y=AREA_Y,w=AREA_W,h=AREA_H,
\ RLo=R_MIN,Rhi=R_MAX,GLo=G_MIN,Ghi=G_MAX,BLo=B_MIN,Bhi=B_MAX,Baffle_W=OBJECT_W,Baffle_H=OBJECT_H)
(Bingo) ? WriteFile (COORDINATES,"current_frame", "Bra", "RGBIRL_X", "sep", "RGBIRL_Y")
\ : WriteFile (COORDINATES,"current_frame", "Bra", "NOTFOUND")
""")


EDIT: Note, RT_RgbInRangeLocate() RGB32 and RGB24, however above demo RGB32 Only as Layer only for RGB32.

[StainlessS]

RT_Stats functions current @ v1.23

There follows a list of all function names together with CPP style argument specifiers that inform
Avisynth the argument types and optional names. Optional arguments have square brackets surrounding
their name as in [name] and are followed by a type specifier character that gives the type.
Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args,
followed by optional 'arg1' of type bool and optional 'arg2' of type int.

# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
###################################

RT_ColorSpaceXMod,"c"
RT_ColorSpaceYMod,"c[Laced]b"
RT_Stats,"c"
RT_GraphLink,"cc+b*"
RT_Subtitle, "cs.*[align]i[x]i[y]i[vcent]b[expx]b[expy]b"
// ------
RT_BitNOT,"i"
RT_BitAND,"ii"
RT_BitOR ,"ii"
RT_BitXOR,"ii"
RT_BitASL,"ii"
RT_BitASR,"ii"
RT_BitLSL,"ii"
RT_BitLSR,"ii"
RT_BitTST,"ii"
RT_BitCLR,"ii"
RT_BitSET,"ii"
RT_BitCHG,"ii"
RT_BitROR,"ii"
RT_BitROL,"ii"
RT_BitSetCount,"i"
// ------
RT_Hex,"i[width]i"
RT_HexValue,"s[pos]i"
RT_NumberString,"i[base]i[width]i"
RT_NumberValue,"s[base]i[pos]i"
// ------
RT_Call, "s[Hide]b[Debug]b"
RT_GetLastError, ""
RT_GetLastErrorString, ""
// ------
RT_Debug, "s+[name]b[repeat]b"
// ------
RT_GetProcessName, "[Parent]b[debug]b"
// ------
RT_Undefined,""
RT_VarExist,"s"
RT_Ord,"s[pos]i"
RT_Timer,""
// ------
RT_FileQueryLines,"s"
RT_ReadTxtFromFile,"s[Lines]i[Start]i"
RT_WriteFileList,"ss[append]b"
RT_TxtWriteFile,"ss[append]b"
RT_FileDelete,"s"
//
RT_FSelOpen ,"[title]s[dir]s[filt]s[fn]s[multi]b[debug]b"
RT_FSelSaveAs,"[title]s[dir]s[filt]s[fn]s[debug]b"
RT_FSelFolder,"[title]s[dir]s[debug]b"
// ------
RT_YDifference,"c[n]i[delta]i[x]i[y]i[w]i[h]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaCorrelation,"cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaDifference,"cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaSceneChange,"cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i" \
"[Bias]f[Gain]f[Cont]f[RPow]f[SPow]f[SPMid]f[Pord]b"
RT_LumaMovement,"cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[mode]i[blkw]i[blkh]i" \
"[Bias]f[Gain]f[Cont]f[RPow]f[SPow]f[SPMid]f[Pord]b"
//
RT_YPlaneMin, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i"
RT_YPlaneMax, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i"
RT_YPlaneMinMaxDifference, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i"
RT_YPlaneMedian, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i"
RT_AverageLuma, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i"
RT_YPlaneStdev, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i"
RT_YInRange, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[lo]i[hi]i"
RT_YStats, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[lo]i[hi]i[flgs]i[prefix]s"
///
RT_RgbChanMin, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i"
RT_RgbChanMax, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i"
RT_RgbChanMinMaxDifference, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i"
RT_RgbChanMedian, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i"
RT_RgbChanAve, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i"
RT_RgbChanStdev, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i"
RT_RgbChanInRange, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[lo]i[hi]i"
RT_RgbChanStats, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[lo]i[hi]i[flgs]i[prefix]s"
///
RT_RGB32AsFloat,"c[n]i[delta]i[x]i[y]i"
RT_RGB32AsInt,"c[n]i[delta]i[x]i[y]i"
RT_FloatAsRGBA,"f"
///
RT_YankChain, "c[n]i[delta]i"
// ------
RT_TxtQueryLines,"s"
RT_TxtGetLine,"s[Line]i"
RT_StrAddStr,"ss+"
RT_TxtAddStr,"ss+"
RT_QuoteStr,"s"
RT_StrReplace,"sss[sig]b"
RT_StrReplaceDeep,"sss[sig]b"
RT_StrReplaceMulti,"sss[sig]b"
RT_StrPad,"si[c]s"
RT_FindStr,"ss[sig]b[pos]i"
// ------
RT_TxtSort,"s[mode]i"
// ------
RT_GetWorkingDir,""
RT_FilenameSplit,"s[get]i"
RT_GetFileExtension,"s"

// ------
RT_QueryLumaMinMax,"c[Samples]i[Ignore]f[Prefix]s[Debug]b[X]i[Y]i[W]i[H]i[Matrix]i"
RT_QueryBorderCrop,"c[Samples]i[Thresh]f[Laced]b[xMod]i[yMod]i[wMod]i[hMod]i[Relative]b[Prefix]s[RLBT]i[Debug]b[Ignore]f[Matrix]i" \
"[ScanPerc]f[Baffle]i[ScaleAutoThreshRGB]b[ScaleAutoThreshYUV]b"
RT_YInRangeLocate, "c[n]i[delta]i[x]i[y]i[w]i[h]i[baffle]i[thresh]f[lo]i[hi]i[matrix]i[prefix]s[debug]b[baffle_w]i[baffle_h]i"
// ------
RT_RgbInRange, "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Rlo]i[Rhi]i[Glo]i[Ghi]i[Blo]i[Bhi]i"
RT_RGBInRangeLocate,"c[n]i[delta]i[x]i[y]i[w]i[h]i[baffle]i[thresh]f[Rlo]i[Rhi]i[Glo]i[Ghi]i[Blo]i[Bhi]i[prefix]s[debug]b" \
"[baffle_w]i[baffle_h]i"
// ------
RT_GetSAR, "cf"
RT_GetDAR, "cf"
RT_SignalDAR, "f"
RT_GetCropDAR,"cf[x]f[y]f[w]f[h]f"

[StainlessS]

@Jmac698

If you see this (I saw you scurrying around last night :p ),
try out RgbAmplifier on your Moon2 set (after PlanetCrop,
with/without PlanetCrop luma leveling), and with/without
speedup to eg 7FPS (clearly shows clouds moving across the
moon images). Makes for some interesting shots.

[jmac698]

interesting, will do! Funny, I was just reading about that exact topic at random and thinking about uses for exaggerating differences, and in fact there's a much more sophicated way to do this
http://people.csail.mit.edu/mrub/vidmag/

[Forensic]

Jmac698, don't be hatin on my RgbAmplifier. I invented this in 2010 but could not use it on anything except small short test videos due to out of memory errors. I brought it back to life when RTStats introduced a new tool and, with the talents on StainlessS and Gavino, it became a very useable plug. Eulerian Video Magnification also lets you limit the affected color range but I think it is designed for clear high-res high-fps videos. RgbAmplifier is designed to work on noisy low-res low-fps videos (like surveillance videos). It is proving to be quite useful for my fellow video forensic analysts.

[StainlessS]

Very interesting link JMac.
Limited use though due to no motion requirement, lie detector, baby not breathing alarm, perhaps some kind of strain gauge.
Dont bother trying RgbAmplifier on your original Moon set, looks appalling mulicolored mess with full amp of 10.0.

EDIT: Forensic, I tried RGBAmplifier on the Face.mp4 with radius between about 7 to 25, it does actually reveal the
same facial flush of the vidmag link, it does however reveal compression artifacts and all minute differences, and does not
look anywhere near as good as vidmag. Not really surprising though as vidmag has a different purpose.
I also did a quick hack and tried magnifying difference between current frame and the average of
(current-off)+(current+off) with off of about 13 frames, even that worked quite well but again was quite noisy.

[Forensic]

I believe that they simply came to a similar methodology as RgbAmplifier. I also propose that RgbAmplifier results would match theirs (for color variance detection) in clarity if they had provided the uncompressed version of Face.mp4. By sharing a h.264 version, the file has quantization data errors (the noise) and significant pixel disruptions at he boundaries of the slices (aka, pixel blocks). No way to know for sure without that file.

[StainlessS]

I also did a quick hack (now deleted) where I used ClipBlend(delay=0) [blend ALL frames encountered] and saved last frame.
Called RGBAmp hack version with only args Magnifier and new arg clip c2 (the whole clip blended frame) and amplified diff
between current frame and the single blended frame. Still amp'ed the face flush and also minute differences as face moved
with hair going strangely wiry, and reduced quantization anomalies.
Would have been nice to have original uncompressed source to try for comparison, but I think they must also have some kind of
temporal smoothing with frames close to current and perhaps at 'beat' distance from current.
I did not yet read their source, think I'll stop with those experiments.

[StainlessS]

RT_Stats v1.24, new version, see 1st post.

v1.24, Fixed Interlaced Y stepping.
Added Mask args to Y/RGB stats, Added RT_YPNorm, RT_RgbChanPNorm (*** EXPERIMENTAL ***)
Added RT_TxtFindStr and RT_FileFindStr and RT_String.

New Functions added.


v1.24, New.
RT_TxtFindStr(String S,string FndS,bool "Sig"=True,int "Pos"=1,int "Start"=0,int "Lines"=0)
Finds unnamed string FndS in unnamed multi-line text string S, starting at line 'Start' and searching 'Lines' number of string lines.
S: is a multi-line text string, ie a newline [CHR(10)] separated list of strings.
FndS: is a single string and will be considered terminated early at the first carriage return [Chr(13)] or newline [Chr(10)],
carriage returns and newlines will not be findable using this function.
Sig: Default=True, does case significant comparison, insignificant if false.
Pos: Default=1, is the character start position in an individual line of the multi-line string S to start searching
for the FndS string. Allows you to skip search on first Pos-1 characters within all single lines of multi-line string S.
If Pos less than 1, then will return -ve number (usually -1), Not Found.
Start: Default 0, is the 0 relative starting line where the searching begins within the multi-line string S.
Lines: Default 0(all lines). Number of lines in multi-line string to seach, starting at line 'Start'.
A -ve return is 'Not Found'. On success returns the line number of the first instance of a found string in the Start and Lines
range of lines. An empty "" FndS will always return -ve result, ie Not Found (as will all errors).
see RT_TxtQueryLines, to inquire number of single lines in a chr(10) separated multi-line string.
see RT_TxtGetLine to extract a single line from a multi-line string [without trailing Chr(10) or Chr(13)].
NOTE, Character position 'Pos' is 1 relative whereas multi-line line 'Start' index is 0 relative.



v1.24, New
RT_String(String format, dat1,...,datn,int "Esc"=1)

Returns a formatted string. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the text string that is
returned, uses C/CPP printf() style formatting.
Format: compulsory string controlling format and describing the datn type args that are expected.
datn: Variable number of data args of any type (excluding clip).
Esc: Default 1, converts embedded escape sequences in format string, 2 convert escape sequences in both format and datn strings,
0 no escape sequence conversion done.
It is necessary to specify the "Esc" name if you wish to alter default, as Avisynth cannot tell when the variable number of data datn
args ends.
You might wish to turn off escape conversion if you want to send a multi-line string to Subtitle() without conversion of '\n' to Chr(10).

printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned (o=octal, x=Hex).
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

Formatting Insertion point is marked with '%' character (as in Avisynth String function), if you wish to use a percent
character within the returned string, it should be inserted twice ie '%%' will produce a single '%'.
EDIT: '%' in data datn arg strings do not need double % character.

A Backslash character '\' introduces escape sequences, to insert a backslash character itself, you must supply a double
backslash sequence ie '\\'.
When 'Esc' is non zero, converts embedded escape character sequences (Case Significant):-
'\\' Converted to '\' Single Backslash
'\n' Converted to Chr(10) NewLine
'\r' Converted to Chr(13) Carriage Return
'\t' Converted to Chr(9) Horizontal TAB
'\v' Converted to Chr(11) Vertical TAB
'\f' Converted to Chr(12) FormFeed
'\b' Converted to Chr(8) BackSpace
'\a' Converted to Chr(7) Bell
'\x', where x is ANY OTHER CHARACTER not included above, will be copied verbatim, ie '\x'.

eg
RT_String("Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013)
would return same as:- "Hello there Fred and Ted." + Chr(10) + "Goodbye 2013."

Be aware that when 'Esc'=2, the string is constructed via two passes, the 1st pass inserts the data args into the format string,
2nd pass converts escape sequences, so the escape sequences could also exist in the datn args. Also, beware of constructing strings
which when joined together, form an escape sequence.
Take care when handling filenames with path backslash, suggest using 'Esc'=1 when data datn strings contain eg Filenames.



v1.24, New.
RT_FileFindStr(String FileName,string FndS,bool "Sig"=True,int "Pos"=1,int "Start"=0,int "Lines"=0)
Finds unnamed string 'FndS' in 'FileName' text file, starting at line 'Start' and searching 'Lines' number of text lines.
FileName: Name of a file containing text to search.
FndS: is a string and will be considered terminated early at the first carriage return [Chr(13)] or newline [Chr(10)], carriage returns
and newlines will not be findable using this function.
Sig: Default=True, does case significant comparison, insignificant if false.
Pos: Default=1, is the character start position in an individual line of text to start searching for the FndS string.
Allows you to skip search on first Pos-1 characters within all lines of text.
If Pos less than 1, then will return -ve number (usually -1), Not Found.
Start: Default 0, is the 0 relative starting line where the searching begins in the text file.
Lines: Default 0(all lines). Number of lines text to seach, starting at 'Start' line.
A -ve return is 'Not Found'. On success returns the line number of the first instance of a found string in the Start and Lines
range of lines. An empty "" FndS will always return -ve result, ie Not Found.
see RT_FileQueryLines, to inquire number lines in a text file.
see RT_ReadTxtFromFile to extract a range of lines from a text file.
NOTE, Character position 'Pos' is 1 relative whereas multi-line line 'Start' index is 0 relative.




RT_YPNorm(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1,int "Matrix"=(width<=720?2:3),clip "mask"=NOT_USED,int "MaskMin"=128,"MaskMax"=255)
Return int -1 if no valid pixels in Mask clip.
Returns FLOAT value greater or equal to 0.0, being the "Minkowski P-norm" (range depends upon values of 'd' and 'u') for frame(n+delta)
area x,y,w,h, and selected Chan channel (0=R,1=G,2=B, RGB32 3=ALPHA)
mu, Float, default 0.0 (0.0 -> 255.0)
d, int, default 1 (1 -> 255) # downscale
p, int, default 1 (1 -> 16) # power
u, int, default 1 (1 -> 255) # final upscale before returning result (experimental)
Formula is: sum_over_pixels[ ((pixel-mu)/d)^p ]^(1/p) * u
or in words: d and u are scaling aids. The differences between the pixel values and mu are scaled, taken to the power of p and added up
over the frame. The sum is taken to the p-th root and finally rescaled.
mu=0, d=1, p=1, u=1 yields the average.
mu=average, d=1, p=2, u=1 yields the standard deviation (uncorrected sample standard deviation).
Implemented as requested by Martin53 (thankyou). *** EXPERIMENTAL ***
http://en.wikipedia.org/wiki/P-norm#The_p-norm_in_finite_dimensions
http://en.wikipedia.org/wiki/Minkowski_distance



RT_RgbChanPNorm(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
int "chan"=0,float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1,int "mask"=NOT_USED, int "MaskMin"=128,"MaskMax"=255)
Return int -1 if no valid pixels in Mask clip.
Returns FLOAT value greater or equal to 0.0, being the "Minkowski P-norm" (range depends upon values of 'd' and 'u') for frame(n+delta)
area x,y,w,h, and selected Chan channel (0=R,1=G,2=B, RGB32 3=ALPHA)
mu, Float, default 0.0 (0.0 -> 255.0)
d, int, default 1 (1 -> 255) # downscale
p, int, default 1 (1 -> 16) # power
u, int, default 1 (1 -> 255) # final upscale before returning result (experimental)
Formula is: sum_over_pixels[ ((pixel-mu)/d)^p ]^(1/p) * u
or in words: d and u are scaling aids. The differences between the pixel values and mu are scaled, taken to the power of p and added up
over the frame. The sum is taken to the p-th root and finally rescaled.
mu=0, d=1, p=1, u=1 yields the average.
mu=average, d=1, p=2, u=1 yields the standard deviation (uncorrected sample standard deviation).
Implemented as requested by Martin53 (thankyou). *** EXPERIMENTAL ***
http://en.wikipedia.org/wiki/P-norm#The_p-norm_in_finite_dimensions
http://en.wikipedia.org/wiki/Minkowski_distance


NOT NEW but extended for mask, similar changes for RT_YStats, doc not shown.
EDIT: INSERTED THIS CODE BLOCK

********************************************
********* RGB MASKED Channel FUNCTIONS *****
********************************************

RGB32, RGB24 source clip.

The source clip c must be RGB32 or RGB24.

The compiletime/runtime clip functions share some common characteristics.
The 'n' and (where used) 'n2' args are the optional frame numbers and default to 'current_frame' if not specified.
The x,y,w,h, coords specify the source rectangle under scrutiny and are specified as for Crop(), the default 0,0,0,0 is full frame.
If 'interlaced' is true, then every other line is ommited from the scan, so if eg x=0,y=1, then scanlines 1,3,5,7 etc are scanned,
if eg x=0,y=4 then scanlines 4,6,8,10 etc are scanned. The 'h' coord specifies the full height scan ie same whether interlaced is true
or false, although it will not matter if the 'h' coord is specified as eg odd or even, internally the height 'h' is reduced by 1 when
interlaced=true and 'h' is even.

Optional mask clip Planar ONLY [v2.6 colorpspaces OK]
The Planar mask clip, governs which pixels are processed by the functions. Where a luma pixel in selected area of the the mask clip is
in range "MaskMin" to "MaskMax" inclusive, then those pixels will be processed.
Mask MUST, be same dimensions and have at least the same number of frames as clip c.
Calling without mask OR with MaskMin=0,MaskMax=255 will effectively ignore the mask and scan full x,y,w,h area.

The Chan arg specifies which R or G or B channel to process [the multi-functional RT_RgbChanStats() function also allows a Chan arg of -1,
which processes R and G and B simulaneously for all functions selected by flgs arg, also allowed is chan arg of -2 which additionally
processes the ALPHA channel if RGB32 but throws an error if RGB24].
Default Chan is 0 (Red), 1 = Green, 2=Blue channel and 3=ALPHA channel when RGB32 ONLY else error.




RT_RgbChanStats(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0,int "lo"=128,int "hi"=lo,int "flgs"=255,string "prefix"="RCS_",
float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1,int "mask"=NOT_USED, int "MaskMin"=128,"MaskMax"=255)
Returns multiple results as for above single frame RGB channel sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", are as for all other clip functions, "threshold" used only for "RT_RgbChanMin", "RT_RgbChanMax" and
"RT_RgbChanMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_RgbChanInRange" equivalent routine with same functionality.
"mu" and "d" and "p" and "u" are used only with the "RT_RgbChanPNorm" equivalent routine with same functionality.
The new arg "Flgs" selects which results you want returned and the string "Prefix" that is prepended
to the returned global variable names.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.
Returns 0 if no pixels found in search area of mask within MaskMin and MaskMax.

Flgs_Bit_Number Add_To_Flgs Equivalent_Function Global_Var_Set_Excluding_Prefix_and_Chan_postfix
0 1($01) RT_RgbChanMin() "Min" (0->255)
1 2($02) RT_RgbChanMax() "Max" (0->255)
2 4($04) RT_RgbChanMinMaxDifference() "MinMaxDiff" (0->255)
3 8($08) RT_RgbChanMedian() "Med" (0->255)
4 16($10) RT_RgbChanAve() "Ave" (0.0->255.0)
5 32($20) RT_RgbChanStdev() "Stdev" (0.0->255.0)
6 64($40) RT_RgbChanInRange() "InRng" (0.0->1.0)
7 128($80) RT_RgbChanPNorm() "PNorm" (0.0->??? depends upon d and u)
The Channel Postfix is of the form "_0", where 0 is RED, 1 is GREEN and 2 is Blue, 3 is ALPHA(RGB32 ONLY), and is appended to the
base name described above. So eg RT_RgbChanMin for RED channel 0 with default Prefix is "RCS_Min_0".
RT_RgbChanstats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=255($FF) are all bits set and so sets ALL global vars at once.
RT_RgbChanStats(flgs=1+2+16) would set global vars "RCS_Min_0", "RCS_Max_0" and "RCS_Ave_0" for full frame current_frame, Red Channel.

In addition to above Global Variables, RT_RgbChanStats() sets an int Global variable (where default prefix) of "RCS_PixelCount_0" being
the number of Red Channel pixels in mask area X,Y,W,H between MaskMin and MaskMax inclusive.

NOTE, RT_RgbChanStats() allows Chan to be -1, where ALL three R, and G, and B channels are processed simultaneouly for ALL functions
selected by flgs arg (ALPHA Channel is NOT processed).
A chan arg of -2 (RGB32 ONLY allowed) will additionally process the ALPHA channel as well as R+G+B.
Also Note, (where default Prefix) RCS_PixelCount_x is also set (identically) for all channels when Chan == -1 or -2.

[StainlessS]

Short lived v1.24 posted above, have decided to change RT_String() Esc arg to int, 0 = no escape conversion, 1 = conversion only in format string,
2 = convert escape sequences in both format string and inserted data strings, default 1.

Will update zip on mediafire soon. (dont bother downloading v1.24).

[StainlessS]

Short lived v1.24 updated to v1.25, Changed RT_String Esc arg to type int. See 1st post.

2nd Code block two posts previous updated.

EDIT: This may or may not be my last ever post on the D9.

EDIT:
Life intercedes and well, cannot buck life. If I should not resurect, then would be most happy
if others would take it upon themselves to continue the daft stuff done already.
Peace Bro's.

EDIT: Ok, hence forth, just watching, not particating in general everyday stuff, just not making so much noise, thats all.

[raffriff42]

Thanks for everything, StainlessS, wherever you are :thanks: :thanks: :thanks: :thanks:

[martin53]

StainlessS, can you please help,
RT_RgbChanStats(cRGB, mask=c24, Chan=-1, threshold=MINMAX_THRESH, flgs=$90, prefix="fRGB2_", MaskMin=max(fY_yMax-ydif,1), MaskMax=ymax, d=255, u=255, p=power)
fNorm = 1.0/(KR*fRGB2_PNorm_0 + KG*fRGB2_PNorm_1 + KB*fRGB2_PNorm_2)

throws the error (DebugView, through try... catch... function)
[7540] RT_Debug: [...] I don't know what "fRGB2_PNorm_0" means

chan=0 does not help. RT_Stats() is "v1.25 - 28 Sept 2013"

MYStats.dll (V1.06),

MRGBChanStats(cRGB, c24, Chan=-1, threshold=MINMAX_THRESH, flgs=$90, prefix="fRGB2_", MaskMin=max(fY_yMax-ydif,1), MaskMax=ymax, d=255, u=255, p=power)
fNorm = 1.0/(KR*fRGB2_PNorm_0 + KG*fRGB2_PNorm_1 + KB*fRGB2_PNorm_2)

works fine.

[StainlessS]

M53, shall sort it out whatever it is.
Sorry for the problem.

[StainlessS]

RT_Stats v1.26, see 1st post.

[martin53]

StainlessS,
:thanks: :)

As I learned from the AWB thread, the following is a designed feature:
With the example script,
c = ColorBars(pixel_type="YV24")
cRGB = c.ConvertToRGB(matrix="PC.601")
c.GScriptClip("GScript("+chr(34)+chr(34)+chr(34)+""" # "
Try {
c = last
for (i=16, 235) {
RT_RgbChanStats(cRGB, mask=c, Chan=-1, threshold=0, flgs=$10, prefix="fRGB_i_", MaskMin=i, MaskMax=i)
RT_Debug(RT_String("i %d %d, R %f, G %f, B %f", i, fRGB_i_Pixelcount_0, fRGB_i_Ave_0, fRGB_i_Ave_1, fRGB_i_Ave_2))
}
c
} catch(err) {
RT_Debug("GScriptClip ", err)
return last
}
"""+chr(34)+chr(34)+chr(34)+")", after_frame=true, args="cRGB") #"

DebugView shows, that the three variables stay unchanged for the many brightness values not contained in the mask.
When the starting value i is reduced to 15, the error occurs:
[5672] RT_Debug: GScriptClip I don't know what "fRGB_i_Ave_0" means

To avoid script errors, the user can put a global fRGB_i_Ave_0 = 0.0 etc. before the RT_RgbChanStats() call. But it must be a global var, otherwise the thereby created local var will occlude the global var, which is a strange effect when not understood.

[StainlessS]

From this thread:http://forum.doom9.org/showthread.php?p=1648164#post1648164
OK, fine with everything,
was confused a bit with the handling of variables and posted a question in the RT_Stats thread.
When existing variables are unaltered in the case of pixelcount==0, I need to set them to a neutral value in advance. Say I use chan=-1 then I can not know in advance whether all three channels will return data - although I'd think if one channel does, all channels should.
My observation was that when I initialize variables before I call RT_RgbChanStats, the function does not change them.
I observed some non-credibly low averages with real world footage, but was retarded with my tests by the variables thing.
Maybe I confused the system with local over global variables. Will check that.
However, the whole thing is a bit tricky to master, I think, for newbies, and maybe it should be redesigned a bit.

OK, I should labour the Global Var point in docs and note the specific behaviour.
In the case of chan == -1, all three pixelcounts are zeroed when no valid pixels found in mask, in the case of individual
channel number, only the relevant channel pixelcount is zeroed. On any call, all relevant pixel counts would be identical,
multiple pixelcounts (1 per channel) were deemed better so as to avoid possibility of user having to temporary store eg
red pixelcount when calling for eg Blue channel using different mask.
Myself I would take an Else code path when RT_RgbChanStats returns 0, but I could set pertinent Global vars to -1
if that is your requirement. I would prefer to keep as Global (and in this case cannot change existing behaviour to local),
and yes it can be tricky and awkward having to use 'Global var = whatever'.
You want ALL relevant Global vars to be set to -1 (int) when no valid pixels found in mask ? (excluding pixelcounts).

EDIT: If using an else code path, then would not require try/catch in above previous post.
Result of RT_RgbChanStats/MRgbChanStats are bit flags of which variables were set, 0 == nothing was set.
The only time you can ignore resulting bit flags is when calling function maskless where all pixels in selected area are valid.
I think it might be better to keep as it is without setting 'unset' variables to -1, and use the result bit flags to see if
variables are valid, it might also perhaps help locate coding errors so long as you DONT wrap in try/catch, whereas
setting 'unset' vars to -1 might tend to conceal coding errors.

c = ColorBars(pixel_type="YV24")
cRGB = c.ConvertToRGB(matrix="PC.601")
c.GScriptClip("GScript("+chr(34)+chr(34)+chr(34)+""" # "
c = last
for (i=16, 235) {
Bingo = RT_RgbChanStats(cRGB, mask=c, Chan=-1, threshold=0, flgs=$10, prefix="fRGB_i_", MaskMin=i, MaskMax=i)
if(Bingo!=0) { # EDIT: Could also use eg (fRGB_i_Pixelcount_0 != 0) instead of Bingo
RT_Debug(RT_String("i %d %d, R %f, G %f, B %f", i, fRGB_i_Pixelcount_0, fRGB_i_Ave_0, fRGB_i_Ave_1, fRGB_i_Ave_2))
} Else {
RT_Debug(RT_String("i %d No Valid Pixels",i))
}
c
"""+chr(34)+chr(34)+chr(34)+")", after_frame=true, args="cRGB") #"


1st time I've seen usage of RT_String by someone else, much nicer and less tedious than lots of String() concatenation,
especially with String(x,"%6.2f") style fomatting.
Think I might do a RT_DebugF() type func to avoid having to call RT_Debug with temporary RT_String().

PS, Love your

"GScript("+chr(34)+chr(34)+chr(34)+"""
...
"""+chr(34)+chr(34)+chr(34)+")"

did you devise it yourself ?

EDIT: Is this good enough for your high standards ?

RT_RgbChanStats(clip c,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0,int "lo"=128,int "hi"=lo,int "flgs"=255,string "prefix"="RCS_",
float "mu"=0.0,int "d"=1,int "p"=1,int "u"=1,int "mask"=NOT_USED, int "MaskMin"=128,"MaskMax"=255)
Returns multiple results as for above single frame RGB channel sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", are as for all other clip functions, "threshold" used only for "RT_RgbChanMin", "RT_RgbChanMax" and
"RT_RgbChanMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_RgbChanInRange" equivalent routine with same functionality.
"mu" and "d" and "p" and "u" are used only with the "RT_RgbChanPNorm" equivalent routine with same functionality.
The new arg "Flgs" selects which results you want returned and the string "Prefix" that is prepended
to the returned global variable names.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.
Returns 0 if no pixels found in search area of mask within MaskMin and MaskMax.

Flgs_Bit_Number Add_To_Flgs Equivalent_Function Global_Var_Set_Excluding_Prefix_and_Chan_postfix
0 1($01) RT_RgbChanMin() "Min" (0->255)
1 2($02) RT_RgbChanMax() "Max" (0->255)
2 4($04) RT_RgbChanMinMaxDifference() "MinMaxDiff" (0->255)
3 8($08) RT_RgbChanMedian() "Med" (0->255)
4 16($10) RT_RgbChanAve() "Ave" (0.0->255.0)
5 32($20) RT_RgbChanStdev() "Stdev" (0.0->255.0)
6 64($40) RT_RgbChanInRange() "InRng" (0.0->1.0)
7 128($80) RT_RgbChanPNorm() "PNorm" (0.0->??? depends upon d and u)
The Channel Postfix is of the form "_0", where 0 is RED, 1 is GREEN and 2 is Blue, 3 is ALPHA(RGB32 ONLY), and is appended to the
base name described above. So eg RT_RgbChanMin for RED channel 0 with default Prefix is "RCS_Min_0".
RT_RgbChanstats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=255($FF) are all bits set and so sets ALL global vars at once.
RT_RgbChanStats(flgs=1+2+16) would set global vars "RCS_Min_0", "RCS_Max_0" and "RCS_Ave_0" for full frame current_frame, Red Channel.

In addition to above Global Variables, RT_RgbChanStats() sets an int Global variable (where default prefix) of "RCS_PixelCount_0" being
the number of Red Channel pixels in mask area X,Y,W,H between MaskMin and MaskMax inclusive.

NOTE, RT_RgbChanStats() allows Chan to be -1, where ALL three R, and G, and B channels are processed simultaneouly for ALL functions
selected by flgs arg (ALPHA Channel is NOT processed).
A chan arg of -2 (RGB32 ONLY allowed) will additionally process the ALPHA channel as well as R+G+B.
Also Note, (where default Prefix) RCS_PixelCount_x is also set (identically) for all channels when Chan == -1 or -2.
NOTE, If no valid flg bits set (eg $FF00), then returns 0, RCS_PixelCount_x and all other variables remain as before call.
Assuming some valid flg bits, if no valid pixels were found in mask then function returns 0, and only RCS_PixelCount_x would be set
to 0, no other variables are touched (remain as before call, undefined if not previously existing).
ALSO NOTE, variables are GLOBAL vars, if you set similarly named variables prior to call but without using 'Global' eg
RCS_PixelCount_0 = -1, then this local variable name will hide the Global variable of the same name,
it should be eg Global RCS_PixelCount_0 = -1. See Global Variables here:- http://avisynth.nl/index.php/Script_variables
Example usage:
Scriptclip("""
got = RT_RgbChanStats(c,mask=Mask,chan=-1,flgs=$10) # Ave
(got != 0)
\ ? RT_Debug(RT_String("AveR = %f AveG = %f AveB = %f Pixels = %d",RCS_Ave_0,RCS_Ave_1,RCS_Ave_2,RCS_PixelCount_0))
\ : RT_Debug("NO VALID PIXELS FOUND")
""")

[martin53]

I would vote for global vars always written and even 0 instead of -1 because that is widely known init behaviour, but can live with all kinds. The user can store previous results somewhere else, I see no good reason for keeping the previous result when it does not fit to the latest parameters anyway. Is there a time save advantage?
Have some concern, that the nonexistent/global/local context might be too much for sporadic/new scripters, too. And it might throw unexpected errors as well with published scripts, I feel. Requires a lot of thought to make scripts work under any thinkable circumstances (footage).

(Bingo!=0) is tidier than (Prefix_Pixelcount_Channel), will adopt that.

The skeleton
GScriptClip("GScript("+chr(34)+chr(34)+chr(34)+""" # "
Try {
c = last
...
} catch(err) {
RT_Debug("GScriptClip ", err)
return last
}
"""+chr(34)+chr(34)+chr(34)+")", args="...") #"
is just my standard template for all runtime environments, the Try...catch had nothing to do with the RT_RgbChan function, just meant to give a useful error message in DebugView in case of script error.
It is a bit harder to locate error line numbers with this skeleton, but else much more convenient because GScript is always at hand. Fiddled out the chr(34)+... with the FSubstitute() script (which btw is also unfinished work because the SVPflow adaption does not yet yield good effect; there even were times when I wondered if I should ask Gavino to combine the two brilliant and helpful GScript and GRunt plugins). I don't like it too much. It came as an act of desperation when I was unable to get TeX-style quotes working. The V2.60 manual says: "The manual used to mention TeX-style quotes, but it has been confirmed that AviSynth doesn't work this way since v1.03" and that's true, so so true, ...
The Try...catch construct came when AWB just returned the unaltered input clip without showing any error - because the error dis not occur in the visible clip processing, but in a helper clip processing. I'm talking of the kind of error messages that do not crash te clip but just show up like a subtitle.
RT_String - fully agree! It's worth reading the RT_Stats doc ;) I can only encourage you with the RT_DebugF() function.

Is it a concept principle of AviSynth, that variables by reference are impossible? Else that would be certainly a preferable way to return more than 1 scalar base type. Although I can propose an alternative: The RT_RgbChanStats and other similar functions could return a n by 1 pixel RGBA clip, and with the help of the weird function RT_RGB32AsFloat(), n real-type vars can be extracted. I mention this because the time save option requires global vars, and global vars mean that only one instance of AWB can be used in a script - except with big effort: transfer the prefixes as string parameters to the instances instead of using a hard coded string. This makes scripts hard (impossible) which e.g. use two different AWB parameter sets to display results side by side and compare.

(Btw hope my english is not impolite or too buggy. You learn some vocabulary and grammar in school, but not to express yourself - please no flattery in response)

EDIT yes, it's good - for those who read the manual ;)

[StainlessS]

Is it a concept principle of AviSynth, that variables by reference are impossible?

Think so yes, guess big G could answer that with authority.

OK, you convinced me about changing to local vars with the 'two different AWB parameter sets' argument, have not seen any
scripts that rely on Global var status.
However, am most reluctant to set invalid vars to anything that could be valid results, better to know if there are problems in a script
than to hide them. For the same reason I never eg correct invalid coordinates, that kind of thing can hide coding errors for decades.
The only thing worse than that is eg string buffer allocation, forgetting to add 1 to strlen() in C, which due to optimizations in the
Standard C Library may also escape notice for decades until an unusual condition occurs. The Standard C library will return the next
higher multiple of 8 bytes (32 bit m/c) so forgetting to add 1 to short strings will hide the error, will only be revealed when malloc of
exact multiple of 8 is requested and when multiple of 8 + 1 is actually required. An example might be a 'Title' string in a database,
'Mr', 'Mrs', 'Doctor' will escape notice for perhaps decades, and then one day you enter 'Princess' and bang, corruption creeping
through the database until noticed, and finding the bug will be a horrendous experience.

It will also be a little slower if unnecessary setting of invalid vars, you can always preset the vars to whatever value you like
before the call (and it will not be my fault if you use those invalid values).

Shall do mod to local vars now and do the RT_DebugF() thing too.

[StainlessS]

New Version RT_Stats v1.27,see 1st post.

v1.27, Added RT_FunctionExist, RT_DebugF. Changed all returned Global vars to Local vars except for RT_SignalDAR (Sets default DAR for MeGUI).

New Funcs

RT_FunctionExist(string)
Given the name (string) of the Function that you want to test for existence, returns true if exists, else false.



RT_DebugF(string format,dat1,...,datn,string "name"="RT_DebugF:",int "TabSz"=4)
Sends a formatted string to DebugView Window. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the
text string that is sent to DebugView, uses C/CPP printf() style formatting.
Format: compulsory string controlling format and describing the datn type args that are expected.
datn: Variable number of data args of any type (excluding clip).
Name, Default "RT_DebugF:". Allows you to change the default name prepended to each line of text output to DebugView Window.
TabSz, Default 4 (1 ->32). TAB step setting used for Chr(9) tab ('\t') character.
To set either Name or TabSz, you must use the named form eg Name="NewName:" or TabSz=8 as Avisynth does not know where the data args
end.

printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned (o=octal, x=Hex).
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

Formatting Insertion point is marked with '%' character in the format string (as in Avisynth String function), if you wish to use
a percent character within the output string, it should be inserted twice in the format string ie '%%' will produce a single '%'.
The data datn arg strings do not require a double '%' character.

A Backslash character '\' introduces escape sequences, to insert a backslash character itself, you must supply a double
backslash sequence ie '\\'.
Converts embedded escape character sequences (Case Significant):-
'\\' Converted to '\' Single Backslash
'\n' Converted to Chr(10) NewLine
'\r' [and Chr(13)] Converted to Chr(10) NewLine
'\t' [and Chr(9)] Converted to multiple SPACE's. Horizontal TAB (Tab positions relative to name + SPACE [if not ending in SPACE])
'\x', where x is ANY OTHER CHARACTER not included above, will be copied verbatim, ie '\x'.

Escape sequences are replaced only when occurring in the format string, if you need escapes in data string then use RT_String and
RT_Debug functions.

Example:
RT_DebugF("Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013)
would output same as:- RT_Debug("Hello there Fred and Ted." + Chr(10) + "Goodbye 2013.")

[Forensic]

RT_FunctionExist is awesome. Try/Catch was too generic and could not discern between a missing function or a failed function. Thank you.

[StainlessS]

Glad you like it Forensic, have to say though that it was a doddle to implement, at most 2 minutes.
(EDIT: Already implemented in Avisynth interface and available to plugin writers, so just needed supply name string and return result)

[StainlessS]

Following on from this OT post: http://forum.doom9.org/showthread.php?p=1650239#post1650239

I shall start to implement suggested RT_Array style funcs, shall try to post today as I am busy for next few days.
It is not much harder to implement fixed size strings, so long as the array type is fixed on allocation. Will also implement for bool.
Perhaps at some future point it could be implemented for BYTE sized int, ie binary Array.

EDIT: Shall flag it as Beta version, to give others time to review implementation .

[martin53]

StainlessS,
I look forward to use 1 dimensional arrays with clip length bounds mostly. For example, would be time saving to fill it with scene change information at script start, or similar frame related information that costs time to retrieve again and again.

Deshakers and I don't know what functions could then be written based on the data in the array instead as low pass filters for the difference between current and previous frame - and all the while spoiling the non linear concept that way.

You wrote that you are going to store the data in files. I assume the plugin will automatically create and destroy them, and to the possible extent, care that they are in the HD cache for fast access...

[Gavino]

I shall start to implement suggested RT_Array style funcs, shall try to post today as I am busy for next few days.
How would these compare with AviSynthLib?

[StainlessS]

How would these compare with AviSynthLib?

Dont know, have got that lib but cant recall if I've ever looked at it. Why not judge for yourself.

RT_stats v1.28 Beta, here: LINK REMOVED

New funcs

Added to v1.28 Beta

***
***
***

RT_IncrGlobal(string GlobalName,int "Init"=1)
Given the name of a Global Int variable, will increment that Global (ie add 1 to it) and return the result.
If Global int does not exist, will initialize it to the optional Init arg, default 1.
So if GlobalInt is already initialized to 1, then RT_IncrGlobal("GlobalInt") sets GlobalInt to 2
(equiv:- Global GlobalInt = GlobalInt + 1).
If GlobalName exists in local scope as a local variable, then it will set the same named Global Variable
to the value of the local variable + 1, local variables 'hide' similarly named Global variables, and a plugin
cannot tell the difference between local and global variables, so make sure you do not use local variables
of similar names to global ones.
Will throw an error if GlobalName exists but variable is not of type Int.


***
***
***

RT_ArrayAlloc(string FileName,int "Type"=1,int "Nels"=0,int "StringLenMax"=256)
Creates a file to hold Array elements of type Bool, Int, Float or String.
Filename, Array Filename, no default.
Type, int, default 1(0 -> 3). 0=Bool, 1=Int, 2 = Float, 3 = String.
Nels, default 0. Number of array elements, initialized to zeros.
StringLenMax, default 256(greater than zero). Fixed Maximum string length excluding nul term. Error if string length exceeds.
Only used for Type=3 (string).
Returns number of elements in array. All errors produce an error alert.
***
***
***

RT_ArrayGetType(string FileName)
Filename, Array Filename, no default.
Returns int, Type as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNels(string FileName)
Filename, Array Filename, no default.
Returns int, Nels as set by RT_ArrayAlloc, or current Nels if RT_ArrayAppend used to add vars to end of array.

***
***
***

RT_ArrayGetElSize(string FileName)
Filename, Array Filename, no default.
Returns int, element size, Type 0(bool)=1, 1(int)=4, 2(float)=4, 3(string)=StringLenMax as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNelMax(string FileName)
Filename, Array Filename, no default.
Returns int, maximum number of elements that could theoretically be stored in array, based on (maximum +ve integer - header size) / ElSize.

***
***
***

RT_ArrayGet(string FileName,int Nel)
Filename, Array Filename, no default.
Nel, int, zero relative. Valid range 0 -> RT_ArrayGetNels(FileName) - 1.
Returns Array(Nel) of variable Type as set in RT_ArrayAlloc.

***
***
***

RT_ArraySet(string FileName,int Nel,Var)
Filename, Array Filename, no default.
Nel, int, zero relative. Valid range 0 -> RT_ArrayGetNels(FileName) - 1.
Var, a variable of Type as set in RT_ArrayAlloc.
Returns number of elements in array.

***
***
***

RT_ArrayAppend(string FileName,Var)
Adds a variable to end of array and increments the Nel count.
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc.
Returns number of elements in array.


RT_Array_Test.Avs

/*
RT_ArrayAlloc(string FileName,int "Type"=1,int "Nels"=0,int "StringLenMax"=256)
RT_ArrayGetType(string FileName)
RT_ArrayGetNels(string FileName)
RT_ArrayGetElSize(string FileName)
RT_ArrayGetNelMax(string FileName)
RT_ArrayGet(string FileName,int nel)
RT_ArraySet(string FileName,int nel,var)
RT_ArrayAppend(string FileName,var)
*/

ARRAY="TestArray"
TYPE = 3 # 0=Bool,1=Int,2=Float,3=String
NELS = 128 # Inital Number of Array elements (init'ed to zero's)
STRINGLENMAX = 16 # Excluding Nul Term

ALPHA = "ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ"
ALPHA=ALPHA+ALPHA # Len 260

GScript("""
RT_DebugF("")
RT_ArrayAlloc(ARRAY,Type=TYPE,Nels=NELS,stringlenmax=STRINGLENMAX)
RT_DebugF("Type = %d",RT_ArrayGetType(ARRAY))
RT_DebugF("Nels = %d",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
RT_DebugF("\nChecking Init")
for(i=0,NELS-1) {
if(TYPE==0) { # Bool
if(RT_ArrayGet(ARRAY,i) != False) {
RT_DebugF("BOOL(%d) Init != False",i)
}
} Else if(TYPE==1) { # Int
if(RT_ArrayGet(ARRAY,i) != 0) {
RT_DebugF("INT(%d) Init != 0",i)
}
} Else if(TYPE==2) { # Float
if(RT_ArrayGet(ARRAY,i) != 0) {
RT_DebugF("FLOAT(%d) Init != 0.0",i)
}
} Else if(TYPE==3) { # String
if(RT_ArrayGet(ARRAY,i) != "") {
RT_DebugF("STRING(%d) Init != %c%c",i,34,34)
}
}
}
Try {RT_ArrayGet(ARRAY,NELS) RT_DebugF("Init END Check FAILED")} catch (err) {RT_DebugF("Init END Check Passed")}
RT_DebugF("Checking Set/Get")
for(i=0,NELS-1) {
if(TYPE==0) {vs = RT_BitTST(i,0) }
Else if(TYPE==1) {vs = i }
Else if(TYPE==2) {vs = Float(i) }
Else if(TYPE==3) {vs=RightStr(ALPHA + "_" + String(i),STRINGLENMAX) }
RT_ArraySet(ARRAY,i,vs)
vg=RT_ArrayGet(ARRAY,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
RT_DebugF("Checking Append/Get")
for(i=NELS,NELS*2-1) {
if(TYPE==0) {vs = RT_BitTST(i,0) }
Else if(TYPE==1) {vs = i }
Else if(TYPE==2) {vs = Float(i) }
Else if(TYPE==3) {vs=RightStr(ALPHA + "_" + String(i),STRINGLENMAX) }
RT_ArrayAppend(ARRAY,vs)
vg=RT_ArrayGet(ARRAY,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
if(RT_ArrayGetNels(ARRAY) != NELS * 2) {
RT_DebugF("Final Check ERROR, RT_ArrayGetNels != NELS *2 (is %d)",RT_ArrayGetNels(ARRAY))
}
RT_DebugF("ALL Checks Done\n")
RT_DebugF("Type = %d",RT_ArrayGetType(ARRAY))
RT_DebugF("Nels = %d (Should have doubled)",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
""")
return colorbars()


avs output to debugview


00000005 1.97394645 RT_DebugF:
00000006 1.98214269 RT_DebugF: Type = 3
00000007 1.98232949 RT_DebugF: Nels = 128
00000008 1.98250961 RT_DebugF: ElSize = 16
00000009 1.98278558 RT_DebugF: NelMax = 134217711 ($7FFFFEF)
00000010 1.98286307 RT_DebugF:
00000011 1.98288894 RT_DebugF: Checking Init
00000012 1.99807060 RT_DebugF: Init END Check Passed
00000013 1.99814630 RT_DebugF: Checking Set/Get
00000014 2.23899317 RT_DebugF: Checking Append/Get
00000015 2.56599188 RT_DebugF: ALL Checks Done
00000016 2.56603646 RT_DebugF:
00000017 2.56626225 RT_DebugF: Type = 3
00000018 2.56645536 RT_DebugF: Nels = 256 (Should have doubled)
00000019 2.56664443 RT_DebugF: ElSize = 16
00000020 2.56693482 RT_DebugF: NelMax = 134217711 ($7FFFFEF)


by Martin53, You wrote that you are going to store the data in files. I assume the plugin will automatically create and destroy them, and to the possible extent, care that they are in the HD cache for fast access...

Will create array, but NOT destroy, you might want to use array again later. Can use RT_FileDelete. Have done nothing to inhibit cache.
Gotta try and get some sleep now.

EDIT: With a array of type string and StringLenMax set sufficiently high, could create a database of avs scripts (or script filenames)
to eg load required plugins and avoid much typing/copying. Just a daft idea. :helpful:

[martin53]

Thank you for the work you did!
I would like to point out some details that I see.
- When scripts are published, the fixed filename, probably including path, might be an obstacle. I made the observation that depending on opening a script in AvsPmod, VDub via file/open or VDub via contextmenu "open with" or via drag & drop over the app window, the current path is sometimes the script path, sometimes the executable path. Classic file IO allowed a file to have a number, or better, an alias; and only the open command needs the file name. Otherwise many lines need to be adapted, and readability is bad.
- I often have several scripts open at the same time. So I'd need to implement a mechanism with an 'exist' loop to find the next free filename, and use that. Only starting from v2.60, AviSynth provides predefined variables for script path and name, and depending on method how script is opened, these variables are not set properly - see old thread (http://forum.doom9.org/showthread.php?p=1258062#post1258062) where we discussed all that stuff.
- If the script crashes, no cleanup funcs (RT_FileDelete()) on script level can be called in most cases (can be done with try/catch, but not every script is enclosed in try/catch etc)

So I'd like to propose these changes:
- RT_ArrayAlloc() allows filename to be empty, will then use %Temp%\UniqueFile itself, maybe return this name instead of Nels
- RT_ArrayAlloc() allows an additional string for the 'handle name' or 'variable name' of the array
- other functions then work with the latter instead of the file name: more naming freedom, better readability
- at least if RT_ArrayAlloc() was called with an empty file name, file is automatically deleted in plugin destructor. Alternatively an additional bool parameter to control this.

[Gavino]

Classic file IO allowed a file to have a number, or better, an alias; and only the open command needs the file name. Otherwise many lines need to be adapted, and readability is bad.
That's easily solved by assigning the name to a variable and using that instead of a literal in the function calls. Then only one line (the assignment) needs to be changed. Indeed, StainlessS's example uses this technique.

[martin53]

OK, that's true.
My objection that I'd prefer automatic filename handling is still valid.

@StainlessS: Is the filename parameter able to expand environment strings like %temp%, or do you suggest to use e.g. the GetSystemEnv plugin for that?

[martin53]

StainlessS,
would like to ask you for two more extensions of RT_Stats.
Overall objective: Plugin checks for published scripts. Users frequently try without installing the prerequisites, are disappointed and need forum help. A better platform to check the installed environment from the script would be helpful.

First extension: RT_plugindir2_5().
Return value: String with registry key HKLM\SOFTWARE\AviSynth\plugindir2_5.
Specific objective: get path name of installed plugins (assumed user does not include them via include commands from somewhere else, but then user is also able to comment out assertion lines)

Second extension: adaption of all 4 GetSystemEnv plugin (http://forum.doom9.org/showthread.php?t=142485) functions, but specifically RT_GetSystemEnv() and RT_GetFileInfo().
Specific objective: enable user to read/write files in %TEMP% directory rather than C:\, WorkingDir or ScriptDir (latter might be write protected, first is not a good place to litter, no other path does exist on all OS installations), and get modification date of plugin dlls. Also, get absolute path of %windir%\system32\avisynth.dll.
RT_GetFileInfo() must return the date and time string as "YYYY-MM-DD hh:mm:ss", not locale-specific, see below

A published script could then assert the existence not only of certain plugins, as withtry { BlankClip.RT_Stats } catch(err) { assert(false, "Please get RT_Stats plugin for this script") }
assert(RT_FunctionExist("mt_lut"), "Please get mt_masktools")
...
but also the minimum version - without requiring that the plugins themselves expose version information - with assert((RT_GetSystemEnv("windir")+"\system32\avisynth.dll").RT_GetFileInfo("ModificationTime")>"2013-01-13", "Please get AviSynth v2.6a4 or higher")
assert(RT_plugindir2_5.RT_GetFileInfo("ModificationTime")>"2013-10-16", "RT_Stats is too old. Please get v1.28")

RT_Stats has really become a swiss knife, and the advantage of including the functions in RT_Stats is that only one assertion for the existence of RT_Stats itself must be made without the help of it.

[StainlessS]

EDIT: @M53,
Not terribly sensible at the moment, did not like almost any of your previous ideas.
but always willing to listen, and when I'm again alive/awake/not drunk/ shall review again.

[StainlessS]

Sorry, M53, not ignoring you, will get around to answering. Please be patient.

[StainlessS]

New Version RT_Stats v1.28Beta2 here:- LINK REMOVED



Added to v1.28 Beta-2


RT_IncrGlobal(string GlobalName,int "Init"=1)
Given the name of a Global Int variable, will increment that Global (ie add 1 to it) and return the result.
If Global int does not exist, will initialize it to the optional Init arg, default 1.
So if GlobalInt is already initialized to 1, then RT_IncrGlobal("GlobalInt") sets GlobalInt to 2
(equiv:- Global GlobalInt = GlobalInt + 1).
If GlobalName exists in local scope as a local variable, then it will set the same named Global Variable
to the value of the local variable + 1, local variables 'hide' similarly named Global variables, and a plugin
cannot tell the difference between local and global variables, so make sure you do not use local variables
of similar names to global ones.
Will throw an error if GlobalName exists but variable is not of type Int.

***
***
***

*****************************************************
********** RT_ARRAY FUNCTIONS ***********
*****************************************************

The RT_ArrayXXXX type functions are primitives intended to be lightweight. Added to v1.28Beta2 are 16 Attribute int's for
whatever purpose the script writer desires, eg dimensions for multi-dimension arrays (multi to single dimension handled by user script)
or some kind of bit flags. It could also be possible in script to create script functions to eg create a dictionary or database using
separate ARRAY's together, perhaps using a MASTER ARRAY to hold filenames of the subordinate arrays, and in the Attributes, the
subordinate array variable type. Perhaps the first subordinate array could hold the name of a plugin and the 2nd the full file path on
YOUR system (or a path relative to a user_plugin_dir on YOUR system). I am waiting for YOU to amaze me with your creativity.

***
***
***

RT_ArrayAlloc(string FileName,int "Type"=1,int "Nels"=0,int "StringLenMax"=256)
Creates a file to hold Array elements of type Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in array as an unsigned 8 bit int, upper 24 bits ignored.
Filename, Array Filename, no default.
Type, int, default 1(0 -> 4). 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.
Nels, default 0. Number of array elements, initialized to zeros (Bool=False,Float=0.0,String="").
StringLenMax, default 256(greater than zero). Fixed Maximum string length excluding nul term. Error if string length exceeds.
Only used for Type=3 (string).
Returns number of elements in array. All RT_ArrayXXXX() errors produce an error alert.

***
***
***

RT_ArrayGetType(string FileName)
Filename, Array Filename, no default.
Returns int, Type as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNels(string FileName)
Filename, Array Filename, no default.
Returns int, Nels as set by RT_ArrayAlloc, or current Nels if RT_ArrayAppend used to add vars to end of array.

***
***
***

RT_ArrayGetElSize(string FileName)
Filename, Array Filename, no default.
Returns int, element size, Type 0(bool)=1, 1(int)=4, 2(float)=4, 3(string)=StringLenMax, 4(BIN)=1, as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNelMax(string FileName)
Filename, Array Filename, no default.
Returns int, maximum number of elements that could theoretically be stored in array, based on (maximum +ve integer - header size) / ElSize.

***
***
***

RT_ArrayGet(string FileName,int Nel)
Filename, Array Filename, no default.
Nel, int, zero relative. Valid range 0 -> RT_ArrayGetNels(FileName) - 1.
Returns Array(Nel) of variable Type as set in RT_ArrayAlloc.
Type BIN returns an Int range 0 -> 255.

***
***
***

RT_ArraySet(string FileName,int Nel,Var)
Filename, Array Filename, no default.
Nel, int, zero relative. Valid range 0 -> RT_ArrayGetNels(FileName) - 1.
Var, a variable of Type as set in RT_ArrayAlloc.
Returns number of elements in array.

***
***
***

RT_ArrayAppend(string FileName,Var)
Adds a variable to end of array and increments the Nel count.
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc.
Returns number of elements in array.

***
***
***

RT_ArrayGetAttrib(string FileName,int ix)
There are 16 avalable int attributes in the Array file header, that can be used for whatever purpose you like,
perhaps array dimensions where user script handles multiple dimension array.
RT_ArrayAlloc() initializes all Attributes to 0.
Filename, Array Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Returns the value of the user set attribute (or 0 if not yet set).

***
***
***

RT_ArraySetAttrib(string FileName,int ix,int value)
There are 16 avalable int attributes in the Array file header, that can be used for whatever purpose you like,
perhaps array dimensions where user script handles multiple dimension array.
RT_ArrayAlloc() initializes all Attributes to 0.
Filename, Array Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Value, int, The value that Attrib(ix) will be set to.
Returns value.


*****************************************************
********** OTHER FUNCTIONS ***********
*****************************************************

RT_Version()
Returns the Float version number of RT_Stats, but slightly modified for beta versions.
A non beta version would return a number like 1.28, with two decimal digits of precision.
A version of v1.28Beta2 shown by RT_Stats() on frame, will return 1.2792 rather than 1.28.
9 in the 1/1000's position signifies the beta version in the 1/10000's column.
This allows direct comparison of version numbers including Beta versions

***
***
***

RT_VersionString()
Return RT_Stats version as a string eg "1.28Beta2".

***
***
***

RT_VersionDll()
Return RT_Stats(26).dll version as a Float eg 2.6 for Avistynth v2.6, or 2.5 for Avisynth 2.5.

***
***
***

RT_ScriptDir()
Returns String, Path to Script Directory. Same as ScriptDir in Avisynth v2.6.
Returns "", if cannot find ScriptDir.

***
***
***

RT_ScriptFile()
Returns String, filename of Script without path. Same as ScriptFile in Avisynth v2.6.
Returns "", if cannot find ScriptFile.


***
***
***

RT_ScriptName()
Returns String, filename of Script including path. Same as ScriptName in Avisynth v2.6.
Returns "", if cannot find ScriptName.


***
***
***

RT_PluginDir()
Returns String, path to Avisynth plugins directory.
Returns "", if plugin dir not found.


***
***
***

RT_InternalFunctions()
Returns String, SPACE separated list of internal filter and function names.
Returns "", if cannot find InternalFunctions.
If you like a good read try this:-
S=RT_StrReplace(RT_InternalFunctions," ",Chr(10))
ColorBars.ScriptClip("""RT_Subtitle("%s",S,align=5,y=height+100-current_frame,expx=true,expy=true)""").killaudio

***
***
***

RT_PluginFunctions()
Returns String, SPACE separated list of external filter and function names.
Returns "", if cannot find PluginFunctions.
If you like a good read try this:-
S=RT_StrReplace(RT_PluginFunctions," ",Chr(10))
ColorBars.ScriptClip("""RT_Subtitle("%s",S,align=5,y=height+100-current_frame,expx=true,expy=true)""").killaudio

***
***
***

RT_PluginParam(String FunctionName)
FunctionName, String, name of Avisynth built-in or Plugin filter or function name.
Returns "", if FunctionName Parameter string not found.
Returns String, the Avisynth CPP Interface style argument specifier string used by Avisynth to determine argument types and optional names.
Optional arguments have square brackets surrounding their name as in [name] and are followed by a type specifier character that gives
the type. Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args, followed by optional
'arg1' of type bool and optional 'arg2' of type int.
# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
To show params:-
colorbars().Killaudio()
S=RT_PluginParam("RT_YStats")
S=RT_StrReplace(S,"[",Chr(10)+"[")
RT_Subtitle(s)

***
***
***




- When scripts are published, the fixed filename, probably including path, might be an obstacle. I made the observation that depending on opening a script in AvsPmod, VDub via file/open or VDub via contextmenu "open with" or via drag & drop over the app window, the current path is sometimes the script path, sometimes the executable path. Classic file IO allowed a file to have a number, or better, an alias; and only the open command needs the file name. Otherwise many lines need to be adapted, and readability is bad.
- I often have several scripts open at the same time. So I'd need to implement a mechanism with an 'exist' loop to find the next free filename, and use that. Only starting from v2.60, AviSynth provides predefined variables for script path and name, and depending on method how script is opened, these variables are not set properly - see old thread (http://forum.doom9.org/showthread.php?p=1258062#post1258062) where we discussed all that stuff.
- If the script crashes, no cleanup funcs (RT_FileDelete()) on script level can be called in most cases (can be done with try/catch, but not every script is enclosed in try/catch etc)

So I'd like to propose these changes:
- RT_ArrayAlloc() allows filename to be empty, will then use %Temp%\UniqueFile itself, maybe return this name instead of Nels
- RT_ArrayAlloc() allows an additional string for the 'handle name' or 'variable name' of the array
- other functions then work with the latter instead of the file name: more naming freedom, better readability
- at least if RT_ArrayAlloc() was called with an empty file name, file is automatically deleted in plugin destructor. Alternatively an additional bool parameter to control this.

Dont really like any of the above, sorry.

It is intended that the array function suite be as lightweight as possible (primitives), and that a user can implement eg multi-dimensional
arrays via script functions. Have now implemented 16 int sized array attributes, could be used to store dimensions, or eg bitflags for
whatever usage.
Having to mess with filenames should be in the hands of the client script, perhaps I could implement a unique filename generator or whatever
but best left out of the array routines and auto deletion is a no-go, runtime filters have no destructor.
The ARRAY file is opened and closed on each access, so the name is needed, I would not entertain the idea of using any kind of file handle
and expecting the script user to deal properly with files, also although it may not be impossible to tie the invididual routines together
to use a single memory area, I would not like to try to implement this and have no idea what additional problems (if any) might arise in
eg multi-threaded versions of avisynth.
Having to use a variable holding a handle, is not really any different to using a variable holding a filename.
Perhaps a function to delete filenames on Avisynth exit could be written, but Ive never written an onexit type func and may have additional
complications if more than a single file were required to be deleted, myself I would prefer if half completed files were left in-situ
(after crash), might help in diagnositcs on cause, and dont really like a file being created 'somewhere on some hard drive'.


@StainlessS: Is the filename parameter able to expand environment strings like %temp%, or do you suggest to use e.g. the GetSystemEnv plugin for that?

I was curious about that myself so I tried it. Tried ARRAY="%TEMP%\TestArray" which failed.


Addition Requests,
RT_PluginDir() DONE
RT_GetSystemEnv() OK, Coming
RT_GetFileInfo() OK, Coming

Written some time later:-
Am thinking about making primitives multi-dimensional, either 2 or 3 max dimensions, if so, then args to array funcs will change.

[StainlessS]

See previous post for new version RT_Stats v1.28Beta2

RT_Test_Array.Avs

# RT_Test_Array.Avs

/*
RT_ArrayAlloc(string FileName,int "Type"=1,int "Nels"=0,int "StringLenMax"=256)
RT_ArrayGetType(string FileName)
RT_ArrayGetNels(string FileName)
RT_ArrayGetElSize(string FileName)
RT_ArrayGetNelMax(string FileName)
RT_ArrayGet(string FileName,int nel)
RT_ArraySet(string FileName,int nel,var)
RT_ArrayAppend(string FileName,var)
RT_ArrayGetAttrib(string FileName,int ix)
RT_ArraySetAttrib(string FileName,int ix,int value)
*/

ARRAY="TestArray.ARR" # ARRAY name
START_TYPE = 0 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int)
END_TYPE = 4 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int) ## >= START_TYPE ##
NELS = 100000 # Inital Number of Array elements (init'ed to zero's) [Will be doubled by Append/Get]
STRINGLENMAX = 64 # Excluding Nul Term.
# Take care with NELS and STRINGLENMAX when processing Type=String.
# Approx string usage greater than (NELS * (STRINGLENMAX+1) * 6)
# String memory not released till Avisynth closure, no dead string garbage collection by Avisynth.
DELETE = True # True = Delete ARRAY file when done

ALPHA = "ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ"
ALPHA=ALPHA+ALPHA # Len 260
ALPHA=(STRINGLENMAX<STRLEN(ALPHA)) ? LeftStr(ALPHA,STRINGLENMAX) : ALPHA # cut down string manips and usage

TYPENAMES=RT_String("BOOL\nINT\nFLOAT\nSTRING\nBIN\n")

GScript("""
RT_DebugF("RT_Test_Array.avs")
TOTAL_START=RT_Timer()
RT_DebugF("")
For(TYPE=START_TYPE,END_TYPE) {
RT_DebugF("==============================")
TYPE_START=RT_Timer()
RT_ArrayAlloc(ARRAY,Type=TYPE,Nels=NELS,stringlenmax=STRINGLENMAX)
STOP=RT_Timer()
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Nels = %d",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
RT_DebugF("Alloc Time = %.3f secs (Per element %f microsecs)",STOP-TYPE_START,1000000.0*(STOP-TYPE_START)/NELS)
RT_DebugF("\nCheck Init")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i)) {RT_DebugF("BOOL(%d) Init != False",i)}}
} Else if(TYPE==1) { # Int
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != 0) {RT_DebugF("INT(%d) Init != 0",i)}}
} Else if(TYPE==2) { # Float
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != 0) {RT_DebugF("FLOAT(%d) Init != 0.0",i)}}
} Else if(TYPE==3) { # String
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != ""){RT_DebugF("STRING(%d) Init != %c%c",i,34,34)}}
} Else if(TYPE==4) { # BIN
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != 0) {RT_DebugF("BIN(%d) Init != 0",i)}}
}
Try {RT_ArrayGet(ARRAY,NELS) RT_DebugF("Init END Check FAILED")} catch (err) {RT_DebugF("Init END Check Passed")}
STOP=RT_Timer()
RT_DebugF("Check Init Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Attributes Set/Get")
START=RT_Timer()
for(i=0,15) {
vs=i*1000
RT_ArraySetAttrib(ARRAY,i,vs)
vg = RT_ArrayGetAttrib(ARRAY,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d SetAttrib/GetAttrib Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Attributes Set/Get = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Set only Time Test on %d Nels",NELS)
vs=Select(TYPE,True,42,42.0,RightStr(ALPHA + "_" + String(42),STRINGLENMAX) ,42)
START=RT_Timer()
for(i=0,NELS-1) {
RT_ArraySet(ARRAY,i,vs)
}
STOP=RT_Timer()
RT_DebugF("Check Set Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Get only Time Test on %d Nels",NELS)
START=RT_Timer()
for(i=0,NELS-1) {RT_ArrayGet(ARRAY,i)}
STOP=RT_Timer()
RT_DebugF("Check Get Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Compare Set/Get")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,NELS-1) {
vs = RT_BitTST(i,0) RT_ArraySet(ARRAY,i,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==1) { # Int
for(i=0,NELS-1) {
vs = i RT_ArraySet(ARRAY,i,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==2) { # Float
for(i=0,NELS-1) {
vs = Float(i) RT_ArraySet(ARRAY,i,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==3) { # String
for(i=0,NELS-1) {
vs=RightStr(ALPHA + "_" + String(i),STRINGLENMAX) RT_ArraySet(ARRAY,i,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==4) { # BIN
for(i=0,NELS-1) {
vs = RT_BitAnd(i,$FF) RT_ArraySet(ARRAY,i,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
}
STOP=RT_Timer()
RT_DebugF("Check Compare Set/Get Time = %.3f secs (Per element %f microsecs) [Including Data generation]",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Compare Append/Get")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=NELS,NELS*2-1) {
vs = RT_BitTST(i,0) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==1) { # Int
for(i=NELS,NELS*2-1) {
vs = i RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==2) { # Float
for(i=NELS,NELS*2-1) {
vs = Float(i) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==3) { # String
for(i=NELS,NELS*2-1) {
vs=RightStr(ALPHA + "_" + String(i),STRINGLENMAX) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==4) { # BIN
for(i=NELS,NELS*2-1) {
vs = RT_BitAnd(i,$FF) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
}
if(RT_ArrayGetNels(ARRAY) != NELS * 2) {
RT_DebugF("Final Check ERROR, RT_ArrayGetNels != NELS *2 (is %d)",RT_ArrayGetNels(ARRAY))
}
STOP=RT_Timer()
RT_DebugF("Check Compare Append/Get Time = %.3f secs (Per element %f microsecs) [Including Data generation]",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("ALL Checks Done\n")
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Nels = %d (Should have doubled)",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
STOP=RT_Timer()
RT_DebugF("Total Time Type=%d (%s) = %.3f secs",Type,RT_TxtGetLine(TYPENAMES,Type),STOP-TYPE_START)
}
(DELETE) ? RT_FileDelete(ARRAY) : NOP
TOTAL_STOP=RT_Timer()
RT_DebugF("SCRIPT TOTAL TIME = %.3f secs",TOTAL_STOP-TOTAL_START)
""")
return colorbars()


Results Following post (exceeds D9 16kb limit)

[StainlessS]

See previous two posts

Results from debugview

00000001 0.00000000 RT_DebugF: RT_Test_Array.avs
00000002 0.00013451 RT_DebugF:
00000003 0.00022344 RT_DebugF: ==============================
00000004 0.00160964 RT_DebugF: Type = 0 (BOOL)
00000005 0.00178056 RT_DebugF: Nels = 100000
00000006 0.00194980 RT_DebugF: ElSize = 1
00000007 0.00219927 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000008 0.00229950 RT_DebugF: Alloc Time = 0.015 secs (Per element 0.150000 microsecs)
00000009 0.00238137 RT_DebugF:
00000010 0.00240546 RT_DebugF: Check Init
00000011 8.29076195 RT_DebugF: Init END Check Passed
00000012 8.29092121 RT_DebugF: Check Init Time = 8.281 secs (Per element 82.809990 microsecs)
00000013 8.29100418 RT_DebugF: Check Attributes Set/Get
00000014 8.29455185 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000015 8.29464054 RT_DebugF: Check Set only Time Test on 100000 Nels
00000016 20.87353134 RT_DebugF: Check Set Only Time = 12.579 secs (Per element 125.790001 microsecs)
00000017 20.87361526 RT_DebugF: Check Get only Time Test on 100000 Nels
00000018 29.33578873 RT_DebugF: Check Get Only Time = 8.468 secs (Per element 84.680000 microsecs)
00000019 29.33587074 RT_DebugF: Check Compare Set/Get
00000020 57.12252045 RT_DebugF: Check Compare Set/Get Time = 27.782 secs (Per element 277.820007 microsecs) [Including Data generation]
00000021 57.12260437 RT_DebugF: Check Compare Append/Get
00000022 86.21274567 RT_DebugF: Check Compare Append/Get Time = 29.093 secs (Per element 290.930023 microsecs) [Including Data generation]
00000023 86.21282959 RT_DebugF: ALL Checks Done
00000024 86.21285248 RT_DebugF:
00000025 86.21306610 RT_DebugF: Type = 0 (BOOL)
00000026 86.21323395 RT_DebugF: Nels = 200000 (Should have doubled)
00000027 86.21340179 RT_DebugF: ElSize = 1
00000028 86.21364594 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000029 86.21384430 RT_DebugF: Total Time Type=0 (BOOL) = 86.218 secs
00000030 86.21392822 RT_DebugF: ==============================
00000031 86.21581268 RT_DebugF: Type = 1 (INT)
00000032 86.21598053 RT_DebugF: Nels = 100000
00000033 86.21614838 RT_DebugF: ElSize = 4
00000034 86.21640015 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000035 86.21649933 RT_DebugF: Alloc Time = 0.016 secs (Per element 0.159988 microsecs)
00000036 86.21658325 RT_DebugF:
00000037 86.21660614 RT_DebugF: Check Init
00000038 94.72898102 RT_DebugF: Init END Check Passed
00000039 94.72914124 RT_DebugF: Check Init Time = 8.500 secs (Per element 85.000000 microsecs)
00000040 94.72921753 RT_DebugF: Check Attributes Set/Get
00000041 94.73278809 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.159988 microsecs)
00000042 94.73288727 RT_DebugF: Check Set only Time Test on 100000 Nels
00000043 107.19555664 RT_DebugF: Check Set Only Time = 12.453 secs (Per element 124.530029 microsecs)
00000044 107.19564056 RT_DebugF: Check Get only Time Test on 100000 Nels
00000045 115.66921234 RT_DebugF: Check Get Only Time = 8.484 secs (Per element 84.839943 microsecs)
00000046 115.66929626 RT_DebugF: Check Compare Set/Get
00000047 137.12696838 RT_DebugF: Check Compare Set/Get Time = 21.453 secs (Per element 214.530014 microsecs) [Including Data generation]
00000048 137.12704468 RT_DebugF: Check Compare Append/Get
00000049 159.96316528 RT_DebugF: Check Compare Append/Get Time = 22.844 secs (Per element 228.439941 microsecs) [Including Data generation]
00000050 159.96325684 RT_DebugF: ALL Checks Done
00000051 159.96327209 RT_DebugF:
00000052 159.96348572 RT_DebugF: Type = 1 (INT)
00000053 159.96365356 RT_DebugF: Nels = 200000 (Should have doubled)
00000054 159.96382141 RT_DebugF: ElSize = 4
00000055 159.96406555 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000056 159.96424866 RT_DebugF: Total Time Type=1 (INT) = 73.766 secs
00000057 159.96434021 RT_DebugF: ==============================
00000058 159.96620178 RT_DebugF: Type = 2 (FLOAT)
00000059 159.96636963 RT_DebugF: Nels = 100000
00000060 159.96653748 RT_DebugF: ElSize = 4
00000061 159.96679688 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000062 159.96688843 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000063 159.96696472 RT_DebugF:
00000064 159.96701050 RT_DebugF: Check Init
00000065 168.78349304 RT_DebugF: Init END Check Passed
00000066 168.78366089 RT_DebugF: Check Init Time = 8.812 secs (Per element 88.120117 microsecs)
00000067 168.78373718 RT_DebugF: Check Attributes Set/Get
00000068 168.78729248 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000069 168.78738403 RT_DebugF: Check Set only Time Test on 100000 Nels
00000070 181.40660095 RT_DebugF: Check Set Only Time = 12.625 secs (Per element 126.250000 microsecs)
00000071 181.40667725 RT_DebugF: Check Get only Time Test on 100000 Nels
00000072 189.87194824 RT_DebugF: Check Get Only Time = 8.469 secs (Per element 84.689941 microsecs)
00000073 189.87202454 RT_DebugF: Check Compare Set/Get
00000074 237.55381775 RT_DebugF: Check Compare Set/Get Time = 47.688 secs (Per element 476.880035 microsecs) [Including Data generation]
00000075 237.55390930 RT_DebugF: Check Compare Append/Get
00000076 286.65341187 RT_DebugF: Check Compare Append/Get Time = 49.093 secs (Per element 490.929871 microsecs) [Including Data generation]
00000077 286.65350342 RT_DebugF: ALL Checks Done
00000078 286.65350342 RT_DebugF:
00000079 286.65371704 RT_DebugF: Type = 2 (FLOAT)
00000080 286.65390015 RT_DebugF: Nels = 200000 (Should have doubled)
00000081 286.65405273 RT_DebugF: ElSize = 4
00000082 286.65429688 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000083 286.65451050 RT_DebugF: Total Time Type=2 (FLOAT) = 126.687 secs
00000084 286.65457153 RT_DebugF: ==============================
00000085 286.67214966 RT_DebugF: Type = 3 (STRING)
00000086 286.67233276 RT_DebugF: Nels = 100000
00000087 286.67248535 RT_DebugF: ElSize = 64
00000088 286.67276001 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000089 286.67288208 RT_DebugF: Alloc Time = 0.016 secs (Per element 0.160217 microsecs)
00000090 286.67297363 RT_DebugF:
00000091 286.67297363 RT_DebugF: Check Init
00000092 295.52035522 RT_DebugF: Init END Check Passed
00000093 295.52050781 RT_DebugF: Check Init Time = 8.859 secs (Per element 88.589783 microsecs)
00000094 295.52059937 RT_DebugF: Check Attributes Set/Get
00000095 295.52416992 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000096 295.52426147 RT_DebugF: Check Set only Time Test on 100000 Nels
00000097 308.34729004 RT_DebugF: Check Set Only Time = 12.829 secs (Per element 128.290100 microsecs)
00000098 308.34735107 RT_DebugF: Check Get only Time Test on 100000 Nels
00000099 316.90744019 RT_DebugF: Check Get Only Time = 8.546 secs (Per element 85.459900 microsecs)
00000100 316.90750122 RT_DebugF: Check Compare Set/Get
00000101 365.86749268 RT_DebugF: Check Compare Set/Get Time = 48.969 secs (Per element 489.690247 microsecs) [Including Data generation]
00000102 365.86758423 RT_DebugF: Check Compare Append/Get
00000103 416.47927856 RT_DebugF: Check Compare Append/Get Time = 50.610 secs (Per element 506.099854 microsecs) [Including Data generation]
00000104 416.47937012 RT_DebugF: ALL Checks Done
00000105 416.47937012 RT_DebugF:
00000106 416.47958374 RT_DebugF: Type = 3 (STRING)
00000107 416.47976685 RT_DebugF: Nels = 200000 (Should have doubled)
00000108 416.47991943 RT_DebugF: ElSize = 64
00000109 416.48022461 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000110 416.48040771 RT_DebugF: Total Time Type=3 (STRING) = 129.829 secs
00000111 416.48049927 RT_DebugF: ==============================
00000112 416.48214722 RT_DebugF: Type = 4 (BIN)
00000113 416.48233032 RT_DebugF: Nels = 100000
00000114 416.48251343 RT_DebugF: ElSize = 1
00000115 416.48278809 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000116 416.48287964 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000117 416.48297119 RT_DebugF:
00000118 416.48300171 RT_DebugF: Check Init
00000119 425.30059814 RT_DebugF: Init END Check Passed
00000120 425.30075073 RT_DebugF: Check Init Time = 8.813 secs (Per element 88.129883 microsecs)
00000121 425.30084229 RT_DebugF: Check Attributes Set/Get
00000122 425.30444336 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000123 425.30453491 RT_DebugF: Check Set only Time Test on 100000 Nels
00000124 437.77297974 RT_DebugF: Check Set Only Time = 12.468 secs (Per element 124.679871 microsecs)
00000125 437.77307129 RT_DebugF: Check Get only Time Test on 100000 Nels
00000126 446.27523804 RT_DebugF: Check Get Only Time = 8.500 secs (Per element 85.000000 microsecs)
00000127 446.27529907 RT_DebugF: Check Compare Set/Get
00000128 474.72674561 RT_DebugF: Check Compare Set/Get Time = 28.454 secs (Per element 284.540100 microsecs) [Including Data generation]
00000129 474.72683716 RT_DebugF: Check Compare Append/Get
00000130 504.42544556 RT_DebugF: Check Compare Append/Get Time = 29.703 secs (Per element 297.030029 microsecs) [Including Data generation]
00000131 504.42553711 RT_DebugF: ALL Checks Done
00000132 504.42556763 RT_DebugF:
00000133 504.42578125 RT_DebugF: Type = 4 (BIN)
00000134 504.42593384 RT_DebugF: Nels = 200000 (Should have doubled)
00000135 504.42611694 RT_DebugF: ElSize = 1
00000136 504.42636108 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000137 504.42654419 RT_DebugF: Total Time Type=4 (BIN) = 87.953 secs
00000138 504.42694092 RT_DebugF: SCRIPT TOTAL TIME = 504.453 secs


Core Duo Dual Core 2.13GHz, 3GB DDR2, Half full non-system 320 GB SATA2

[martin53]

1) perhaps I could implement a unique filename generator or whatever but best left out of the array routines
2) auto deletion is a no-go, runtime filters have no destructor
3) Having to use a variable holding a handle, is not really any different to using a variable holding a filename

1) agree, maybe current system time with sufficient fine resolution is good for that (returned as >6 digit string? NTFS uses a 100ns time resolution afaik)
2) oops - agree, of course
3) agree, Gavino stated that too, variable name can be chosen as to indicate the array purpose

Also agree with the aspect that non-deleted file can help with debugging.

If %TEMP% can be resolved one day (outside of array routines is OK for that), then I can clean up that dir from time to time. During script writing, I have *lots* of crashes, :'(

[martin53]

Could not keep myself from trying these funny functions
global PATH = RT_ScriptDir + "\functions.txt"
RT_TxtWriteFile("======== Internal Functions ========", PATH, false)
RT_InternalFunctions.Write
RT_TxtWriteFile("======== Plugin Functions ========", PATH, true)
RT_PluginFunctions.Write
function Write(string s) {
i = s.FindStr(" ")
GScript(""" #"
while(i>0) {
if (i>1) {
fu = s.LeftStr(i-1)
if ((s.MidStr(i)).FindStr(" "+fu+" ") == 0 && (s.MidStr(i)).FindStr("_"+fu+" ") == 0) {
RT_TxtWriteFile(fu + " ( " + fu.RT_PluginParam + " )", PATH, true)
}
}
s = s.MidStr(i+1)
i = s.FindStr(" ")
}
""") #"
}
BlankClip().subtitle("See "+PATH) :D

[StainlessS]

Nice script m53.
Be aware that the latest string returning functions are bugged, I forgot that the internal CPP functions can release
these strings when eg loading new plugin and so could point at free'ed memory, have fixed but currently testing
RT_GetFileTime(), will release fixes later today.

EDIT:
If %TEMP% can be resolved

Done

[StainlessS]

New version RT_Stats v1.28Beta3 here:- LINK REMOVED
Bugged string returning functions fixed.

Additional:


RT_GetSystemEnv(String envname)
Returns a string from the System Environment with the env name of the string arg.
Returns "", if cannot find environment variable of given name.
eg, TEMP=RT_GetSystemEnv("TEMP") # to get TEMP folder path.
and eg
ComSpec=RT_GetSystemEnv("ComSpec") # might return "C:\WINDOWS\system32\cmd.exe"
To see the environment variables from command console, type 'set'.
http://en.wikipedia.org/wiki/Environment_variable

***
***
***

RT_GetFileTime(string filename,int item)
Returns string, one of three times associated with filename file.
Error if cannot access file. (debugview for error string).
Item, int, (0 -> 2). 0=Creation Time, 1=Last Write/Modified time 2=Last Accessed Time,
The return string is always of format "YYYY-MM-DD HH:MM:SS" and so can be used to
compare whether one file time is later than another using string comparison.
The time strings returned are in UTC (Coordinated Universal Time), not local time.


EDIT: Modified RT_Subtitle with additional Int "Esc"=1 arg:-

RT_Subtitle(clip source,string format,dat1,...,datn,int "align",int "x","y",bool "vcent"=false,bool "expx"=false,bool "expy"=false,int "esc"=1)
Standard filter function.
This function is a standard filter graph function, not a runtime/compile time function.
v2.5 plugin dll limited to v2.58 colorspaces.
Prints formatted text to clip frame using Avisynth Info.h source font (monospaced) and DDigit font renderer, faster than using
Subtitle system fonts. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the text string that is
printed on frame, uses C/CPP printf() style formatting. The args 'align', 'x', and 'y', work in a similar fashion to
Avisynth Subtitle()
expx and expy control whether x and y are same as in SubTitle when eg y = -1 (vertical centered) or y=height (bottom aligned),
or as explicit coords when expy = true.

esc int, default 1. (0 -> 2). Escape code processing eg '\n' converted to Chr(10).
0 = No Escape code processing.
1 = Esc codes processed only in format string. (Default)
2 = Esc codes processed in both format and data strings.


Previously worked as for Esc=2.

Example to show colors:-

Colorbars.KillAudio
S=""" \a!\\a! DDIGIT_CC_HILITE
\a-\\a- DDIGIT_CC_DEFAULT
\a0\\a0 DDIGIT_CC_DARKGRAY
\a1\\a1 DDIGIT_CC_DODGERBLUE
\a2\\a2 DDIGIT_CC_ORANGERED
\a3\\a3 DDIGIT_CC_ORCHID
\a4\\a4 DDIGIT_CC_LIME
\a5\\a5 DDIGIT_CC_AQUAMARINE
\a6\\a6 DDIGIT_CC_YELLOW
\a7\\a7 DDIGIT_CC_WHITE
\a8\\a8 DDIGIT_CC_SILVER
\a9\\a9 DDIGIT_CC_CORNFLOWERBLUE
\aA\\aA DDIGIT_CC_ORANGE
\aB\\aB DDIGIT_CC_PLUM
\aC\\aC DDIGIT_CC_CHARTREUSE
\aD\\aD DDIGIT_CC_POWDERBLUE
\aE\\aE DDIGIT_CC_GOLD
\aF\\aF DDIGIT_CC_GAINSBORO
\aG\\aG DDIGIT_CC_Y_0
\aH\\aH DDIGIT_CC_Y_1
\aI\\aI DDIGIT_CC_Y_2
\aJ\\aJ DDIGIT_CC_Y_3
\aK\\aK DDIGIT_CC_Y_4
\aL\\aL DDIGIT_CC_Y_5
\aM\\aM DDIGIT_CC_Y_6
\aN\\aN DDIGIT_CC_Y_7
\aO\\aO DDIGIT_CC_Y_8
\aP\\aP DDIGIT_CC_Y_9
\aQ\\aQ DDIGIT_CC_Y_A
\aR\\aR DDIGIT_CC_Y_B
\aS\\aS DDIGIT_CC_Y_C
\aT\\aT DDIGIT_CC_Y_D
\aU\\aU DDIGIT_CC_Y_E
\aV\\aV DDIGIT_CC_Y_F"""
DELAY=100
ESC=2 # esc=2, ESC codes allowed in data strings
Scriptclip("""RT_Subtitle("%s",S,align=1,y=height+DELAY-current_frame,expy=true,esc=ESC)""")
return last

[StainlessS]

New version RT_Stats v1.28Beta4 here: LINK REMOVED
Added function


RT_LocalTimeString(Bool "file"=True)
Returns current local time as a string.
Where digits Y=Year, M=Month, D=Day, H=Hour, M=Minute, S=Second, m=millisecond.
When bool file==False, then string in format "YYYY-MM-DD HH:MM:SS.mmm"
When bool file==True (Default) string is in format "YYYYMMDD_HHMMSS_mmm"
Also when file==True, function first waits until the system tick count is incremented (about every 10ms)
before inquiring system time. This is to prevent 2 consecutive calls returning the same time string.
Perhaps useful for temporary filename generation.

[martin53]

Highly appreciate that! Will be unable to use in example until end of next week, but then will implement the new features in one of my RTE scripts, giving multi instance local var scope, true non linear access over frame groups. Wow!

[StainlessS]

Post #1 of 7

New Version RT_Stats v1.28Beta5, Here:- http://www.mediafire.com/download/4vk8jvkfo9qq2wu/RT_Stats_25%2626_dll_v1.28Beta5_20131109.zip

Changed:

RT_GetFileTime(string filename,int item)
Returns string, one of three times associated with filename file.
Error if cannot access file. (debugview for error string).
Item, int, (0 -> 2). 0=Creation Time, 1=Last Write/Modified Time 2=Last Accessed Time,
The return string is always of format "YYYY-MM-DD HH:MM:SS.mmm" and so can be used to
compare whether one file time is later than another using string comparison.
The time strings returned are in UTC (Coordinated Universal Time), not local time.
v1.28Beta5, added milliseconds to string.


RT_ArrayXXXX Modified for multi-dimensional arrays of 1, 2 and 3 dimensions.


*****************************************************
********** RT_ARRAY FUNCTIONS ***********
*****************************************************

RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,int "Dim3"=0,int "StringLenMax"=256)
Creates a file to hold Array elements of type Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in array as an unsigned 8 bit int, upper 24 bits ignored.
Filename, Array Filename, no default.
Type, int, default 1(0 -> 4). 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.
Dim1, Dim2, Dim3, all default 0. Number of array elements per dimension (max 3 dimensions),
initialized to zeros (Bool=False,Float=0.0,String="").
With all defaults of 0, would be single dimension array with zero allocated elements, must use RT_ArrayAppend to add elements to the array.
With only Dim1 set to eg 1000, would create single dimension array of 1000 elements (0 -> 999), where RT_ArrayAppend can also be used.
With Dim1 and Dim2 greater than zero, would create a two dimensional array, where RT_ArrayAppend can NOT be used.
With Dim1, Dim2 and Dim3 greater than zero, would create a three dimensional array, where RT_ArrayAppend can NOT be used.
Will produce an error if Dim3 is non zero and Dim2 or Dim1 is zero, also error if Dim2 is non zero and Dim1 is zero.
It may be less efficient if you choose to set a dimension to 1 rather than 0, is eg better to use Dim1=100 than Dim1=100,Dim2=1,
also, as a two dimension array, you could not use RT_ArrayAppend.
StringLenMax, default 256(1 or more). Fixed Maximum string length excluding nul term. Error if string length exceeds.
Only used for Type=3 (string).
Returns total number of elements in array. All RT_ArrayXXXX() errors produce an error alert.

***
***
***

RT_ArrayGetDim(string FileName,int "Dim"=0)
Returns number of dimensions in array or the size of any 1 of the dimensions.
Filename, Array Filename, no default.
Dim, default 0. (Range 0 -> 3).
0, returns the number of dimensions in the array.
Invalid dimensions will return 0, eg RT_ArrayGetDim(FileName,Dim=2) on single dimension array return 0.
Error if Dim smaller than 0 or greater than 3.

***
***
***

RT_ArrayGetType(string FileName)
Filename, Array Filename, no default.
Returns int, Type as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNels(string FileName)
Filename, Array Filename, no default.
Returns int, total number of elements in array as set by RT_ArrayAlloc, or current number of elements if RT_ArrayAppend used to add
elements to end of single dimension array.

***
***
***

RT_ArrayGetElSize(string FileName)
Filename, Array Filename, no default.
Returns int, element size, Type 0(bool)=1, 1(int)=4, 2(float)=4, 3(string)=StringLenMax, 4(BIN)=1, as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNelMax(string FileName)
Filename, Array Filename, no default.
Returns int, maximum number of elements that could theoretically be stored in a single dimension array, based on
(maximum +ve integer - header size) / ElSize.

***
***
***

RT_ArrayGet(string FileName,int ix1,int "ix2",int "ix3")
Returns array element of variable Type as set in RT_ArrayAlloc. Type BIN returns an Int range 0 -> 255.
Filename, Array Filename, no default.
ixl, int, zero relative. 1st subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,1)-1.
ix2, int, zero relative. 2nd subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,2)-1.
Only valid if 2 or 3 dimensional array. For single dimension array, can omit, supply RT_Undefined(), or -1.
ix3, int, zero relative. 3rd subcript of array. Valid range 0 -> RT_ArrayGetDim(FileName,3)-1.
Only valid if 3 dimensional array. For 1 or 2 dimension array, can omit, supply RT_Undefined(), or -1.

***
***
***

RT_ArraySet(string FileName,Var,int ix1,int "ix2",int "ix3")
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc. (NOTE position in arg list changed)
ixl, int, zero relative. 1st subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,1)-1.
ix2, int, zero relative. 2nd subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,2)-1.
Only valid if 2 or 3 dimensional array. For single dimension array, can omit, supply RT_Undefined(), or -1.
ix3, int, zero relative. 3rd subcript of array. Valid range 0 -> RT_ArrayGetDim(FileName,3)-1.
Only valid if 3 dimensional array. For 1 or 2 dimension array, can omit, supply RT_Undefined(), or -1.
Returns total number of elements in array.

***
***
***

RT_ArrayAppend(string FileName,Var)
Adds a variable to end of single dimension array and increments the Nel count.
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc.
Returns number of elements in single dimension array.
Error if multi dimensional array.

***
***
***

RT_ArraySetAttrib(string FileName,int ix, Value)
There are 16 avalable attributes in the Array file header, that can be used for whatever purpose you like,
RT_ArrayAlloc() initializes all Attributes to type Int, 0.
Filename, Array Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Value, Int or Float, The value that Attrib(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.

***
***
***

RT_ArrayGetAttrib(string FileName,int ix)
There are 16 avalable attributes in the Array file header, that can be used for whatever purpose you like,
Filename, Array Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Returns the value of the user set attribute Int or Float (or type Int 0 if not yet set).


Comments on interface implementation appreciated

[StainlessS]

Post #2 of 7

RT_Test_Array_1_Dim.Avs

# RT_Test_Array_1_Dim.Avs, 1 Dimensional array

/*
RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,,int "Dim3"=0,int "StringLenMax"=256)
RT_ArrayGetDim(string FileName,int "Dim"=0)
RT_ArrayGetType(string FileName)
RT_ArrayGetNels(string FileName)
RT_ArrayGetElSize(string FileName)
RT_ArrayGetNelMax(string FileName)
RT_ArrayGet(string FileName,int ix1,int "ix2",int "ix3")
RT_ArraySet(string FileName, var ,int ix1,int "ix2",int "ix3") # Var position in arg list changed
RT_ArrayAppend(string FileName, var)
RT_ArrayGetAttrib(string FileName,int ix)
RT_ArraySetAttrib(string FileName,int ix, value) # Value can be Int or Float.
*/

ARRAY="TestArray.ARR" # ARRAY name
START_TYPE = 0 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int)
END_TYPE = 4 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int) ## >= START_TYPE ##
NELS = 100000 # Inital Number of Array elements (init'ed to zero's) [Will be doubled by Append/Get]
STRINGLENMAX = 64 # Excluding Nul Term.
# Take care with NELS and STRINGLENMAX when processing Type=String.
# Approx string usage greater than (NELS * (STRINGLENMAX+1) * 6)
# String memory not released till Avisynth closure, no dead string garbage collection by Avisynth.
DELETE = True # True = Delete ARRAY file when done

ALPHA = "ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ"
ALPHA=ALPHA+ALPHA # Len 260
ALPHA=(STRINGLENMAX<STRLEN(ALPHA)) ? LeftStr(ALPHA,STRINGLENMAX) : ALPHA # cut down string manips and usage

TYPENAMES=RT_String("BOOL\nINT\nFLOAT\nSTRING\nBIN\n")

GScript("""
RT_DebugF("RT_Test_Arrayt_1_Dim.avs - Single Dimension Array")
TOTAL_START=RT_Timer()
RT_DebugF("")
For(TYPE=START_TYPE,END_TYPE) {
RT_DebugF("==============================")
TYPE_START=RT_Timer()
RT_ArrayAlloc(ARRAY,Type=TYPE,Dim1=NELS,stringlenmax=STRINGLENMAX)
STOP=RT_Timer()
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Dimensions = %d",RT_ArrayGetDim(ARRAY))
RT_DebugF("Nels = %d",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
RT_DebugF("Alloc Time = %.3f secs (Per element %f microsecs)",STOP-TYPE_START,1000000.0*(STOP-TYPE_START)/NELS)
RT_DebugF("\nCheck Init")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i)) {RT_DebugF("BOOL(%d) Init != False",i)}}
} Else if(TYPE==1) { # Int
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != 0) {RT_DebugF("INT(%d) Init != 0",i)}}
} Else if(TYPE==2) { # Float
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != 0.0) {RT_DebugF("FLOAT(%d) Init != 0.0",i)}}
} Else if(TYPE==3) { # String
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != ""){RT_DebugF("STRING(%d) Init != %c%c",i,34,34)}}
} Else if(TYPE==4) { # BIN
for(i=0,NELS-1) {if(RT_ArrayGet(ARRAY,i) != 0) {RT_DebugF("BIN(%d) Init != 0",i)}}
}
Try {RT_ArrayGet(ARRAY,NELS) RT_DebugF("Init END Check FAILED")} catch (err) {RT_DebugF("Init END Check Passed")}
STOP=RT_Timer()
RT_DebugF("Check Init Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Attributes Set/Get")
START=RT_Timer()
for(i=0,15) {
vs= (i % 2 == 0) ? i*1000 : i * PI # Test Int and Float Attributes
RT_ArraySetAttrib(ARRAY,i,vs)
vg = RT_ArrayGetAttrib(ARRAY,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d SetAttrib/GetAttrib Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Attributes Set/Get = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Set only Time Test on %d Nels",NELS)
vs=Select(TYPE,True,42,42.0,RightStr(ALPHA + "_" + String(42),STRINGLENMAX) ,42)
START=RT_Timer()
for(i=0,NELS-1) {
RT_ArraySet(ARRAY,vs,i)
}
STOP=RT_Timer()
RT_DebugF("Check Set Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Get only Time Test on %d Nels",NELS)
START=RT_Timer()
for(i=0,NELS-1) {RT_ArrayGet(ARRAY,i)}
STOP=RT_Timer()
RT_DebugF("Check Get Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Compare Set/Get")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,NELS-1) {
vs = RT_BitTST(i,0) RT_ArraySet(ARRAY,vs,i) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==1) { # Int
for(i=0,NELS-1) {
vs = i RT_ArraySet(ARRAY,vs,i) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==2) { # Float
for(i=0,NELS-1) {
vs = Float(i) RT_ArraySet(ARRAY,vs,i) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==3) { # String
for(i=0,NELS-1) {
vs=RightStr(ALPHA + "_" + String(i),STRINGLENMAX) RT_ArraySet(ARRAY,vs,i) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==4) { # BIN
for(i=0,NELS-1) {
vs = RT_BitAnd(i,$FF) RT_ArraySet(ARRAY,vs,i) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
}
STOP=RT_Timer()
RT_DebugF("Check Compare Set/Get Time = %.3f secs (Per element %f microsecs) [Including Data generation]",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Compare Append/Get")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=NELS,NELS*2-1) {
vs = RT_BitTST(i,0) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==1) { # Int
for(i=NELS,NELS*2-1) {
vs = i RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==2) { # Float
for(i=NELS,NELS*2-1) {
vs = Float(i) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==3) { # String
for(i=NELS,NELS*2-1) {
vs=RightStr(ALPHA + "_" + String(i),STRINGLENMAX) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
} Else if(TYPE==4) { # BIN
for(i=NELS,NELS*2-1) {
vs = RT_BitAnd(i,$FF) RT_ArrayAppend(ARRAY,vs) vg=RT_ArrayGet(ARRAY,i)
if(vs != vg) {
RT_DebugF("%d) TYPE=%d Append/Get Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
}
if(RT_ArrayGetNels(ARRAY) != NELS * 2) {
RT_DebugF("Final Check ERROR, RT_ArrayGetNels != NELS *2 (is %d)",RT_ArrayGetNels(ARRAY))
}
STOP=RT_Timer()
RT_DebugF("Check Compare Append/Get Time = %.3f secs (Per element %f microsecs) [Including Data generation]",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("ALL Checks Done\n")
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Dimensions = %d",RT_ArrayGetDim(ARRAY))
RT_DebugF("Nels = %d (should have doubled)",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
STOP=RT_Timer()
RT_DebugF("Total Time Type=%d (%s) = %.3f secs",Type,RT_TxtGetLine(TYPENAMES,Type),STOP-TYPE_START)
}
(DELETE) ? RT_FileDelete(ARRAY) : NOP
TOTAL_STOP=RT_Timer()
RT_DebugF("SCRIPT TOTAL TIME = %.3f secs",TOTAL_STOP-TOTAL_START)
""")
return colorbars()

[StainlessS]

Post #3 of 7

RT_Test_Array_1_Dim.Log

00000001 0.00000000 RT_DebugF: RT_Test_Array_1_Dim.avs - Single Dimension Array
00000002 0.00013589 RT_DebugF:
00000003 0.00022477 RT_DebugF: ==============================
00000004 0.00121931 RT_DebugF: Type = 0 (BOOL)
00000005 0.00138569 RT_DebugF: Dimensions = 1
00000006 0.00155160 RT_DebugF: Nels = 100000
00000007 0.00171441 RT_DebugF: ElSize = 1
00000008 0.00195542 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000009 0.00204794 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000010 0.00213082 RT_DebugF:
00000011 0.00215442 RT_DebugF: Check Init
00000012 7.93041897 RT_DebugF: Init END Check Passed
00000013 7.93057775 RT_DebugF: Check Init Time = 7.937 secs (Per element 79.370003 microsecs)
00000014 7.93066311 RT_DebugF: Check Attributes Set/Get
00000015 7.93823862 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000016 7.93832684 RT_DebugF: Check Set only Time Test on 100000 Nels
00000017 19.25387573 RT_DebugF: Check Set Only Time = 11.328 secs (Per element 113.279991 microsecs)
00000018 19.25395966 RT_DebugF: Check Get only Time Test on 100000 Nels
00000019 27.35041046 RT_DebugF: Check Get Only Time = 8.094 secs (Per element 80.940002 microsecs)
00000020 27.35049629 RT_DebugF: Check Compare Set/Get
00000021 53.95354843 RT_DebugF: Check Compare Set/Get Time = 26.594 secs (Per element 265.940002 microsecs) [Including Data generation]
00000022 53.95363235 RT_DebugF: Check Compare Append/Get
00000023 81.46318817 RT_DebugF: Check Compare Append/Get Time = 27.515 secs (Per element 275.150055 microsecs) [Including Data generation]
00000024 81.46327209 RT_DebugF: ALL Checks Done
00000025 81.46329498 RT_DebugF:
00000026 81.46350861 RT_DebugF: Type = 0 (BOOL)
00000027 81.46367645 RT_DebugF: Dimensions = 1
00000028 81.46383667 RT_DebugF: Nels = 200000 (should have doubled)
00000029 81.46400452 RT_DebugF: ElSize = 1
00000030 81.46424103 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000031 81.46443939 RT_DebugF: Total Time Type=0 (BOOL) = 81.468 secs
00000032 81.46452332 RT_DebugF: ==============================
00000033 81.46635437 RT_DebugF: Type = 1 (INT)
00000034 81.46652985 RT_DebugF: Dimensions = 1
00000035 81.46669769 RT_DebugF: Nels = 100000
00000036 81.46685791 RT_DebugF: ElSize = 4
00000037 81.46710205 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000038 81.46719360 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000039 81.46727753 RT_DebugF:
00000040 81.46730042 RT_DebugF: Check Init
00000041 89.63143921 RT_DebugF: Init END Check Passed
00000042 89.63159180 RT_DebugF: Check Init Time = 8.172 secs (Per element 81.719971 microsecs)
00000043 89.63167572 RT_DebugF: Check Attributes Set/Get
00000044 89.63928223 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000045 89.63937378 RT_DebugF: Check Set only Time Test on 100000 Nels
00000046 101.12193298 RT_DebugF: Check Set Only Time = 11.485 secs (Per element 114.850014 microsecs)
00000047 101.12201691 RT_DebugF: Check Get only Time Test on 100000 Nels
00000048 109.27951813 RT_DebugF: Check Get Only Time = 8.156 secs (Per element 81.559982 microsecs)
00000049 109.27960205 RT_DebugF: Check Compare Set/Get
00000050 129.26173401 RT_DebugF: Check Compare Set/Get Time = 19.984 secs (Per element 199.840027 microsecs) [Including Data generation]
00000051 129.26181030 RT_DebugF: Check Compare Append/Get
00000052 150.20997620 RT_DebugF: Check Compare Append/Get Time = 20.953 secs (Per element 209.530014 microsecs) [Including Data generation]
00000053 150.21005249 RT_DebugF: ALL Checks Done
00000054 150.21008301 RT_DebugF:
00000055 150.21029663 RT_DebugF: Type = 1 (INT)
00000056 150.21044922 RT_DebugF: Dimensions = 1
00000057 150.21061707 RT_DebugF: Nels = 200000 (should have doubled)
00000058 150.21078491 RT_DebugF: ElSize = 4
00000059 150.21101379 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000060 150.21121216 RT_DebugF: Total Time Type=1 (INT) = 68.750 secs
00000061 150.21128845 RT_DebugF: ==============================
00000062 150.21311951 RT_DebugF: Type = 2 (FLOAT)
00000063 150.21328735 RT_DebugF: Dimensions = 1
00000064 150.21343994 RT_DebugF: Nels = 100000
00000065 150.21360779 RT_DebugF: ElSize = 4
00000066 150.21386719 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000067 150.21395874 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000068 150.21403503 RT_DebugF:
00000069 150.21406555 RT_DebugF: Check Init
00000070 158.67863464 RT_DebugF: Init END Check Passed
00000071 158.67880249 RT_DebugF: Check Init Time = 8.469 secs (Per element 84.689941 microsecs)
00000072 158.67887878 RT_DebugF: Check Attributes Set/Get
00000073 158.68649292 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.160065 microsecs)
00000074 158.68656921 RT_DebugF: Check Set only Time Test on 100000 Nels
00000075 170.10278320 RT_DebugF: Check Set Only Time = 11.406 secs (Per element 114.059914 microsecs)
00000076 170.10287476 RT_DebugF: Check Get only Time Test on 100000 Nels
00000077 178.25221252 RT_DebugF: Check Get Only Time = 8.156 secs (Per element 81.560059 microsecs)
00000078 178.25228882 RT_DebugF: Check Compare Set/Get
00000079 226.23529053 RT_DebugF: Check Compare Set/Get Time = 47.985 secs (Per element 479.850006 microsecs) [Including Data generation]
00000080 226.23538208 RT_DebugF: Check Compare Append/Get
00000081 275.02413940 RT_DebugF: Check Compare Append/Get Time = 48.796 secs (Per element 487.959869 microsecs) [Including Data generation]
00000082 275.02423096 RT_DebugF: ALL Checks Done
00000083 275.02423096 RT_DebugF:
00000084 275.02444458 RT_DebugF: Type = 2 (FLOAT)
00000085 275.02462769 RT_DebugF: Dimensions = 1
00000086 275.02478027 RT_DebugF: Nels = 200000 (should have doubled)
00000087 275.02493286 RT_DebugF: ElSize = 4
00000088 275.02517700 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000089 275.02536011 RT_DebugF: Total Time Type=2 (FLOAT) = 124.828 secs
00000090 275.02545166 RT_DebugF: ==============================
00000091 275.04098511 RT_DebugF: Type = 3 (STRING)
00000092 275.04116821 RT_DebugF: Dimensions = 1
00000093 275.04132080 RT_DebugF: Nels = 100000
00000094 275.04150391 RT_DebugF: ElSize = 64
00000095 275.04174805 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000096 275.04183960 RT_DebugF: Alloc Time = 0.016 secs (Per element 0.160217 microsecs)
00000097 275.04193115 RT_DebugF:
00000098 275.04193115 RT_DebugF: Check Init
00000099 283.48788452 RT_DebugF: Init END Check Passed
00000100 283.48803711 RT_DebugF: Check Init Time = 8.438 secs (Per element 84.379883 microsecs)
00000101 283.48812866 RT_DebugF: Check Attributes Set/Get
00000102 283.49581909 RT_DebugF: Check Attributes Set/Get = 0.015 secs (Per element 0.150146 microsecs)
00000103 283.49588013 RT_DebugF: Check Set only Time Test on 100000 Nels
00000104 295.08233643 RT_DebugF: Check Set Only Time = 11.578 secs (Per element 115.779716 microsecs)
00000105 295.08242798 RT_DebugF: Check Get only Time Test on 100000 Nels
00000106 303.34625244 RT_DebugF: Check Get Only Time = 8.266 secs (Per element 82.660217 microsecs)
00000107 303.34634399 RT_DebugF: Check Compare Set/Get
00000108 351.96871948 RT_DebugF: Check Compare Set/Get Time = 48.625 secs (Per element 486.250000 microsecs) [Including Data generation]
00000109 351.96878052 RT_DebugF: Check Compare Append/Get
00000110 401.32009888 RT_DebugF: Check Compare Append/Get Time = 49.359 secs (Per element 493.589813 microsecs) [Including Data generation]
00000111 401.32015991 RT_DebugF: ALL Checks Done
00000112 401.32019043 RT_DebugF:
00000113 401.32040405 RT_DebugF: Type = 3 (STRING)
00000114 401.32058716 RT_DebugF: Dimensions = 1
00000115 401.32073975 RT_DebugF: Nels = 200000 (should have doubled)
00000116 401.32089233 RT_DebugF: ElSize = 64
00000117 401.32113647 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000118 401.32131958 RT_DebugF: Total Time Type=3 (STRING) = 126.297 secs
00000119 401.32141113 RT_DebugF: ==============================
00000120 401.32302856 RT_DebugF: Type = 4 (BIN)
00000121 401.32318115 RT_DebugF: Dimensions = 1
00000122 401.32336426 RT_DebugF: Nels = 100000
00000123 401.32351685 RT_DebugF: ElSize = 1
00000124 401.32379150 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000125 401.32388306 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000126 401.32400513 RT_DebugF:
00000127 401.32403564 RT_DebugF: Check Init
00000128 409.84115601 RT_DebugF: Init END Check Passed
00000129 409.84130859 RT_DebugF: Check Init Time = 8.516 secs (Per element 85.160217 microsecs)
00000130 409.84136963 RT_DebugF: Check Attributes Set/Get
00000131 409.84899902 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.159912 microsecs)
00000132 409.84909058 RT_DebugF: Check Set only Time Test on 100000 Nels
00000133 421.16717529 RT_DebugF: Check Set Only Time = 11.312 secs (Per element 113.120117 microsecs)
00000134 421.16723633 RT_DebugF: Check Get only Time Test on 100000 Nels
00000135 429.34780884 RT_DebugF: Check Get Only Time = 8.188 secs (Per element 81.879883 microsecs)
00000136 429.34786987 RT_DebugF: Check Compare Set/Get
00000137 456.60794067 RT_DebugF: Check Compare Set/Get Time = 27.250 secs (Per element 272.500000 microsecs) [Including Data generation]
00000138 456.60800171 RT_DebugF: Check Compare Append/Get
00000139 484.82464600 RT_DebugF: Check Compare Append/Get Time = 28.218 secs (Per element 282.179871 microsecs) [Including Data generation]
00000140 484.82470703 RT_DebugF: ALL Checks Done
00000141 484.82473755 RT_DebugF:
00000142 484.82495117 RT_DebugF: Type = 4 (BIN)
00000143 484.82510376 RT_DebugF: Dimensions = 1
00000144 484.82528687 RT_DebugF: Nels = 200000 (should have doubled)
00000145 484.82543945 RT_DebugF: ElSize = 1
00000146 484.82568359 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000147 484.82586670 RT_DebugF: Total Time Type=4 (BIN) = 83.500 secs
00000148 484.82611084 RT_DebugF: SCRIPT TOTAL TIME = 484.843 secs

[StainlessS]

Post #4 of 7

RT_Test_Array_2_Dim.Avs

# RT_Test_Array_2_Dim.Avs, 2 Dimensional array

/*
RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,,int "Dim3"=0,int "StringLenMax"=256)
RT_ArrayGetDim(string FileName,int "Dim"=0)
RT_ArrayGetType(string FileName)
RT_ArrayGetNels(string FileName)
RT_ArrayGetElSize(string FileName)
RT_ArrayGetNelMax(string FileName)
RT_ArrayGet(string FileName,int ix1,int "ix2",int "ix3")
RT_ArraySet(string FileName, var ,int ix1,int "ix2",int "ix3") # Var position in arg list changed
RT_ArrayAppend(string FileName, var)
RT_ArrayGetAttrib(string FileName,int ix)
RT_ArraySetAttrib(string FileName,int ix, value) # Value can be Int or Float.
*/

ARRAY="TestArray.ARR" # ARRAY name
START_TYPE = 0 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int)
END_TYPE = 4 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int) ## >= START_TYPE ##
DIM1 = 10000 # Dim 1, Inital Number of Array elements
DIM2 = 10 # Dim 2
STRINGLENMAX = 64 # Excluding Nul Term.
DELETE = True # True = Delete ARRAY file when done

ALPHA = "ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ"
ALPHA=ALPHA+ALPHA # Len 260
ALPHA=(STRINGLENMAX<STRLEN(ALPHA)) ? LeftStr(ALPHA,STRINGLENMAX) : ALPHA # cut down string manips and usage

TYPENAMES=RT_String("BOOL\nINT\nFLOAT\nSTRING\nBIN\n")
NELS = DIM1 * DIM2

GScript("""
RT_DebugF("RT_Test_Array_2_Dim.avs - Two Dimension Array")
TOTAL_START=RT_Timer()
RT_DebugF("")
For(TYPE=START_TYPE,END_TYPE) {
RT_DebugF("==============================")
TYPE_START=RT_Timer()
RT_ArrayAlloc(ARRAY,Type=TYPE,Dim1=DIM1,Dim2=DIM2,stringlenmax=STRINGLENMAX)
STOP=RT_Timer()
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Dimensions = %d",RT_ArrayGetDim(ARRAY))
RT_DebugF("Dim1 = %d",RT_ArrayGetDim(ARRAY,1))
RT_DebugF("Dim2 = %d",RT_ArrayGetDim(ARRAY,2))
RT_DebugF("Nels = %d",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
RT_DebugF("Alloc Time = %.3f secs (Per element %f microsecs)",STOP-TYPE_START,1000000.0*(STOP-TYPE_START)/NELS)
RT_DebugF("\nCheck Init")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
if(RT_ArrayGet(ARRAY,i,j)) {RT_DebugF("BOOL(%d,%d) Init != False",i,j)}
}
}
} Else if(TYPE==1) { # Int
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
if(RT_ArrayGet(ARRAY,i,j) != 0) {RT_DebugF("INT(%d,%d) Init != 0",i,j)}
}
}
} Else if(TYPE==2) { # Float
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
if(RT_ArrayGet(ARRAY,i,j) != 0.0) {RT_DebugF("FLOAT(%d,%d) Init != 0.0",i,j)}
}
}
} Else if(TYPE==3) { # String
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
if(RT_ArrayGet(ARRAY,i,j) != ""){RT_DebugF("STRING(%d,%d) Init != %c%c",i,j,34,34)}
}
}
} Else if(TYPE==4) { # BIN
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
if(RT_ArrayGet(ARRAY,i,j) != 0) {RT_DebugF("BIN(%d,%d) Init != 0",i,j)}
}
}
}
Try {RT_ArrayGet(ARRAY,DIM1-1,DIM2) RT_DebugF("Init END Check1 FAILED")} catch (err) {RT_DebugF("Init END Check1 Passed")}
Try {RT_ArrayGet(ARRAY,DIM1,0) RT_DebugF("Init END Check2 FAILED")} catch (err) {RT_DebugF("Init END Check2 Passed")}
STOP=RT_Timer()
RT_DebugF("Check Init Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Attributes Set/Get")
START=RT_Timer()
for(i=0,15) {
vs= (i % 2 == 0) ? i*1000 : i * PI # Test Int and Float Attributes
RT_ArraySetAttrib(ARRAY,i,vs)
vg = RT_ArrayGetAttrib(ARRAY,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d SetAttrib/GetAttrib Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Attributes Set/Get = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Set only Time Test on %d Nels",NELS)
vs=Select(TYPE,True,42,42.0,RightStr(ALPHA + "_" + String(42),STRINGLENMAX) ,42)
START=RT_Timer()
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
RT_ArraySet(ARRAY,vs,i,j)
}
}
STOP=RT_Timer()
RT_DebugF("Check Set Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Get only Time Test on %d Nels",NELS)
START=RT_Timer()
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
RT_ArrayGet(ARRAY,i,j)
}
}
STOP=RT_Timer()
RT_DebugF("Check Get Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Compare Set/Get")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
vs = RT_BitTST(i*DIM2+j,0) RT_ArraySet(ARRAY,vs,i,j) vg=RT_ArrayGet(ARRAY,i,j)
if(vs != vg) {
RT_DebugF("%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,TYPE,String(vs),String(vg))
}
}
}
} Else if(TYPE==1) { # Int
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
vs = i*DIM2+j RT_ArraySet(ARRAY,vs,i,j) vg=RT_ArrayGet(ARRAY,i,j)
if(vs != vg) {
RT_DebugF("%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,TYPE,String(vs),String(vg))
}
}
}
} Else if(TYPE==2) { # Float
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
vs = Float(i*DIM2+j) RT_ArraySet(ARRAY,vs,i,j) vg=RT_ArrayGet(ARRAY,i,j)
if(vs != vg) {
RT_DebugF("%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,TYPE,String(vs),String(vg))
}
}
}
} Else if(TYPE==3) { # String
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
vs=RightStr(ALPHA + "_" + String(i*DIM2+j),STRINGLENMAX) RT_ArraySet(ARRAY,vs,i,j) vg=RT_ArrayGet(ARRAY,i,j)
if(vs != vg) {
RT_DebugF("%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,TYPE,String(vs),String(vg))
}
}
}
} Else if(TYPE==4) { # BIN
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
vs = RT_BitAnd(i*DIM2+j,$FF) RT_ArraySet(ARRAY,vs,i,j) vg=RT_ArrayGet(ARRAY,i,j)
if(vs != vg) {
RT_DebugF("%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,TYPE,String(vs),String(vg))
}
}
}
}
STOP=RT_Timer()
RT_DebugF("Check Compare Set/Get Time = %.3f secs (Per element %f microsecs) [Including Data generation]",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("ALL Checks Done\n")
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Dimensions = %d",RT_ArrayGetDim(ARRAY))
RT_DebugF("Dim1 = %d",RT_ArrayGetDim(ARRAY,1))
RT_DebugF("Dim2 = %d",RT_ArrayGetDim(ARRAY,2))
RT_DebugF("Nels = %d (Not doubled, Cannot append to multi-dim array)",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
STOP=RT_Timer()
RT_DebugF("Total Time Type=%d (%s) = %.3f secs",Type,RT_TxtGetLine(TYPENAMES,Type),STOP-TYPE_START)
}
(DELETE) ? RT_FileDelete(ARRAY) : NOP
TOTAL_STOP=RT_Timer()
RT_DebugF("SCRIPT TOTAL TIME = %.3f secs",TOTAL_STOP-TOTAL_START)
""")
return colorbars()

[StainlessS]

Post #5 of 7

RT_Test_Array_2_Dim.Log

00000001 0.00000000 RT_DebugF: RT_Test_Array_2_Dim.avs - Two Dimension Array
00000002 0.00013633 RT_DebugF:
00000003 0.00022618 RT_DebugF: ==============================
00000004 0.00136300 RT_DebugF: Type = 0 (BOOL)
00000005 0.00152955 RT_DebugF: Dimensions = 2
00000006 0.00169800 RT_DebugF: Dim1 = 10000
00000007 0.00189691 RT_DebugF: Dim2 = 10
00000008 0.00206046 RT_DebugF: Nels = 100000
00000009 0.00222277 RT_DebugF: ElSize = 1
00000010 0.00246191 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000011 0.00255518 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000012 0.00263816 RT_DebugF:
00000013 0.00266201 RT_DebugF: Check Init
00000014 8.12602615 RT_DebugF: Init END Check1 Passed
00000015 8.12623787 RT_DebugF: Init END Check2 Passed
00000016 8.12639523 RT_DebugF: Check Init Time = 8.125 secs (Per element 81.250000 microsecs)
00000017 8.12647915 RT_DebugF: Check Attributes Set/Get
00000018 8.13418102 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000019 8.13426781 RT_DebugF: Check Set only Time Test on 100000 Nels
00000020 20.33507538 RT_DebugF: Check Set Only Time = 12.203 secs (Per element 122.029991 microsecs)
00000021 20.33515930 RT_DebugF: Check Get only Time Test on 100000 Nels
00000022 28.66847420 RT_DebugF: Check Get Only Time = 8.343 secs (Per element 83.430008 microsecs)
00000023 28.66855431 RT_DebugF: Check Compare Set/Get
00000024 57.08416367 RT_DebugF: Check Compare Set/Get Time = 28.407 secs (Per element 284.070007 microsecs) [Including Data generation]
00000025 57.08424377 RT_DebugF: ALL Checks Done
00000026 57.08427048 RT_DebugF:
00000027 57.08448792 RT_DebugF: Type = 0 (BOOL)
00000028 57.08465195 RT_DebugF: Dimensions = 2
00000029 57.08481979 RT_DebugF: Dim1 = 10000
00000030 57.08498383 RT_DebugF: Dim2 = 10
00000031 57.08514786 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000032 57.08531570 RT_DebugF: ElSize = 1
00000033 57.08555603 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000034 57.08574295 RT_DebugF: Total Time Type=0 (BOOL) = 57.078 secs
00000035 57.08583450 RT_DebugF: ==============================
00000036 57.08751297 RT_DebugF: Type = 1 (INT)
00000037 57.08768082 RT_DebugF: Dimensions = 2
00000038 57.08785248 RT_DebugF: Dim1 = 10000
00000039 57.08801651 RT_DebugF: Dim2 = 10
00000040 57.08818817 RT_DebugF: Nels = 100000
00000041 57.08837891 RT_DebugF: ElSize = 4
00000042 57.08861923 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000043 57.08871460 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000044 57.08880234 RT_DebugF:
00000045 57.08882523 RT_DebugF: Check Init
00000046 65.49828339 RT_DebugF: Init END Check1 Passed
00000047 65.49848938 RT_DebugF: Init END Check2 Passed
00000048 65.49865723 RT_DebugF: Check Init Time = 8.407 secs (Per element 84.070007 microsecs)
00000049 65.49874115 RT_DebugF: Check Attributes Set/Get
00000050 65.50650787 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000051 65.50659180 RT_DebugF: Check Set only Time Test on 100000 Nels
00000052 77.87017822 RT_DebugF: Check Set Only Time = 12.375 secs (Per element 123.750000 microsecs)
00000053 77.87026215 RT_DebugF: Check Get only Time Test on 100000 Nels
00000054 86.22363281 RT_DebugF: Check Get Only Time = 8.343 secs (Per element 83.430023 microsecs)
00000055 86.22371674 RT_DebugF: Check Compare Set/Get
00000056 107.68625641 RT_DebugF: Check Compare Set/Get Time = 21.469 secs (Per element 214.689941 microsecs) [Including Data generation]
00000057 107.68633270 RT_DebugF: ALL Checks Done
00000058 107.68635559 RT_DebugF:
00000059 107.68656921 RT_DebugF: Type = 1 (INT)
00000060 107.68673706 RT_DebugF: Dimensions = 2
00000061 107.68689728 RT_DebugF: Dim1 = 10000
00000062 107.68707275 RT_DebugF: Dim2 = 10
00000063 107.68723297 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000064 107.68740082 RT_DebugF: ElSize = 4
00000065 107.68763733 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000066 107.68782806 RT_DebugF: Total Time Type=1 (INT) = 50.609 secs
00000067 107.68791962 RT_DebugF: ==============================
00000068 107.68959045 RT_DebugF: Type = 2 (FLOAT)
00000069 107.68975830 RT_DebugF: Dimensions = 2
00000070 107.68992615 RT_DebugF: Dim1 = 10000
00000071 107.69011688 RT_DebugF: Dim2 = 10
00000072 107.69028473 RT_DebugF: Nels = 100000
00000073 107.69044495 RT_DebugF: ElSize = 4
00000074 107.69068909 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000075 107.69078064 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000076 107.69086456 RT_DebugF:
00000077 107.69088745 RT_DebugF: Check Init
00000078 116.36470032 RT_DebugF: Init END Check1 Passed
00000079 116.36490631 RT_DebugF: Init END Check2 Passed
00000080 116.36506653 RT_DebugF: Check Init Time = 8.672 secs (Per element 86.720047 microsecs)
00000081 116.36515045 RT_DebugF: Check Attributes Set/Get
00000082 116.37288666 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.159988 microsecs)
00000083 116.37297058 RT_DebugF: Check Set only Time Test on 100000 Nels
00000084 128.71505737 RT_DebugF: Check Set Only Time = 12.343 secs (Per element 123.430023 microsecs)
00000085 128.71514893 RT_DebugF: Check Get only Time Test on 100000 Nels
00000086 137.08207703 RT_DebugF: Check Get Only Time = 8.360 secs (Per element 83.600006 microsecs)
00000087 137.08215332 RT_DebugF: Check Compare Set/Get
00000088 186.00663757 RT_DebugF: Check Compare Set/Get Time = 48.937 secs (Per element 489.369965 microsecs) [Including Data generation]
00000089 186.00671387 RT_DebugF: ALL Checks Done
00000090 186.00672913 RT_DebugF:
00000091 186.00694275 RT_DebugF: Type = 2 (FLOAT)
00000092 186.00711060 RT_DebugF: Dimensions = 2
00000093 186.00727844 RT_DebugF: Dim1 = 10000
00000094 186.00744629 RT_DebugF: Dim2 = 10
00000095 186.00761414 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000096 186.00776672 RT_DebugF: ElSize = 4
00000097 186.00801086 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000098 186.00819397 RT_DebugF: Total Time Type=2 (FLOAT) = 78.328 secs
00000099 186.00828552 RT_DebugF: ==============================
00000100 186.02537537 RT_DebugF: Type = 3 (STRING)
00000101 186.02552795 RT_DebugF: Dimensions = 2
00000102 186.02571106 RT_DebugF: Dim1 = 10000
00000103 186.02586365 RT_DebugF: Dim2 = 10
00000104 186.02603149 RT_DebugF: Nels = 100000
00000105 186.02619934 RT_DebugF: ElSize = 64
00000106 186.02644348 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000107 186.02655029 RT_DebugF: Alloc Time = 0.016 secs (Per element 0.160065 microsecs)
00000108 186.02664185 RT_DebugF:
00000109 186.02667236 RT_DebugF: Check Init
00000110 194.68681335 RT_DebugF: Init END Check1 Passed
00000111 194.68702698 RT_DebugF: Init END Check2 Passed
00000112 194.68717957 RT_DebugF: Check Init Time = 8.656 secs (Per element 86.559914 microsecs)
00000113 194.68727112 RT_DebugF: Check Attributes Set/Get
00000114 194.69503784 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.160065 microsecs)
00000115 194.69512939 RT_DebugF: Check Set only Time Test on 100000 Nels
00000116 207.24955750 RT_DebugF: Check Set Only Time = 12.547 secs (Per element 125.469971 microsecs)
00000117 207.24963379 RT_DebugF: Check Get only Time Test on 100000 Nels
00000118 215.72695923 RT_DebugF: Check Get Only Time = 8.484 secs (Per element 84.839943 microsecs)
00000119 215.72705078 RT_DebugF: Check Compare Set/Get
00000120 265.41159058 RT_DebugF: Check Compare Set/Get Time = 49.687 secs (Per element 496.869965 microsecs) [Including Data generation]
00000121 265.41171265 RT_DebugF: ALL Checks Done
00000122 265.41174316 RT_DebugF:
00000123 265.41195679 RT_DebugF: Type = 3 (STRING)
00000124 265.41210938 RT_DebugF: Dimensions = 2
00000125 265.41226196 RT_DebugF: Dim1 = 10000
00000126 265.41244507 RT_DebugF: Dim2 = 10
00000127 265.41259766 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000128 265.41278076 RT_DebugF: ElSize = 64
00000129 265.41302490 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000130 265.41320801 RT_DebugF: Total Time Type=3 (STRING) = 79.406 secs
00000131 265.41329956 RT_DebugF: ==============================
00000132 265.41476440 RT_DebugF: Type = 4 (BIN)
00000133 265.41491699 RT_DebugF: Dimensions = 2
00000134 265.41510010 RT_DebugF: Dim1 = 10000
00000135 265.41528320 RT_DebugF: Dim2 = 10
00000136 265.41546631 RT_DebugF: Nels = 100000
00000137 265.41567993 RT_DebugF: ElSize = 1
00000138 265.41592407 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000139 265.41601563 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000140 265.41610718 RT_DebugF:
00000141 265.41613770 RT_DebugF: Check Init
00000142 274.11859131 RT_DebugF: Init END Check1 Passed
00000143 274.11880493 RT_DebugF: Init END Check2 Passed
00000144 274.11895752 RT_DebugF: Check Init Time = 8.704 secs (Per element 87.040100 microsecs)
00000145 274.11901855 RT_DebugF: Check Attributes Set/Get
00000146 274.12680054 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000147 274.12689209 RT_DebugF: Check Set only Time Test on 100000 Nels
00000148 286.32312012 RT_DebugF: Check Set Only Time = 12.188 secs (Per element 121.879883 microsecs)
00000149 286.32318115 RT_DebugF: Check Get only Time Test on 100000 Nels
00000150 294.69894409 RT_DebugF: Check Get Only Time = 8.375 secs (Per element 83.750000 microsecs)
00000151 294.69900513 RT_DebugF: Check Compare Set/Get
00000152 323.51010132 RT_DebugF: Check Compare Set/Get Time = 28.812 secs (Per element 288.120117 microsecs) [Including Data generation]
00000153 323.51019287 RT_DebugF: ALL Checks Done
00000154 323.51022339 RT_DebugF:
00000155 323.51043701 RT_DebugF: Type = 4 (BIN)
00000156 323.51058960 RT_DebugF: Dimensions = 2
00000157 323.51074219 RT_DebugF: Dim1 = 10000
00000158 323.51092529 RT_DebugF: Dim2 = 10
00000159 323.51107788 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000160 323.51126099 RT_DebugF: ElSize = 1
00000161 323.51150513 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000162 323.51168823 RT_DebugF: Total Time Type=4 (BIN) = 58.094 secs
00000163 323.51190186 RT_DebugF: SCRIPT TOTAL TIME = 323.515 secs

[StainlessS]

Post #6 of 7

RT_Test_Array_3_Dim.Avs

# RT_Test_Array_3_Dim.Avs, 3 Dimensional array

/*
RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,,int "Dim3"=0,int "StringLenMax"=256)
RT_ArrayGetDim(string FileName,int "Dim"=0)
RT_ArrayGetType(string FileName)
RT_ArrayGetNels(string FileName)
RT_ArrayGetElSize(string FileName)
RT_ArrayGetNelMax(string FileName)
RT_ArrayGet(string FileName,int ix1,int "ix2",int "ix3")
RT_ArraySet(string FileName, var ,int ix1,int "ix2",int "ix3") # Var position in arg list changed
RT_ArrayAppend(string FileName, var)
RT_ArrayGetAttrib(string FileName,int ix)
RT_ArraySetAttrib(string FileName,int ix, value) # Value can be Int or Float.
*/

ARRAY="TestArray.ARR" # ARRAY name
START_TYPE = 0 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int)
END_TYPE = 4 # 0=Bool,1=Int,2=Float,3=String,4=BIN(BYTE sized int) ## >= START_TYPE ##
DIM1 = 500 # Dim 1, Inital Number of Array elements
DIM2 = 20 # Dim 2
DIM3 = 10 # Dim 3
STRINGLENMAX = 64 # Excluding Nul Term.
DELETE = True # True = Delete ARRAY file when done

ALPHA = "ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ"
ALPHA=ALPHA+ALPHA # Len 260
ALPHA=(STRINGLENMAX<STRLEN(ALPHA)) ? LeftStr(ALPHA,STRINGLENMAX) : ALPHA # cut down string manips and usage

TYPENAMES=RT_String("BOOL\nINT\nFLOAT\nSTRING\nBIN\n")
NELS = DIM1 * DIM2 * DIM3

GScript("""
RT_DebugF("RT_Test_Array_3_Dim.avs - Three Dimension Array")
TOTAL_START=RT_Timer()
RT_DebugF("")
For(TYPE=START_TYPE,END_TYPE) {
RT_DebugF("==============================")
TYPE_START=RT_Timer()
RT_ArrayAlloc(ARRAY,Type=TYPE,Dim1=DIM1,Dim2=DIM2,Dim3=DIM3,stringlenmax=STRINGLENMAX)
STOP=RT_Timer()
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Dimensions = %d",RT_ArrayGetDim(ARRAY))
RT_DebugF("Dim1 = %d",RT_ArrayGetDim(ARRAY,1))
RT_DebugF("Dim2 = %d",RT_ArrayGetDim(ARRAY,2))
RT_DebugF("Dim3 = %d",RT_ArrayGetDim(ARRAY,3))
RT_DebugF("Nels = %d",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
RT_DebugF("Alloc Time = %.3f secs (Per element %f microsecs)",STOP-TYPE_START,1000000.0*(STOP-TYPE_START)/NELS)
RT_DebugF("\nCheck Init")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
if(RT_ArrayGet(ARRAY,i,j,k)) {RT_DebugF("BOOL(%d,%d,%d) Init != False",i,j,k)}
}
}
}
} Else if(TYPE==1) { # Int
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
if(RT_ArrayGet(ARRAY,i,j,k) != 0) {RT_DebugF("INT(%d,%d,%d) Init != 0",i,j,k)}
}
}
}
} Else if(TYPE==2) { # Float
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
if(RT_ArrayGet(ARRAY,i,j,k) != 0.0) {RT_DebugF("FLOAT(%d,%d,%d) Init != 0.0",i,j,k)}
}
}
}
} Else if(TYPE==3) { # String
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
if(RT_ArrayGet(ARRAY,i,j,k) != ""){RT_DebugF("STRING(%d,%d,%d) Init != %c%c",i,j,k,34,34)}
}
}
}
} Else if(TYPE==4) { # BIN
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
if(RT_ArrayGet(ARRAY,i,j,k) != 0) {RT_DebugF("BIN(%d,%d,%d) Init != 0",i,j,k)}
}
}
}
}
Try {RT_ArrayGet(ARRAY,DIM1-1,DIM2-1,DIM3) RT_DebugF("Init END Check1 FAILED")} catch (err) {RT_DebugF("Init END Check1 Passed")}
Try {RT_ArrayGet(ARRAY,DIM1-1,DIM2,0) RT_DebugF("Init END Check2 FAILED")} catch (err) {RT_DebugF("Init END Check2 Passed")}
Try {RT_ArrayGet(ARRAY,DIM1,0,0) RT_DebugF("Init END Check3 FAILED")} catch (err) {RT_DebugF("Init END Check3 Passed")}
STOP=RT_Timer()
RT_DebugF("Check Init Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Attributes Set/Get")
START=RT_Timer()
for(i=0,15) {
vs= (i % 2 == 0) ? i*1000 : i * PI # Test Int and Float Attributes
RT_ArraySetAttrib(ARRAY,i,vs)
vg = RT_ArrayGetAttrib(ARRAY,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d SetAttrib/GetAttrib Error, Set = %s Get = %s",i,TYPE,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Attributes Set/Get = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Set only Time Test on %d Nels",NELS)
vs=Select(TYPE,True,42,42.0,RightStr(ALPHA + "_" + String(42),STRINGLENMAX) ,42)
START=RT_Timer()
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
RT_ArraySet(ARRAY,vs,i,j,k)
}
}
}
STOP=RT_Timer()
RT_DebugF("Check Set Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Get only Time Test on %d Nels",NELS)
START=RT_Timer()
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
RT_ArrayGet(ARRAY,i,j,k)
}
}
}
STOP=RT_Timer()
RT_DebugF("Check Get Only Time = %.3f secs (Per element %f microsecs)",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("Check Compare Set/Get")
START=RT_Timer()
if(TYPE==0) { # Bool
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
vs = RT_BitTST((i*DIM2+j)*DIM3+k,0) RT_ArraySet(ARRAY,vs,i,j,k) vg=RT_ArrayGet(ARRAY,i,j,k)
if(vs != vg) {
RT_DebugF("%d,%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,k,TYPE,String(vs),String(vg))
}
}
}
}
} Else if(TYPE==1) { # Int
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
vs = (i*DIM2+j)*DIM3+k RT_ArraySet(ARRAY,vs,i,j,k) vg=RT_ArrayGet(ARRAY,i,j,k)
if(vs != vg) {
RT_DebugF("%d,%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,k,TYPE,String(vs),String(vg))
}
}
}
}
} Else if(TYPE==2) { # Float
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
vs = Float((i*DIM2+j)*DIM3+k) RT_ArraySet(ARRAY,vs,i,j,k) vg=RT_ArrayGet(ARRAY,i,j,k)
if(vs != vg) {
RT_DebugF("%d,%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,k,TYPE,String(vs),String(vg))
}
}
}
}
} Else if(TYPE==3) { # String
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
vs=RightStr(ALPHA + "_" + String((i*DIM2+j)*DIM3+k),STRINGLENMAX) RT_ArraySet(ARRAY,vs,i,j,k) vg=RT_ArrayGet(ARRAY,i,j,k)
if(vs != vg) {
RT_DebugF("%d,%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,k,TYPE,String(vs),String(vg))
}
}
}
}
} Else if(TYPE==4) { # BIN
for(i=0,DIM1-1) {
for(j=0,DIM2-1) {
for(k=0,DIM3-1) {
vs = RT_BitAnd((i*DIM2+j)*DIM3+k,$FF) RT_ArraySet(ARRAY,vs,i,j,k) vg=RT_ArrayGet(ARRAY,i,j,k)
if(vs != vg) {
RT_DebugF("%d,%d,%d) TYPE=%d Set/Get Error, Set = %s Get = %s",i,j,k,TYPE,String(vs),String(vg))
}
}
}
}
}
STOP=RT_Timer()
RT_DebugF("Check Compare Set/Get Time = %.3f secs (Per element %f microsecs) [Including Data generation]",STOP-START,1000000.0*(STOP-START)/NELS)
RT_DebugF("ALL Checks Done\n")
RT_DebugF("Type = %d (%s)",RT_ArrayGetType(ARRAY),RT_TxtGetLine(TYPENAMES,Type))
RT_DebugF("Dimensions = %d",RT_ArrayGetDim(ARRAY))
RT_DebugF("Dim1 = %d",RT_ArrayGetDim(ARRAY,1))
RT_DebugF("Dim2 = %d",RT_ArrayGetDim(ARRAY,2))
RT_DebugF("Dim3 = %d",RT_ArrayGetDim(ARRAY,3))
RT_DebugF("Nels = %d (Not doubled, Cannot append to multi-dim array)",RT_ArrayGetNels(ARRAY))
RT_DebugF("ElSize = %d",RT_ArrayGetElSize(ARRAY))
RT_DebugF("NelMax = %d ($%X)",RT_ArrayGetNelMax(ARRAY),RT_ArrayGetNelMax(ARRAY))
STOP=RT_Timer()
RT_DebugF("Total Time Type=%d (%s) = %.3f secs",Type,RT_TxtGetLine(TYPENAMES,Type),STOP-TYPE_START)
}
(DELETE) ? RT_FileDelete(ARRAY) : NOP
TOTAL_STOP=RT_Timer()
RT_DebugF("SCRIPT TOTAL TIME = %.3f secs",TOTAL_STOP-TOTAL_START)
""")
return colorbars()

[StainlessS]

Post #7 of 7

RT_Test_Array_3_Dim.Log

00000001 0.00000000 RT_DebugF: RT_Test_Array_3_Dim.avs - Three Dimension Array
00000002 0.00014211 RT_DebugF:
00000003 0.00023225 RT_DebugF: ==============================
00000004 0.00131004 RT_DebugF: Type = 0 (BOOL)
00000005 0.00147912 RT_DebugF: Dimensions = 3
00000006 0.00164855 RT_DebugF: Dim1 = 500
00000007 0.00181547 RT_DebugF: Dim2 = 20
00000008 0.00198199 RT_DebugF: Dim3 = 10
00000009 0.00218009 RT_DebugF: Nels = 100000
00000010 0.00234457 RT_DebugF: ElSize = 1
00000011 0.00258637 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000012 0.00267946 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000013 0.00276241 RT_DebugF:
00000014 0.00278579 RT_DebugF: Check Init
00000015 8.27883625 RT_DebugF: Init END Check1 Passed
00000016 8.27904987 RT_DebugF: Init END Check2 Passed
00000017 8.27925873 RT_DebugF: Init END Check3 Passed
00000018 8.27941895 RT_DebugF: Check Init Time = 8.281 secs (Per element 82.809998 microsecs)
00000019 8.27950382 RT_DebugF: Check Attributes Set/Get
00000020 8.28724003 RT_DebugF: Check Attributes Set/Get = 0.015 secs (Per element 0.149994 microsecs)
00000021 8.28732872 RT_DebugF: Check Set only Time Test on 100000 Nels
00000022 20.20360184 RT_DebugF: Check Set Only Time = 11.907 secs (Per element 119.070000 microsecs)
00000023 20.20368767 RT_DebugF: Check Get only Time Test on 100000 Nels
00000024 28.68160629 RT_DebugF: Check Get Only Time = 8.484 secs (Per element 84.840012 microsecs)
00000025 28.68168640 RT_DebugF: Check Compare Set/Get
00000026 56.60816193 RT_DebugF: Check Compare Set/Get Time = 27.922 secs (Per element 279.220001 microsecs) [Including Data generation]
00000027 56.60824585 RT_DebugF: ALL Checks Done
00000028 56.60826874 RT_DebugF:
00000029 56.60848236 RT_DebugF: Type = 0 (BOOL)
00000030 56.60865021 RT_DebugF: Dimensions = 3
00000031 56.60882187 RT_DebugF: Dim1 = 500
00000032 56.60898590 RT_DebugF: Dim2 = 20
00000033 56.60914993 RT_DebugF: Dim3 = 10
00000034 56.60932159 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000035 56.60948563 RT_DebugF: ElSize = 1
00000036 56.60972977 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000037 56.60992432 RT_DebugF: Total Time Type=0 (BOOL) = 56.609 secs
00000038 56.61000824 RT_DebugF: ==============================
00000039 56.61172485 RT_DebugF: Type = 1 (INT)
00000040 56.61191177 RT_DebugF: Dimensions = 3
00000041 56.61207962 RT_DebugF: Dim1 = 500
00000042 56.61225128 RT_DebugF: Dim2 = 20
00000043 56.61241913 RT_DebugF: Dim3 = 10
00000044 56.61258316 RT_DebugF: Nels = 100000
00000045 56.61275101 RT_DebugF: ElSize = 4
00000046 56.61299133 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000047 56.61309433 RT_DebugF: Alloc Time = 0.016 secs (Per element 0.159988 microsecs)
00000048 56.61317444 RT_DebugF:
00000049 56.61319733 RT_DebugF: Check Init
00000050 65.18122864 RT_DebugF: Init END Check1 Passed
00000051 65.18144226 RT_DebugF: Init END Check2 Passed
00000052 65.18164825 RT_DebugF: Init END Check3 Passed
00000053 65.18180847 RT_DebugF: Check Init Time = 8.562 secs (Per element 85.619957 microsecs)
00000054 65.18190002 RT_DebugF: Check Attributes Set/Get
00000055 65.18953705 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.160065 microsecs)
00000056 65.18962860 RT_DebugF: Check Set only Time Test on 100000 Nels
00000057 77.02680206 RT_DebugF: Check Set Only Time = 11.828 secs (Per element 118.279953 microsecs)
00000058 77.02688599 RT_DebugF: Check Get only Time Test on 100000 Nels
00000059 85.54342651 RT_DebugF: Check Get Only Time = 8.515 secs (Per element 85.149986 microsecs)
00000060 85.54350281 RT_DebugF: Check Compare Set/Get
00000061 107.05323029 RT_DebugF: Check Compare Set/Get Time = 21.516 secs (Per element 215.159973 microsecs) [Including Data generation]
00000062 107.05330658 RT_DebugF: ALL Checks Done
00000063 107.05332947 RT_DebugF:
00000064 107.05354309 RT_DebugF: Type = 1 (INT)
00000065 107.05371857 RT_DebugF: Dimensions = 3
00000066 107.05387878 RT_DebugF: Dim1 = 500
00000067 107.05404663 RT_DebugF: Dim2 = 20
00000068 107.05421448 RT_DebugF: Dim3 = 10
00000069 107.05437469 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000070 107.05453491 RT_DebugF: ElSize = 4
00000071 107.05478668 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000072 107.05496979 RT_DebugF: Total Time Type=1 (INT) = 50.453 secs
00000073 107.05506134 RT_DebugF: ==============================
00000074 107.05671692 RT_DebugF: Type = 2 (FLOAT)
00000075 107.05688477 RT_DebugF: Dimensions = 3
00000076 107.05705261 RT_DebugF: Dim1 = 500
00000077 107.05722046 RT_DebugF: Dim2 = 20
00000078 107.05738068 RT_DebugF: Dim3 = 10
00000079 107.05754852 RT_DebugF: Nels = 100000
00000080 107.05771637 RT_DebugF: ElSize = 4
00000081 107.05796051 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000082 107.05805969 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000083 107.05814362 RT_DebugF:
00000084 107.05816650 RT_DebugF: Check Init
00000085 115.89083862 RT_DebugF: Init END Check1 Passed
00000086 115.89105225 RT_DebugF: Init END Check2 Passed
00000087 115.89126587 RT_DebugF: Init END Check3 Passed
00000088 115.89143372 RT_DebugF: Check Init Time = 8.844 secs (Per element 88.440018 microsecs)
00000089 115.89151001 RT_DebugF: Check Attributes Set/Get
00000090 115.89917755 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000091 115.89926147 RT_DebugF: Check Set only Time Test on 100000 Nels
00000092 127.76137543 RT_DebugF: Check Set Only Time = 11.859 secs (Per element 118.590012 microsecs)
00000093 127.76145935 RT_DebugF: Check Get only Time Test on 100000 Nels
00000094 136.27801514 RT_DebugF: Check Get Only Time = 8.516 secs (Per element 85.160057 microsecs)
00000095 136.27810669 RT_DebugF: Check Compare Set/Get
00000096 185.36102295 RT_DebugF: Check Compare Set/Get Time = 49.094 secs (Per element 490.939972 microsecs) [Including Data generation]
00000097 185.36109924 RT_DebugF: ALL Checks Done
00000098 185.36112976 RT_DebugF:
00000099 185.36134338 RT_DebugF: Type = 2 (FLOAT)
00000100 185.36149597 RT_DebugF: Dimensions = 3
00000101 185.36167908 RT_DebugF: Dim1 = 500
00000102 185.36184692 RT_DebugF: Dim2 = 20
00000103 185.36201477 RT_DebugF: Dim3 = 10
00000104 185.36218262 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000105 185.36233521 RT_DebugF: ElSize = 4
00000106 185.36257935 RT_DebugF: NelMax = 536870847 ($1FFFFFBF)
00000107 185.36276245 RT_DebugF: Total Time Type=2 (FLOAT) = 78.313 secs
00000108 185.36285400 RT_DebugF: ==============================
00000109 185.38035583 RT_DebugF: Type = 3 (STRING)
00000110 185.38052368 RT_DebugF: Dimensions = 3
00000111 185.38069153 RT_DebugF: Dim1 = 500
00000112 185.38085938 RT_DebugF: Dim2 = 20
00000113 185.38102722 RT_DebugF: Dim3 = 10
00000114 185.38119507 RT_DebugF: Nels = 100000
00000115 185.38136292 RT_DebugF: ElSize = 64
00000116 185.38160706 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000117 185.38171387 RT_DebugF: Alloc Time = 0.015 secs (Per element 0.149994 microsecs)
00000118 185.38179016 RT_DebugF:
00000119 185.38182068 RT_DebugF: Check Init
00000120 194.16348267 RT_DebugF: Init END Check1 Passed
00000121 194.16371155 RT_DebugF: Init END Check2 Passed
00000122 194.16390991 RT_DebugF: Init END Check3 Passed
00000123 194.16407776 RT_DebugF: Check Init Time = 8.781 secs (Per element 87.810059 microsecs)
00000124 194.16415405 RT_DebugF: Check Attributes Set/Get
00000125 194.17181396 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element 0.159912 microsecs)
00000126 194.17190552 RT_DebugF: Check Set only Time Test on 100000 Nels
00000127 206.27981567 RT_DebugF: Check Set Only Time = 12.109 secs (Per element 121.090088 microsecs)
00000128 206.27989197 RT_DebugF: Check Get only Time Test on 100000 Nels
00000129 214.92962646 RT_DebugF: Check Get Only Time = 8.641 secs (Per element 86.409912 microsecs)
00000130 214.92970276 RT_DebugF: Check Compare Set/Get
00000131 264.40823364 RT_DebugF: Check Compare Set/Get Time = 49.484 secs (Per element 494.839905 microsecs) [Including Data generation]
00000132 264.40829468 RT_DebugF: ALL Checks Done
00000133 264.40832520 RT_DebugF:
00000134 264.40853882 RT_DebugF: Type = 3 (STRING)
00000135 264.40869141 RT_DebugF: Dimensions = 3
00000136 264.40887451 RT_DebugF: Dim1 = 500
00000137 264.40902710 RT_DebugF: Dim2 = 20
00000138 264.40921021 RT_DebugF: Dim3 = 10
00000139 264.40936279 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000140 264.40954590 RT_DebugF: ElSize = 64
00000141 264.40975952 RT_DebugF: NelMax = 33554427 ($1FFFFFB)
00000142 264.40997314 RT_DebugF: Total Time Type=3 (STRING) = 79.046 secs
00000143 264.41006470 RT_DebugF: ==============================
00000144 264.41146851 RT_DebugF: Type = 4 (BIN)
00000145 264.41165161 RT_DebugF: Dimensions = 3
00000146 264.41183472 RT_DebugF: Dim1 = 500
00000147 264.41198730 RT_DebugF: Dim2 = 20
00000148 264.41217041 RT_DebugF: Dim3 = 10
00000149 264.41235352 RT_DebugF: Nels = 100000
00000150 264.41250610 RT_DebugF: ElSize = 1
00000151 264.41278076 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000152 264.41287231 RT_DebugF: Alloc Time = 0.000 secs (Per element 0.000000 microsecs)
00000153 264.41296387 RT_DebugF:
00000154 264.41296387 RT_DebugF: Check Init
00000155 273.27609253 RT_DebugF: Init END Check1 Passed
00000156 273.27630615 RT_DebugF: Init END Check2 Passed
00000157 273.27651978 RT_DebugF: Init END Check3 Passed
00000158 273.27667236 RT_DebugF: Check Init Time = 8.875 secs (Per element 88.750000 microsecs)
00000159 273.27676392 RT_DebugF: Check Attributes Set/Get
00000160 273.28442383 RT_DebugF: Check Attributes Set/Get = 0.000 secs (Per element 0.000000 microsecs)
00000161 273.28451538 RT_DebugF: Check Set only Time Test on 100000 Nels
00000162 285.04803467 RT_DebugF: Check Set Only Time = 11.766 secs (Per element 117.660217 microsecs)
00000163 285.04812622 RT_DebugF: Check Get only Time Test on 100000 Nels
00000164 293.58193970 RT_DebugF: Check Get Only Time = 8.531 secs (Per element 85.309753 microsecs)
00000165 293.58203125 RT_DebugF: Check Compare Set/Get
00000166 322.62448120 RT_DebugF: Check Compare Set/Get Time = 29.047 secs (Per element 290.470276 microsecs) [Including Data generation]
00000167 322.62457275 RT_DebugF: ALL Checks Done
00000168 322.62460327 RT_DebugF:
00000169 322.62481689 RT_DebugF: Type = 4 (BIN)
00000170 322.62496948 RT_DebugF: Dimensions = 3
00000171 322.62515259 RT_DebugF: Dim1 = 500
00000172 322.62530518 RT_DebugF: Dim2 = 20
00000173 322.62548828 RT_DebugF: Dim3 = 10
00000174 322.62564087 RT_DebugF: Nels = 100000 (Not doubled, Cannot append to multi-dim array)
00000175 322.62582397 RT_DebugF: ElSize = 1
00000176 322.62606812 RT_DebugF: NelMax = 2147483391 ($7FFFFEFF)
00000177 322.62628174 RT_DebugF: Total Time Type=4 (BIN) = 58.219 secs
00000178 322.62649536 RT_DebugF: SCRIPT TOTAL TIME = 322.640 secs



Results on Core Duo Dual Core 2.13GHz, 3GB DDR2, Half full non-system 320 GB SATA2

[StainlessS]

New version RT_Stats v1.28Beta51 Here:- LINK REMOVED

Changes:-

Changed Beta version in RT_VersionString to show two digits of Beta eg "1.28Beta51".

Several changes to RT_Array type funcs.

*****************************************************
********** RT_ARRAY FUNCTIONS ***********
*****************************************************

RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,int "Dim3"=0,int "StringLenMax"=256)
Creates a file to hold Array elements of type Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in array as an unsigned 8 bit int, upper 24 bits ignored.

Filename, Array Filename, no default.

Type, int, default 1(0 -> 4). 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.

Dim1, Dim2, Dim3, all Default 0, range 0 or greater. Number of array elements per dimension (max 3 dimensions).
It makes little sense to set either Dim2 or Dim3 to 1 although this will not cause an error condition, just be less efficient,
a 3 dimensional array with all dimensions 1, refers only to a single element but but will take slightly longer to calculate the
linear array offset.

Dim2 and Dim3 fix the number of array dimensions for the lifetime of the array, and also the size of dimensions 2 and 3.

Dim2 Dim3
0 0 Single dimension array
0 >0 Illegal, Throws error condition.
>0 0 Two dimension array, Dim2 fixes size of 2nd dimension.
>0 >0 Three dimension array, Dim2 and Dim3 fix size of 2nd and 3rd dimensions.

Dim1 governs whether array elements are allocated or not. If Dim1=0, then no elements of the array are preallocated and you must call
either RT_ArrayExtend or RT_ArrayAppend (single dimension array ONLY) to use the array. The size of the first dimension can grow whether
it is a 1, 2 or 3 dimension array, the size of the 2nd and 3rd dimensions (if any) are fixed.
With all defaults of 0, would be fixed to a single dimension array with zero allocated elements, must use RT_ArrayAppend to add
elements to the array or RT_ArrayExtend to allocate additional elements and initalize them to zeros
(initialized to zeros, Bool=False,Float=0.0,String="").

With only Dim1 set to eg 1000, would create single dimension array of 1000 elements (0 -> 999), where RT_ArrayAppend and RT_ArrayExtend
can also be used.
With Dim1 and Dim2 greater than zero, would create a two dimensional array, where RT_ArrayAppend can NOT be used, but RT_ArrayExtend
can be used to allocate and add initialized elements to the Dim1 dimension, intialized to zero.
With Dim1, Dim2 and Dim3 greater than zero, would create a three dimensional array, where RT_ArrayAppend can NOT be used, but
RT_ArrayExtend can be used to allocate and add initialized elements to the Dim1 dimension.
Will produce an error if Dim3 is non zero and Dim2 is zero.

StringLenMax, default 256(1 or more). Fixed Maximum string length excluding nul term. Error if string length exceeds.
Only used for Type=3 (string).

Returns total number of elements in array.
All RT_ArrayXXXX() errors produce an error alert.

***
***
***

RT_ArrayGetDim(string FileName,int "Dim"=0)
Returns number of dimensions in array or the size of any 1 of the dimensions.
Filename, Array Filename, no default.
Dim, default 0. (Range 0 -> 3).
0, returns the number of dimensions in the array.
Invalid dimensions will return 0, eg RT_ArrayGetDim(FileName,Dim=2) on single dimension array return 0.
Error if Dim smaller than 0 or greater than 3.

***
***
***

RT_ArrayGetType(string FileName)
Filename, Array Filename, no default.
Returns int, Type as set by RT_ArrayAlloc. 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.

***
***
***

RT_ArrayGetElSize(string FileName)
Filename, Array Filename, no default.
Returns int, element size, Type 0(bool)=1, 1(int)=4, 2(float)=4, 3(string)=StringLenMax, 4(BIN)=1, as set by RT_ArrayAlloc.

***
***
***

RT_ArrayGetNels(string FileName)
Filename, Array Filename, no default.
Returns int, total number of elements in array as set by RT_ArrayAlloc, or current number of elements if RT_ArrayAppend used to add
elements to end of single dimension array, or RT_ArrayExtend used to increase Dim1 dimension.
For a 3 dimensional array where Dim1=10, Dim2=20,Dim3=30 would return 10*20*30 ie 6000.

***
***
***

RT_ArrayGetDim1Max(string FileName)
Filename, Array Filename, no default.
Returns int, maximum theoretical Dim1 dimension for the current array with current fixed size Dim2 and Dim3 dimensions.
For 1 dimension Array: (maximum +ve integer - header size) / ElSize.
For 2 dimension Array: (maximum +ve integer - header size) / (ElSize * Dim2).
For 3 dimension Array: (maximum +ve integer - header size) / (ElSize * Dim2 * Dim3).

***
***
***

RT_ArrayGetNelMax(string FileName)
Filename, Array Filename, no default.
Returns int, maximum number of elements that could theoretically be stored in current array, taking into account the
current number of dimensions and the size of dimensions 2 and 3.

***
***
***

RT_ArrayGet(string FileName,int ix1,int "ix2",int "ix3")
Returns array element of variable Type as set in RT_ArrayAlloc. Type BIN returns an Int range 0 -> 255.
Filename, Array Filename, no default.
ixl, int, zero relative. 1st subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,1)-1.
ix2, int, zero relative. 2nd subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,2)-1.
Only valid if 2 or 3 dimensional array. For single dimension array, can omit, supply RT_Undefined(), or -1.
ix3, int, zero relative. 3rd subcript of array. Valid range 0 -> RT_ArrayGetDim(FileName,3)-1.
Only valid if 3 dimensional array. For 1 or 2 dimension array, can omit, supply RT_Undefined(), or -1.

***
***
***

RT_ArraySet(string FileName,Var,int ix1,int "ix2",int "ix3")
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc. (NOTE position in arg list changed)
ixl, int, zero relative. 1st subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,1)-1.
ix2, int, zero relative. 2nd subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,2)-1.
Only valid if 2 or 3 dimensional array. For single dimension array, can omit, supply RT_Undefined(), or -1.
ix3, int, zero relative. 3rd subcript of array. Valid range 0 -> RT_ArrayGetDim(FileName,3)-1.
Only valid if 3 dimensional array. For 1 or 2 dimension array, can omit, supply RT_Undefined(), or -1.
Returns total number of elements in array.

***
***
***

RT_ArrayExtend(string FileName,int "Add"=1)
Extends first dimension of 1, 2 and 3 dimension arrays and initializes additional elements to zero.
(initialized to zeros, Bool=False,Float=0.0,String="").
Filename, Array Filename, no default.
Add, Int default 1. How much to increase size of 1st dimension.
Returns new size of 1st dimension.

***
***
***

RT_ArrayAppend(string FileName,Var)
Adds a variable to end of single dimension array and increments the Nel count.
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc.
Returns number of elements in single dimension array.
Error if multi dimensional array.

***
***
***

RT_ArraySetAttrib(string FileName,int ix, Value)
There are 16 available attributes in the Array file header, that can be used for whatever purpose you like,
RT_ArrayAlloc() initializes all Attributes to type Int, 0.
Filename, Array Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Value, Int or Float, The value that Attrib(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.

***
***
***

RT_ArrayGetAttrib(string FileName,int ix)
There are 16 available attributes in the Array file header, that can be used for whatever purpose you like,
Filename, Array Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Returns the value of the user set attribute Int or Float (or type Int 0 if not yet set).


Unless suggestions for change, will likely remain as is.

EDIT: Array files are not deleted after use, but can use eg RT_FileDelete(ARRAY) if required.
Can use an existing ARRAY file to store eg plugin path and filenames, or as storage for a library of avs scripts
or many other possible uses.

EDIT: The elements in the currently highest dimension are physically closest to each other in file, so if you say wanted
to eg store a byte value for RGB red channel pixel in an array for a 100 frame clip of dimensions 640x480 then it would
perhaps be best set up with 1st dimension the frame number 0 -> 99, 2nd dimension the Y position 0 ->479,
and the X position 0 -> 639. eg RT_ArrayAlloc(ARRAY,type=4,dim1=100,dim2=480,dim3=640). # Type=4=BIN=8 bit BYTE
Due to filesystem disk caching, this would allow fastest scanning when, for each frame (least often changing), then for each
Y position (next least often changing), then for each X position (most often changing, innermost in nested for/next loops).
Above also most closely matches how pixel data is arranged in memory and so memory cache would also be most
efficiently used.

[martin53]

StainlessS,
I started to implement the array feature in the AWB script. Please review if this meets the intended usage:

TEMP = RT_GetSystemEnv("TEMP")
NOW = RT_LocalTimeString()
...
GScript(RT_StrReplace(""" #"
#######################################
# 1) Unique Global Variables Initialization
#######################################
#name of the file to hold the RTE array contents
global AWB_Array%%% = TEMP+"\AWB_Array_%%%_"+NOW+".tmp"
#Dim2: [0]=SceneStart, [1]=Red, [2]=Grn, [3]=Blu
RT_ArrayAlloc(AWB_Array%%%, Type=2, Dim1=cRGB.FrameCount, Dim2=4)
#'file destructor'
cRGB.CallCmd("CMD /C del "+AWB_Array%%%, "-3", Hide=true)

[StainlessS]

As you use

RT_ArrayAlloc(AWB_Array%%%, Type=2, Dim1=cRGB.FrameCount, Dim2=4)


I presume that R,G,B are float, frame number would also be stored float, however as you are creating array for entire frame set, you dont
need frame number in array, ie array(n,0) would be n.
If you were wanting individual selected frame numbers with R,G,B as float, then would myself use two arrays, one for framenum and one for R,G,B,
(or 1x dim1=2 array of int packed as single int if RGB 0->255, with frame numbers in arr(n,0))

The call to CallCmd looks wrong to me, "-3" means 3 frames but missing start frame (as trim ie trim(10,-3) would be frames 10,11,12)
Dont think you can use CallCmd inside Scriptclip although its a long time since I used it (at least not to delete file).
What is the intention of -3 ?

Am currently playing with (just started about an hour ago) a DBase thing, up to 16 fields per record of same types as ARRAY but mixed types in record fields.

going to pub now, be good :)

[martin53]

Wanted an array with first dimension frame number, second dimension 4 (meanwhile 3) to store R, G and B values per frame.

Definition:
RT_ArrayAlloc(gbDone%%%, Type=0, Dim1=cRGB.FrameCount)
RT_ArrayAlloc(gfRGB%%%, Type=2, Dim1=cRGB.FrameCount, Dim2=3)

Write:
RT_ArraySet(gfRGB%%%, R, current_frame+delta, 0)
RT_ArraySet(gfRGB%%%, G, current_frame+delta, 1)
RT_ArraySet(gfRGB%%%, B, current_frame+delta, 2)

Read:
R = RT_ArrayGet(gfRGB%%%, current_frame+delta, 0)
G = RT_ArrayGet(gfRGB%%%, current_frame+delta, 1)
B = RT_ArrayGet(gfRGB%%%, current_frame+delta, 2)

(you guessed it, g is for global, b is for bool, f for float, then comes the speaking part of the variable name and the marker string for the instance number)

... so did not want to store frame number itself, but only use it as index. cRGB is the clip. Can you please provide me an example if I misunderstood this?

CallCmd is not inside ScriptClip, sorry my sketch was poor. It is after the ScriptClip in the clip graph. I use the CallCmd version you improved based on Nic's (October 2012). CallCmd.txt Line 74 says "-3 = In Destructor (after last frame)" and that's what I want...

[StainlessS]

I changed frame range spec several versions ago (while still beta). Get later version with open/close options. (CallCmd)
EDIT: frame range spec was changed before it had it's own thread, while was still posting in original Nic Call() thread.
You have used later versions since then.
CallCmd thread is a few posts after this one at date of this post, indeed you posted in it a few days back.
CallCmd here:- http://forum.doom9.org/showthread.php?t=166063


As far the rest of your post, looks good to me.

EDIT: With current CallCmd, suggest something like (untested)

CallCmd(Close="CMD /C /Q del " + Chr(34) + AWB_Array%%% + Chr(34) ,Hide=True)


EDIT: Might be worth seeing if you can get CallCmd CLI command to do a wildcard arrays delete if multiple used per instance.
eg something like "gfRGB%%%.ARR" to "*RGB%%%.ARR"

EDIT: Added /Q switch to DEL command, "/Q Quiet mode, do not ask if ok to delete on global wildcard", maybe not necessary.

[StainlessS]

Page #1 of 3

New version, RT_Stats v1.28, see 1st post. (Added Array funcs pretty much as previous posted EDIT: v1.28Beta51)

Beta version v0.0Beta RT_DBase (will be added to RT_Stats), working well.
Here:- LINK REMOVED, Added to RT_Stats


*****************************************************
********** DBASE FUNCTIONS ***********
*****************************************************

RT_DBaseAlloc(String Filename,Int Records,String TypeString,Int "StringLenMax"=256)
Creates a file to hold DataBase records. Each record can have up to 16 fields of varying type, Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in array as an unsigned 8 bit int, upper 24 bits ignored.

Filename, DBASE Filename, no default.

Records, int zero or more. Number of records to create, initalized to zeros (initialized to zeros, Bool=False,Float=0.0,String="").

TypeString, string holding record construction data. One character for each field, valid characters are "bifsn" or Uppercase.
'b' = BOOL, 'i' = INT, 'f' = FLOAT, 'S' = STRING, 'n' = BIN.
The 's' STRING construction character can be followed by numeric digits describing maximum string length of fixed string (excluding
any nul term character), otherwise StringLenMax is used by default instead.

StringLenMax Default=256. Default length of String fields if no explicit string length defined following TypeString S constructor char.

Example TypeString:-
"ifffs1024bsn", 1st field Int, 3 Float fields, 1 string field of length=1024, 1 Bool field, 1 string field using length of
StringLenMax, and 1 BIN field. 8 Fields in all (0->7).
Returns number of records created.

***
***
***

RT_DBaseFields(String Filename)
Filename, DBASE Filename, no default.
Return number of fields per record.

***
***
***

RT_DBaseFieldType(String Filename,Int Field)
Filename, DBASE Filename, no default.
Field, range 0 -> RT_DBaseFields(Filename) - 1
Return type of Field. 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.

***
***
***

RT_DBaseFieldSize(String Filename,Int Field)
Filename, DBASE Filename, no default.
Field, range 0 -> RT_DBaseFields(Filename) - 1
Return size of Field in bytes. Bool=1, Int=4, Float=4, String length as set by RT_DBaseAlloc(), BIN=1.

***
***
***

RT_DBaseRecords(String Filename)
Filename, DBASE Filename, no default.
Returns current number of records in Filename database.

***
***
***

RT_DBaseRecordSize(String Filename)
Filename, DBASE Filename, no default.
Returns record size, ie sum of field sizes.

***
***
***

RT_DBaseRecordsMax(String Filename)
Filename, DBASE Filename, no default.
Returns maximum theoretical number of records that could be stored in file based on (Maximum +ve int - header size) / Record Size.
(where record size is sum of all field sizes).

***
***
***

RT_DBaseGetField(String Filename,Int Record,Int Field)
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Field, range 0 -> RT_DBaseFields(Filename) - 1
Returns contents of a field belonging to the particular record.

***
***
***

RT_DBaseSetField(String Filename,Int Record,Int Field, Var)
Sets a single Field in a Record to Var.
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Field, range 0 -> RT_DBaseFields(Filename) - 1
Var, variable of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseSet(String Filename,Int Record, Var_1, ... , Var_n)
Simultaneously sets all fields of a record. All fields must be supplied.
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Var_1, ... ,Var_n, Field variables of types as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseExtend(String Filename,Int "Add"=1)
Adds a number of blank records to Filename Database. (initialized to zeros, Bool=False, Float=0.0, String="").
Filename, DBASE Filename, no default.
Add, Int default 1, Number of blank records to add to the DataBase.
Returns the current number of records.

***
***
***

RT_DBaseAppend(String Filename, FieldVar_1, ... , FieldVar_n)
Creates a record from Field Vars and appends it to the end of the DataBase file. All fields must be supplied.
Filename, DBASE Filename, no default.
FieldVar_1, ... , FieldVar_n. A variable for each field of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseGetAttrib(String Filename,Int ix)
There are 16 available attributes in the DataBase file header, that can be used for whatever purpose you like,
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Returns the value of the user set attribute Int or Float (or type Int 0 if not yet set).

***
***
***

RT_DBaseSetAttrib(String Filename,Int ix, Var)
There are 16 available attributes in the DataBase file header, that can be used for whatever purpose you like,
RT_DBaseAlloc() initializes all Attributes to type Int, 0.
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 15.
Value, Int or Float, The value that Attrib(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.

[StainlessS]

Page #2 of 3

# DBase_Timer_Test.Avs - As Knowledge of fields in record not hard coded, cannot test Extend/Append (too long gen random data).

/*

RT_DBaseAlloc(String Filename,Int Records,String TypeString,Int "StringLenMax"=256)
RT_DBaseFields(String Filename)
RT_DBaseFieldType(String Filename,Int Field)
RT_DBaseFieldSize(String Filename,Int Field)
RT_DBaseRecords(String Filename)
RT_DBaseRecordSize(String Filename)
RT_DBaseRecordsMax(String Filename)
RT_DBaseGetField(String Filename,Int Record,Int Field)
RT_DBaseSetField(String Filename,Int Record,Int Field, Var)
RT_DBaseSet(String Filename,Int Record, Var1, ... , Varn)
RT_DBaseExtend(String Filename,Int Add)
RT_DBaseAppend(String Filename, Var_1, ... , Var_n)
RT_DBaseGetAttrib(String Filename,Int ix)
RT_DBaseSetAttrib(String Filename,Int ix, Var)

*/

# Config
DB = "DBASE.RDB" # DBase FileName
TYPESTRING = "bifs64n" # Describes the DBase Fields and optional size of any String Fields
# b=Bool, i=Int,f=Float,s=String(eg "s256" is StringLen=256),n=BIN.
RECORDS = 100000 # Initial number of records in DBase
DEFAULT_STRINGLENMAX = 256 # If TYPESTRING type 's' String type Field does not have explicit size trailer.
DELETE = False

##################################
TYPENAMES=RT_String("Bool\nInt\nFloat\nString\nBin\n")
ALPHA = "ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZ"
ALPHA=ALPHA+ALPHA # Len 260
GScript("""
TOTAL_START=RT_Timer()
RT_DebugF("DBase_Timer_Test.Avs")
START=RT_Timer()
RT_DBaseAlloc(DB,RECORDS,TYPESTRING,stringlenmax=DEFAULT_STRINGLENMAX)
STOP=RT_Timer()
RT_DebugF("TYPESTRING = %c%s%c",34,TYPESTRING,34)
RT_DebugF("Records = %d",RT_DBaseRecords(DB))
RT_DebugF("RecordSize = %d",RT_DBaseRecordSize(DB))
RT_DebugF("RecordsMax = %d ($%X)",RT_DBaseRecordsMax(DB),RT_DBaseRecordsMax(DB))
FIELDS=RT_DBaseFields(DB)
RT_DebugF("Fields = %d",FIELDS)
for(i=0,Fields - 1) {
TYPE_S=RT_TxtGetLine(TYPENAMES,RT_DBaseFieldType(DB,i))
RT_DebugF(" %2d ) Type = %d(%6s) Size = %d",i,RT_DBaseFieldType(DB,i),TYPE_S,RT_DBaseFieldSize(DB,i))
}
RT_DebugF("RT_DBaseAlloc() = %.3f secs (Per Field Ave %f microsecs)",STOP-START,1000000.0*(STOP-START)/(RECORDS*FIELDS))
RT_DebugF("Check Init")
START_INIT=RT_Timer()
# You would likely know the Types in your own database (we do not, MUCH SLOWER as we have to inquire from DBASE on each Field Interation)
# Probably faster to do nested Record, Field, if we knew DBASE types in advance
for(field=0,FIELDS - 1) {
type = RT_DBaseFieldType(DB,field) # You would likely know the types of your own database.
TYPE_S=RT_TxtGetLine(TYPENAMES,type)
vs=Select(type,False,0,0.0,"",0)
RT_DebugF("Check Init on field %d Type=%d(%6s)",field,type,TYPE_S)
START=RT_Timer()
for(record=0,RECORDS - 1 ) {
vg=RT_DBaseGetField(DB,record,field)
if(vg != vs) {
RT_DebugF("ERROR, Record:Field %d:%d Type %d Set=%s Get=%s",record,field,type,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Init on field %d Type=%d(%6s) = %.3f secs (Per field Ave %f microsecs)",field,type,TYPE_S,STOP-START,1000000.0*(STOP-START)/RECORDS)
}
STOP_INIT=RT_Timer()
RT_DebugF("TOTAL Check Init = %.3f secs (Per Field Ave %f microsecs)",STOP_INIT-START_INIT,1000000.0*(STOP_INIT-START_INIT)/(RECORDS*FIELDS))
Try {RT_DBaseGetField(RT_DBaseRecords(DB),0)RT_DebugF("Init END Check FAILED")} catch (err) {RT_DebugF("Init END Check Passed")}
RT_DebugF("Check Attributes Set/Get")
START=RT_Timer()
for(i=0,15) {
vs= (i % 2 == 0) ? i*1000 : i * PI # Test Int and Float Attributes
type=(i % 2 == 0) ? 1 : 2
RT_DBaseSetAttrib(DB,i,vs)
vg = RT_DBaseGetAttrib(DB,i)
if(vg != vs) {
RT_DebugF("%d) TYPE=%d SetAttrib/GetAttrib Error, Set = %s Get = %s",i,type,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Attributes Set/Get = %.3f secs (Per element Ave %f microsecs)",STOP-START,1000000.0*(STOP-START)/16)

RT_DebugF("Check Set/Get Compare")
Rand(seed=true)
START_SETGET=RT_Timer()
for(field=0,FIELDS - 1) {
type = RT_DBaseFieldType(DB,field)
TYPE_S=RT_TxtGetLine(TYPENAMES,field)
fieldsize=RT_DBaseFieldSize(DB,field)
rnd=Rand() + 1 # ALL fields types set same as just timing
RT_DebugF("Check Set on field %d Type=%d(%6s)",field,type,TYPE_S)
vs=Select(Type,RT_BitTst(rand,0),rand,Float(rand),RightStr(ALPHA + "_" + String(rand),fieldsize) ,RT_BitAnd(rand,$FF))
START=RT_Timer()
for(record=0,RECORDS - 1 ) {
RT_DBaseSetField(DB,record,field, vs)
}
STOP=RT_Timer()
RT_DebugF("Check Set on field %d Type=%d(%6s) = %.3f secs (Per field Ave %f microsecs)",field,type,TYPE_S,STOP-START,1000000.0*(STOP-START)/RECORDS)
RT_DebugF("Check Get on field %d Type=%d(%6s)",field,type,TYPE_S)
START=RT_Timer()
for(record=0,RECORDS - 1 ) {
vg=RT_DBaseGetField(DB,record,field)
if(vg != vs) {
RT_DebugF("ERROR, Record:Field %d:%d Type %d Set=%s Get=%s",record,field,type,String(vs),String(vg))
}
}
STOP=RT_Timer()
RT_DebugF("Check Get on field %d Type=%d(%6s) = %.3f secs (Per field Ave %f microsecs)",field,type,TYPE_S,STOP-START,1000000.0*(STOP-START)/RECORDS)
}
STOP_SETGET=RT_Timer()
RT_DebugF("TOTAL Check Set/Get Compare = %.3f secs (Per field Ave %f microsecs)",STOP_SETGET-START_SETGET,1000000.0*(STOP_SETGET-START_SETGET)/(RECORDS*FIELDS))
TOTAL_STOP=RT_Timer()
RT_DebugF("TOTAL SCRIPT TIME = %.3f seconds",TOTAL_STOP-TOTAL_START)
(DELETE) ? RT_FileDelete(DB) : NOP
""")
colorbars.KillAudio


log

00000001 0.00000000 RT_DebugF: DBase_Timer_Test.Avs
00000002 0.02195171 RT_DebugF: TYPESTRING = "bifs64n"
00000003 0.02216067 RT_DebugF: Records = 100000
00000004 0.02236121 RT_DebugF: RecordSize = 74
00000005 0.02266201 RT_DebugF: RecordsMax = 29020045 ($1BACF8D)
00000006 0.02283370 RT_DebugF: Fields = 5
00000007 0.02320408 RT_DebugF: 0 ) Type = 0( Bool) Size = 1
00000008 0.02356772 RT_DebugF: 1 ) Type = 1( Int) Size = 4
00000009 0.02393370 RT_DebugF: 2 ) Type = 2( Float) Size = 4
00000010 0.02430234 RT_DebugF: 3 ) Type = 3(String) Size = 64
00000011 0.02467171 RT_DebugF: 4 ) Type = 4( Bin) Size = 1
00000012 0.02478274 RT_DebugF: RT_DBaseAlloc() = 0.031 secs (Per Field Ave 0.062000 microsecs)
00000013 0.02487412 RT_DebugF: Check Init
00000014 0.02558443 RT_DebugF: Check Init on field 0 Type=0( Bool)
00000015 8.00883389 RT_DebugF: Check Init on field 0 Type=0( Bool) = 7.984 secs (Per field Ave 79.839996 microsecs)
00000016 8.00947380 RT_DebugF: Check Init on field 1 Type=1( Int)
00000017 15.78374863 RT_DebugF: Check Init on field 1 Type=1( Int) = 7.766 secs (Per field Ave 77.660004 microsecs)
00000018 15.78439236 RT_DebugF: Check Init on field 2 Type=2( Float)
00000019 23.52473068 RT_DebugF: Check Init on field 2 Type=2( Float) = 7.750 secs (Per field Ave 77.500000 microsecs)
00000020 23.52537155 RT_DebugF: Check Init on field 3 Type=3(String)
00000021 31.30543900 RT_DebugF: Check Init on field 3 Type=3(String) = 7.781 secs (Per field Ave 77.809998 microsecs)
00000022 31.30608177 RT_DebugF: Check Init on field 4 Type=4( Bin)
00000023 39.04590607 RT_DebugF: Check Init on field 4 Type=4( Bin) = 7.734 secs (Per field Ave 77.340012 microsecs)
00000024 39.04608154 RT_DebugF: TOTAL Check Init = 39.015 secs (Per Field Ave 78.030006 microsecs)
00000025 39.04807663 RT_DebugF: Init END Check Passed
00000026 39.04816818 RT_DebugF: Check Attributes Set/Get
00000027 39.05691910 RT_DebugF: Check Attributes Set/Get = 0.016 secs (Per element Ave 999.927490 microsecs)
00000028 39.05701065 RT_DebugF: Check Set/Get Compare
00000029 39.05791855 RT_DebugF: Check Set on field 0 Type=0( Bool)
00000030 50.78600693 RT_DebugF: Check Set on field 0 Type=0( Bool) = 11.734 secs (Per field Ave 117.340012 microsecs)
00000031 50.78610229 RT_DebugF: Check Get on field 0 Type=0( Bool)
00000032 59.15937424 RT_DebugF: Check Get on field 0 Type=0( Bool) = 8.375 secs (Per field Ave 83.750000 microsecs)
00000033 59.15987396 RT_DebugF: Check Set on field 1 Type=1( Int)
00000034 70.91230011 RT_DebugF: Check Set on field 1 Type=1( Int) = 11.750 secs (Per field Ave 117.499962 microsecs)
00000035 70.91239929 RT_DebugF: Check Get on field 1 Type=1( Int)
00000036 79.30217743 RT_DebugF: Check Get on field 1 Type=1( Int) = 8.391 secs (Per field Ave 83.909988 microsecs)
00000037 79.30267334 RT_DebugF: Check Set on field 2 Type=2( Float)
00000038 91.03527832 RT_DebugF: Check Set on field 2 Type=2( Float) = 11.734 secs (Per field Ave 117.340012 microsecs)
00000039 91.03538513 RT_DebugF: Check Get on field 2 Type=2( Float)
00000040 99.40809631 RT_DebugF: Check Get on field 2 Type=2( Float) = 8.375 secs (Per field Ave 83.750000 microsecs)
00000041 99.40859985 RT_DebugF: Check Set on field 3 Type=3(String)
00000042 111.33905792 RT_DebugF: Check Set on field 3 Type=3(String) = 11.922 secs (Per field Ave 119.220047 microsecs)
00000043 111.33914948 RT_DebugF: Check Get on field 3 Type=3(String)
00000044 119.83779144 RT_DebugF: Check Get on field 3 Type=3(String) = 8.500 secs (Per field Ave 85.000000 microsecs)
00000045 119.83829498 RT_DebugF: Check Set on field 4 Type=4( Bin)
00000046 131.56878662 RT_DebugF: Check Set on field 4 Type=4( Bin) = 11.735 secs (Per field Ave 117.350014 microsecs)
00000047 131.56887817 RT_DebugF: Check Get on field 4 Type=4( Bin)
00000048 139.94674683 RT_DebugF: Check Get on field 4 Type=4( Bin) = 8.375 secs (Per field Ave 83.750000 microsecs)
00000049 139.94692993 RT_DebugF: TOTAL Check Set/Get Compare = 100.891 secs (Per field Ave 201.782013 microsecs)
00000050 139.94709778 RT_DebugF: TOTAL SCRIPT TIME = 139.953 seconds

[StainlessS]

Page #3 of 3

# DBase_Append_Test.Avs - Knowledge of fields in record, can test Extend/Append

/*

RT_DBaseAlloc(String Filename,Int Records,String TypeString,Int "StringLenMax"=256)
RT_DBaseFields(String Filename)
RT_DBaseFieldType(String Filename,Int Field)
RT_DBaseFieldSize(String Filename,Int Field)
RT_DBaseRecords(String Filename)
RT_DBaseRecordSize(String Filename)
RT_DBaseRecordsMax(String Filename)
RT_DBaseGetField(String Filename,Int Record,Int Field)
RT_DBaseSetField(String Filename,Int Record,Int Field, Var)
RT_DBaseSet(String Filename,Int Record, Var1, ... , Varn)
RT_DBaseExtend(String Filename,Int Add)
RT_DBaseAppend(String Filename, Var_1, ... , Var_n)
RT_DBaseGetAttrib(String Filename,Int ix)
RT_DBaseSetAttrib(String Filename,Int ix, Var)

*/

# Config
DB = "DBASE.RDB" # DBase FileName
RECORDS = 10000 # Initial number of records in DBase
DELETE = False
##################################
GScript("""
TYPENAMES=RT_String("Bool\nInt\nFloat\nString\nBin\n")
SLEN=64
TYPESTRING=RT_String("bifs%dn",SLEN)
RT_DebugF("DBase_Append_Test.Avs")
RT_DBaseAlloc(DB,0,TYPESTRING)
RT_DebugF("TYPESTRING = %c%s%c",34,TYPESTRING,34)
RT_DebugF("Records = %d",RT_DBaseRecords(DB))
RT_DebugF("RecordSize = %d",RT_DBaseRecordSize(DB))
RT_DebugF("RecordsMax = %d ($%X)",RT_DBaseRecordsMax(DB),RT_DBaseRecordsMax(DB))
FIELDS=RT_DBaseFields(DB)
RT_DebugF("Fields = %d",FIELDS)
for(i=0,Fields - 1) {
TYPE_S=RT_TxtGetLine(TYPENAMES,RT_DBaseFieldType(DB,i))
RT_DebugF(" %2d ) Type = %d(%6s) Size = %d",i,RT_DBaseFieldType(DB,i),TYPE_S,RT_DBaseFieldSize(DB,i))
}
RT_DebugF("Extending to %d records",RECORDS)
RT_DBaseExtend(DB,RECORDS)
RT_DebugF("Check Extended Init To Zero")
vs_0 = False vs_1 = 0 vs_2 = 0.0 vs_3 = "" vs_4 = 0
for(record=0,RECORDS - 1 ) {
vg_0=RT_DBaseGetField(DB,record,0)
vg_1=RT_DBaseGetField(DB,record,1)
vg_2=RT_DBaseGetField(DB,record,2)
vg_3=RT_DBaseGetField(DB,record,3)
vg_4=RT_DBaseGetField(DB,record,4)
if((vs_0 != vg_0) || (vs_1 != vg_1) || (vs_2 != vg_2) || (vs_3 != vg_3) || (vs_4 != vg_4)) {
RT_DebugF("ERROR, @ Record %d",record)
}
}
Try {RT_DBaseGetField(RT_DBaseRecords(DB),0)RT_DebugF("Init END Check FAILED")} catch (err) {RT_DebugF("Init END Check Passed")}
RT_DebugF("RT_DBaseSet")
for(record=0,RECORDS - 1 ) {
vs_0 = RT_BitTst(record,0)
vs_1 = record
vs_2 = Float(record)
vs_3 = RT_StrPad("",SLEN,Chr((record % 25) + 65))
vs_4 = RT_BitAnd(record,$FF)
RT_DBaseSet(DB,record,vs_0,vs_1,vs_2,vs_3,vs_4)
}
RT_DebugF("RT_DBaseSet/Get Compare")
for(record=0,RECORDS - 1 ) {
vs_0 = RT_BitTst(record,0)
vs_1 = record
vs_2 = Float(record)
vs_3 = RT_StrPad("",SLEN,Chr((record % 25) + 65))
vs_4 = RT_BitAnd(record,$FF)
vg_0=RT_DBaseGetField(DB,record,0)
vg_1=RT_DBaseGetField(DB,record,1)
vg_2=RT_DBaseGetField(DB,record,2)
vg_3=RT_DBaseGetField(DB,record,3)
vg_4=RT_DBaseGetField(DB,record,4)
if((vs_0 != vg_0) || (vs_1 != vg_1) || (vs_2 != vg_2) || (vs_3 != vg_3) || (vs_4 != vg_4)) {
RT_DebugF("ERROR, @ Record %d",record)
}
}
RT_DebugF("RT_DBase Append()")
for(i=RECORDS,RECORDS*2 - 1 ) {
vs_0 = RT_BitTst(i,0)
vs_1 = i
vs_2 = Float(i)
vs_3 = RT_StrPad("",SLEN,Chr((i % 25) + 65))
vs_4 = RT_BitAnd(i,$FF)
RT_DBaseAppend(DB,vs_0,vs_1,vs_2,vs_3,vs_4)
}
RT_DebugF("RT_DBaseAppend() Compare")
for(i=RECORDS,RECORDS*2 - 1 ) {
vs_0 = RT_BitTst(i,0)
vs_1 = i
vs_2 = Float(i)
vs_3 = RT_StrPad("",SLEN,Chr((i % 25) + 65))
vs_4 = RT_BitAnd(i,$FF)
vg_0=RT_DBaseGetField(DB,i,0)
vg_1=RT_DBaseGetField(DB,i,1)
vg_2=RT_DBaseGetField(DB,i,2)
vg_3=RT_DBaseGetField(DB,i,3)
vg_4=RT_DBaseGetField(DB,i,4)
if((vs_0 != vg_0) || (vs_1 != vg_1) || (vs_2 != vg_2) || (vs_3 != vg_3) || (vs_4 != vg_4)) {
RT_DebugF("ERROR, @ Record %d",record)
}
}
Try {RT_DBaseGetField(DB,RECORDS*2-1,4) RT_DebugF("FINAL_END_1 Check PASSED")} catch (err) {RT_DebugF("FINAL_END_1 Check FAILED")}
Try {RT_DBaseGetField(DB,RECORDS*2,0) RT_DebugF("FINAL_END_2 Check FAILED")} catch (err) {RT_DebugF("FINAL_END_2 Check PASSED")}
RT_DebugF("TYPESTRING = %c%s%c",34,TYPESTRING,34)
RT_DebugF("Records = %d (Should be %d)",RT_DBaseRecords(DB),RECORDS*2)
RT_DebugF("RecordSize = %d",RT_DBaseRecordSize(DB))
RT_DebugF("RecordsMax = %d ($%X)",RT_DBaseRecordsMax(DB),RT_DBaseRecordsMax(DB))
FIELDS=RT_DBaseFields(DB)
RT_DebugF("Fields = %d",FIELDS)
for(i=0,Fields - 1) {
TYPE_S=RT_TxtGetLine(TYPENAMES,RT_DBaseFieldType(DB,i))
RT_DebugF(" %2d ) Type = %d(%6s) Size = %d",i,RT_DBaseFieldType(DB,i),TYPE_S,RT_DBaseFieldSize(DB,i))
}
RT_DebugF("ALL DONE")
(DELETE) ? RT_FileDelete(DB) : NOP
""")
colorbars.KillAudio


log

00000001 0.00000000 RT_DebugF: DBase_Append_Test.Avs
00000002 0.00174137 RT_DebugF: TYPESTRING = "bifs64n"
00000003 0.00192230 RT_DebugF: Records = 0
00000004 0.00209310 RT_DebugF: RecordSize = 74
00000005 0.00236371 RT_DebugF: RecordsMax = 29020045 ($1BACF8D)
00000006 0.00253493 RT_DebugF: Fields = 5
00000007 0.00290801 RT_DebugF: 0 ) Type = 0( Bool) Size = 1
00000008 0.00327155 RT_DebugF: 1 ) Type = 1( Int) Size = 4
00000009 0.00363812 RT_DebugF: 2 ) Type = 2( Float) Size = 4
00000010 0.00399721 RT_DebugF: 3 ) Type = 3(String) Size = 64
00000011 0.00435195 RT_DebugF: 4 ) Type = 4( Bin) Size = 1
00000012 0.00444715 RT_DebugF: Extending to 10000 records
00000013 0.00640291 RT_DebugF: Check Extended Init To Zero
00000014 3.87600493 RT_DebugF: Init END Check Passed
00000015 3.87609553 RT_DebugF: RT_DBaseSet
00000016 11.99874210 RT_DebugF: RT_DBaseSet/Get Compare
00000017 22.74699402 RT_DebugF: RT_DBase Append()
00000018 31.03572845 RT_DebugF: RT_DBaseAppend() Compare
00000019 41.88741302 RT_DebugF: FINAL_END_1 Check PASSED
00000020 41.88766479 RT_DebugF: FINAL_END_2 Check PASSED
00000021 41.88776398 RT_DebugF: TYPESTRING = "bifs64n"
00000022 41.88793945 RT_DebugF: Records = 20000 (Should be 20000)
00000023 41.88810730 RT_DebugF: RecordSize = 74
00000024 41.88834381 RT_DebugF: RecordsMax = 29020045 ($1BACF8D)
00000025 41.88851547 RT_DebugF: Fields = 5
00000026 41.88890457 RT_DebugF: 0 ) Type = 0( Bool) Size = 1
00000027 41.88928223 RT_DebugF: 1 ) Type = 1( Int) Size = 4
00000028 41.88965988 RT_DebugF: 2 ) Type = 2( Float) Size = 4
00000029 41.89003372 RT_DebugF: 3 ) Type = 3(String) Size = 64
00000030 41.89041138 RT_DebugF: 4 ) Type = 4( Bin) Size = 1
00000031 41.89050674 RT_DebugF: ALL DONE

[Forensic]

#StainlessS: My script finds the average luminosity differential by analyzing every adjacent pixel pair within the first video frame, with an option to only process every other 2x2 horizontal and vertical block to ensure any slice (e.g. h.264, h.265) boundaries are excluded. My script does not accept a frame range, assuming that the user will decide which frame to send to the function with each call (as only the first frame is processed). By calling this function before and again after a given filter, this script quantifies a filter's affect on luminosity contrast. NOTE: The perimeter pixels are intentionally ignored as they often incur higher quantization errors.
I use this dozens of times with every video enhancement. The problem is that my AviSynth script is sooooo slow due to the thousands of RT_YDifference calls. Unfortunately, it may be months before I can successfully write and compile a valid AviSynth CPP file. Would you consider adding this capability to RT_Stats?

Function AvgLumaDif(Clip Clip, bool "slices") {
GScript("""
slices = default(slices, false)
current_frame=0
value=float(0)
count=float(0)
step=1
if (slices) {step=4}
for (y=1,Clip.height-step,step) { for (x=1,Clip.width-step,step) {
Try {
value=value+RT_YDifference(Clip, delta=0, x=x, y=y, w=1, h=1, x2=x+1, y2=y)+RT_YDifference(Clip, delta=0, x=x, y=y, w=1, h=1, x2=x, y2=y+1)
\+RT_YDifference(Clip, delta=0, x=x+1, y=y, w=1, h=1, x2=x+1, y2=y+1)+RT_YDifference(Clip, delta=0, x=x, y=y+1, w=1, h=1, x2=x+1, y2=y+1)
count=count+1
}
Catch (err) {}
} }
if (count>0) {value=value/(4*count)}
return value
""")
}

[StainlessS]

Forensic, give me a couple of days and I see what I can do.

[Forensic]

I appreciate the consideration.

[StainlessS]

Function AvgLumaDif(Clip Clip, bool "slices") {
GScript("""
slices = default(slices, false)
current_frame=0
value=float(0)
count=float(0)
step=1
if (slices) {step=4}
for (y=1,Clip.height-step,step) { for (x=1,Clip.width-step,step) {
Try {
value=value+RT_YDifference(Clip, delta=0, x=x, y=y, w=1, h=1, x2=x+1, y2=y)+RT_YDifference(Clip, delta=0, x=x, y=y, w=1, h=1, x2=x, y2=y+1)
\+RT_YDifference(Clip, delta=0, x=x+1, y=y, w=1, h=1, x2=x+1, y2=y+1)+RT_YDifference(Clip, delta=0, x=x, y=y+1, w=1, h=1, x2=x+1, y2=y+1)
count=count+1
}
Catch (err) {}
} }
if (count>0) {value=value/(4*count)}
return value
""")
}


Have so far modified above to this (untested)


Function AvgLumaDif(Clip Clip, bool "slices", int "n") {
GScript("""
step= (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
value=0.0 count=0.0
for (y=1,Clip.height-step,step) {
for (x=1,Clip.width-step,step) {
Try {
value=value
\ + RT_YDifference(Clip,n,delta=0,x=x, y=y, w=1, h=1, x2=x+1, y2=y)
\ + RT_YDifference(Clip,n,delta=0,x=x, y=y, w=1, h=1, x2=x, y2=y+1)
\ + RT_YDifference(Clip,n,delta=0,x=x+1,y=y, w=1, h=1, x2=x+1, y2=y+1)
\ + RT_YDifference(Clip,n,delta=0,x=x, y=y+1, w=1, h=1, x2=x+1, y2=y+1)
count=count+1
} Catch (err) {}
}
}
if (count>0) {value=value/(4*count)}
return value
""")
}


Can you clarify your intention as it don't seem right to me.
With step=1, the 2x2 pixel blocks overlap, with step=4, you only process one 2x2 pixel block out of every 4.
If intention is to process every 2x2 pixel block once only, then see no alternative to step=2, skipping 2 pixel
perimeter if slice=true, skip 1 pixel perimeter if slice=false (with little difference from slice=true except blocks
are offset x,y by 1 pixel).

EDIT: Perhaps something like this (modified from above described, anticipating required intention):

Function AvgLumaDif(Clip Clip, bool "slices", int "n") {
Clip
perimeter = (Defined(slices) && slices) ? 4 : 2 # EDIT: Or perhaps 2:1
n = Default(n,0) # default current_frame = 0
value=0.0
GScript("""
for (y=perimeter,Height-(2+perimeter),2) {
for (x=perimeter,Width-(2+perimeter),2) {
value=value
\ + RT_YDifference(Clip,n,delta=0,x=x, y=y, w=1, h=1, x2=x+1, y2=y)
\ + RT_YDifference(Clip,n,delta=0,x=x, y=y, w=1, h=1, x2=x, y2=y+1)
\ + RT_YDifference(Clip,n,delta=0,x=x+1,y=y, w=1, h=1, x2=x+1, y2=y+1)
\ + RT_YDifference(Clip,n,delta=0,x=x, y=y+1, w=1, h=1, x2=x+1, y2=y+1)
}
}
count = ((Width / 2) - perimeter) * ((Height / 2) - perimeter)
if (count>0) {value=value/(4*count)}
""")
return value
}


EDITED:UNTESTED

EDIT: are RGB and YUY2 colorspaces required ? (RGB -> YUV_Y)

[Forensic]

Yes, the intention is for all the 2x2 blocks to overlap when Slices is set to false (thus step = 1). In that way, every possible horizontal and vertical pixel pair gets evaluated. When Slices is true, only 1/16th as many pixels pairs will be tested, but that is to ensure all tested pixels are always at least one pixel away from slice boundaries (which can occur at nearly any multiple of every four pixels under h.265, as I understand it). This is important because perimeter pixels, occurring at either the entire video boundary or the boundaries between slices, can disproportionally represent any contrast changes as a result of the compression's disproportional quantization errors spanning those boundaries.

Your first rewrite of my script is functionally identical to mine, except that you added "n" so a specific frame can be called rather than inferred. EITHER WAY, THIS IS WHAT I INTENDED. The second rewrite "Perhaps something like this (modified from above described, anticipating required intention)" is less suitable as it unnecessarily skips 75% of the available pixel pairs, and thus is less accurate, albeit marginally. I would only want that if, even as a DLL, functionality is far too slow otherwise.

BTW: I have to include "current_frame=0" for my script to work on my primary computer. However, that is a mute point once this script becomes a DLL.

[StainlessS]

unnecessarily skips 75% of the available pixel pairs
Rather than duplicating comparisons nearly all of the time, ie non duplicating only around edges with perimeter with remainder comparisons all done twice,
that what you mean ?

When Slices is true, only 1/16th as many pixels pairs will be tested, but that is to ensure all tested pixels are always at least one pixel away from slice boundaries

In that case would would not perimeter be required to be 5, step = 4 ?


0000000000000000
0000000000000000
0000000000000000
0000000000000000
0000111111111111
00001xx11xx11xx1
00001xx11xx11xx1
0000111111111111


x's being when slice = true, 1's being slice boundaries,0's being perimeter.

EDIT: Or rather, perimeter = 4, start pos = perimeter + 1 (perimeter = (slice=true) ? 4 : 0)

EDIT: Arh forget what I said, I was thinking you wanted perimeter around edge of frame, and now also around each slice, I understand now. :)

[Forensic]

Just to be sure we are on the same page.


x= process, -=don't process

slice=false slice=true
------------------------ ------------------------
-xxxxxxxxxxxxxxxxxxxxxx- -xx--xx--xx--xx--xx--xx-
-xxxxxxxxxxxxxxxxxxxxxx- -xx--xx--xx--xx--xx--xx-
-xxxxxxxxxxxxxxxxxxxxxx- ------------------------
-xxxxxxxxxxxxxxxxxxxxxx- ------------------------
-xxxxxxxxxxxxxxxxxxxxxx- -xx--xx--xx--xx--xx--xx-
-xxxxxxxxxxxxxxxxxxxxxx- -xx--xx--xx--xx--xx--xx-
-xxxxxxxxxxxxxxxxxxxxxx- ------------------------
-xxxxxxxxxxxxxxxxxxxxxx- ------------------------
-xxxxxxxxxxxxxxxxxxxxxx- -xx--xx--xx--xx--xx--xx-
-xxxxxxxxxxxxxxxxxxxxxx- -xx--xx--xx--xx--xx--xx-
------------------------ ------------------------

[Gavino]

I have to include "current_frame=0" for my script to work on my primary computer.
That's because, if 'n' is not specified to RT_YDifference, it defaults to the value of current_frame. Since your function is called at compile time, current_frame does not exist unless you set it yourself. In StainlessS's version of the function, 'n' is specified, so current_frame is irrelevant.

[Forensic]

Thank you. That makes sense.

[StainlessS]

Forensic
Find below, 1) equivalent to original func, 2) modified using similar-ish logic, 3) suggested implementation,
if OK will probably start coding tomorrow, (pub is calling to me :) ).

EDIT: If not OK, then need to know how you want Odds & Ends dealt with.

Still want to know, is YUY2 and RGB (RGB -> YUV_Y) required ? *** FORGET THIS QUESTION ***


Function AvgLumaDif(Clip c, bool "slices", int "n") { # Equivalent to Original
c
step= (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
value=0.0 count=0
GScript("""
for (y=1,Height-step,step) {
for (x=1,Width-step,step) {
Try {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1)
count=count+1
} Catch (err) {}
}
}
if (count>0) {value=value/(4*count)}
""")
return value
}

Function AvgLumaDif_2(Clip c, bool "slices", int "n") { # Same logic, Modifed to remove nonsense eg width-step for limit
c
step= (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
value=0.0 count=0
GScript("""
for (y=1,H-3,step) {
for (x=1,W-3,step) {
Try {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1)
count=count+1
} Catch (err) {}
}
}
if (count>0) {value=value/(4*count)}
""")
return value
}

Function AvgLumaDif_MOD(Clip c, bool "slices", int "n") { # Suggested implementation, same result as modified logic above
c
step = (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0
GScript("""
for (y=1,H-3,step) {
for (x=1,W-3,step) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1)
}
}
count = (step==1) ? ((W-3) * (H-3)) : (W * H)
value = value / (4*count)
""")
return value
}

W=512+3
ColorBars(width=W,height=64)
A=AvgLumaDif()
B=AvgLumaDif_2()
C=AvgLumaDif_MOD()
RT_DebugF("%f %f %f",A,B,C)
RT_SubTitle("%f %f %f",A,B,C)


EDIT: I'll try to comment source so you can understand and include in your own functions dll.
EDIT: Although it will not use RT_ funcs, will read frame data directly so will not be too similar to script (speed),
forget previous question on YUY2 and RGB, I shall implement for those too as it may assist for future projects.

[Forensic]

I see a minor math error when slices is TRUE (see below)

Function AvgLumaDif_MOD(Clip c, bool "slices", int "n") { # Suggested implementation, same result as modified logic above
c
step = (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0
GScript("""
for (y=1,H-3,step) {
for (x=1,W-3,step) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1)
}
}
count = (step==1) ? ((W-3) * (H-3)) : (W * H /16)
value = value / (4*count)
""")
return value
}

[StainlessS]

yes of course u r correct. that error would not have escaped notice on implemented plug. I should have used w/4 * h/4.

[Forensic]

StainlessS: "slices" should default to false and the final script does not denote that. Does the boolean naturally default to false when a state is not specified?

DOCUMENTATION: AvgLumaDif returns the average of all pixel pair differentials for a given frame "n" (default=0) ignoring perimeter pixels and, when optional "slices" is true (default=false), all 4x4 pixel block boundaries are treated as perimeter pixels.

[StainlessS]

F, ( perhaps I should have answered in PM, where I think the question originated)

step = (Defined(slices) && slices) ? 4 : 1

The logic there does all that you desire (well maybe not all but all you could expect).

Defined(Slices) is a boolean also Slices is a Boolean.
Result of two Booleans is a Boolean.
If Defined(Slices) aint true, then Slices dont matter at all.. (under condition && condition)

More on what you require, step is only 4, if Slices is both Defined, and True, Else step is 1. but am not
altogether above mistakes, could have it altogether back to front. ( been suffering from a 'sort' of illness of late
we who suffer from it know it as L.O.A., "lack of alcohol"), really bad thing to get. (Not terminal, just feels like it is).

[Gavino]

Another way of looking at it is that
Defined(slices) && slices
is equivalent to (and could be replaced by) Default(slices, false)

DOCUMENTATION: AvgLumaDif returns the average of all pixel pair differentials for a given frame "n" (default=0) ignoring perimeter pixels and, when optional "slices" is true (default=false), all 4x4 pixel block boundaries are treated as perimeter pixels.
Does your original algorithm (and StainlessS's refinement) actually do that?
It seems to me that when slices is false there is some double counting going on. As StainlessS said:
duplicating comparisons nearly all of the time, ie non duplicating only around edges with perimeter with remainder comparisons all done twice
This means that, not only are you duplicating work unnecessarily, but also, because all pixels are not treated equally, you are skewing the results.

[Forensic]

Another way of looking at it is that
Defined(slices) && slices
is equivalent to (and could be replaced by) Default(slices, false)


Does your original algorithm (and StainlessS's refinement) actually do that?
It seems to me that when slices is false there is some double counting going on. As StainlessS said:

This means that, not only are you duplicating work unnecessarily, but also, because all pixels are not treated equally, you are skewing the results.

Gavino: When slices is set to false, every pixel (except perimeter) is analyzed four times (twice horizontally and twice vertically) but each pixel PAIR is only processed once. Since the goal is to calculate the Y differential between adjacent pairs, this method provides the most complete results. Consider the four pairs of pixel at 3,3 when slices is false

---- ---- ---- ----
---- ---- --x- ----
-xx- --xx --x- --x-
---- ---- ---- --x-
---- ---- ---- ----

[Gavino]

each pixel PAIR is only processed once.
I don't think that's true.
In your example, each of the four pairs is processed twice.
For example pixels (3,3) and (3,4) are compared first when x=2 and y=3, then again when x=3 and y=3.
Similarly, (3,3) and (4,3) are compared first when x=3 and y=2, then again when x=3 and y=3.

If all relevant pairs were compared twice, it would be unnecesary work, but would not affect the result.
However, some pairs are compared only once, eg (1,1) and (2,1), or (3,1) and (3,2) - any pair where at least one pixel is on the edge of the processed region.

I think for each (x,y) the algorithm needs to compare that pixel with only at most two other pixels: (x,y+1) and (x+1,y), omitting a comparison when the other pixel is outside the region to be processed.

[Forensic]

Darn!!!! Gavino is correct. The problem is fixed by having either X or Y increment by two when slices is false. Good catch Gavino. Thank you.


Function AvgLumaDif_MOD(Clip c, bool "slices", int "n") { # Suggested implementation, same result as modified logic above
c
step = (Defined(slices) && slices) ? 4 : 1
xstep = (step==4) ? 4 : 2
n = Default(n,0) # default current_frame = 0
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0
GScript("""
for (y=1,H-3,step) {
for (x=1,W-3,xstep) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1)
}
}
count = (step==1) ? ((W-3) * (H-3) *2) : (W * H /4)
value = value / count
""")
return value
}

[Gavino]

The problem is fixed by having either X or Y increment by two when slices is false.
No, that's still not right.
Some pairs are still compared twice (eg (1,2) and (2,2)), while others are not compared at all (eg (2,2) and (3,2)).
As I suggested, the algorithm when step=1 needs to be:
for (y=1,H-3) {
for (x=1,W-3) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
}
}
for (y=1,H-3) { # do last column
value=value + RT_YDifference(c,n,delta=0, x=W-2, y=y, x2=W-2, y2=y+1, w=1, h=1)
}
for (x=1,W-3) { # do last row
value=value + RT_YDifference(c,n,delta=0, x=x, y=H-2, x2=x+1, y2=H-2, w=1, h=1)
}
with count = 2*(W-3)*(H-3) + (H-3) + (W-3)

[StainlessS]

Damn, have already implemented as previous script, he tis (I shall make the mods later)


Function AvgLumaDif_1(Clip c, bool "slices", int "n",int "Matrix") { # Equivalent to Original
c
step= (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
Matrix = Default(Matrix, Height < 600 ? 2 : 3) # REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
value=0.0 count=0.0
GScript("""
for (y=1,Height-step,step) {
for (x=1,Width-step,step) {
Try {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
count=count+1
} Catch (err) {}
}
}
if (count>0) {value=value/(4*count)}
""")
return value
}

Function AvgLumaDif_2(Clip c, bool "slices", int "n",int "Matrix") { # Same logic, Modifed to remove nonsense eg width-step for limit
c
step= (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
Matrix = Default(Matrix, Height < 600 ? 2 : 3) # REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
value=0.0 count=0.0
GScript("""
for (y=1,H-3,step) {
for (x=1,W-3,step) {
Try {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
count=count+1
} Catch (err) {}
}
}
if (count>0) {value=value/(4*count)}
""")
return value
}

Function AvgLumaDif_MOD(Clip c, bool "slices", int "n",int "Matrix") { # Suggested implementation, same result as modified logic above
c
step = (Defined(slices) && slices) ? 4 : 1
n = Default(n,0) # default current_frame = 0
Matrix = Default(Matrix, Height < 600 ? 2 : 3) # REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0
GScript("""
for (y=1,H-3,step) {
for (x=1,W-3,step) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
}
}
count = (step==1) ? ((W-3) * (H-3)) : (W/4 * H/4)
value = value / (4*count)
""")
return value
}

W=512
H=64
T=False
Avisource("D:\avs\test.avi").ConvertToYV12().BilinearResize(W,H)

A=AvgLumaDif_1(T)
B=AvgLumaDif_2(T)
C=AvgLumaDif_MOD(T)
D=AvgLumaDif(T,n=0) # dll defaults to current_frame rather than 0, can change in source.
RT_DebugF ("%f %f %f %f",A,B,C,D)
RT_SubTitle("%f %f %f %f",A,B,C,D)
return last



/*
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/

#include <windows.h>
#include <stdio.h> // dprintf
#include "Avisynth.h"


#define DEFAULT_CURRENT_FRAME // comment out to default n to 0

//#define BUGPRINT // uncomment to print something to debugview


#ifdef BUGPRINT
// Perhaps useful for debug purposes
int dprintf(char* fmt, ...) {
char printString[2048];
char *p=printString;
va_list argp;
va_start(argp, fmt);
vsprintf(p, fmt, argp);
va_end(argp);
for(;*p++;);
--p; // @ null term
if(printString == p || p[-1] != '\n') {
p[0]='\n'; // append n/l if not there already
p[1]='\0';
OutputDebugString(printString);
} else {
OutputDebugString(printString);
}
return p-printString; // strlen printString
}
#endif


#ifdef DEFAULT_CURRENT_FRAME
// helper function
AVSValue GetVar(IScriptEnvironment* env, const char* name) {
try {return env->GetVar(name);} catch (IScriptEnvironment::NotFound) {} return AVSValue();
}
#endif


// AvgLumaDif function
AVSValue __cdecl AvgLumaDif(AVSValue args, void* user_data, IScriptEnvironment* env) {
const char *myName="AvgLumaDif: "; // In case of error, we have pointer to our own name handy

// args is an array of type AVSValue (variant) holding calling args

// Below not necessary as clip is compulsory arg (ie avisynth already made sure clip was supplied)
// if(!args[0].IsClip()) env->ThrowError("%sMust have a source clip",myName);

PClip child = args[0].AsClip(); // Clip, compulsory arg, no default
const bool slices = args[1].AsBool(false); // slices, default false

int n;

// Allows selection of default for n,current_frame or 0

#ifdef DEFAULT_CURRENT_FRAME
if(args[2].IsInt()) {n = args[2].AsInt(); } // Frame n
else {
AVSValue cn = GetVar(env,"current_frame");
if (!cn.IsInt()) env->ThrowError("%s'current_frame' only available in runtime scripts",myName);
n = cn.AsInt(); // current_frame
}
#else
n = args[2].AsInt(0); // n, default 0
#endif

// Get reference to VideoInfo
const VideoInfo &vi = child->GetVideoInfo();

const int matrix = args[3].AsInt(vi.height<600?2:3); // matrix, default PC range, 601/709 dependent upon height
// Matrix: REC601 : 1=REC709 : 2 = PC601 : 3 = PC709

#ifdef BUGPRINT
// Output something interesting when debugging
dprintf("%s slices=%s n=%d matrix=%d",myName,slices?"True":"False",n,matrix);
#endif

// width=0 means no video, most plugins dont bother with below check (I dont either usually, but probably should)
if(vi.num_frames <= 0 || vi.width==0) env->ThrowError("%sClip must have video",myName);

n = (n < 0) ? 0 : (n >= vi.num_frames) ? vi.num_frames - 1 : n; // range limit n to valid frames

const int step = (slices) ? 4 : 1;

// Ignore RHS odds and ends, FULL 4x4 pixel blocks only
const int W = vi.width & (~0x03); // clear bits 0 and 1, ie make mod 4
const int H = vi.height & (~0x03);
if(W<=0 || H <=0) env->ThrowError("%sIllegal Frame Size (at least 4x4)",myName);

PVideoFrame src = child->GetFrame(n,env); // pointer to video frame
const int pitch = src->GetPitch(PLANAR_Y); // PLANAR_Y no effect on non-Planar (equates to 0)
const BYTE *srcp = src->GetReadPtr(PLANAR_Y);
const int ystride = pitch*step; // how much to step data pointer for each y iteration

int dif = 0; // init, dif sum as int (not double)

const int x_end=W-3, y_end=H-3; // end limits
int x,y;
if(vi.IsPlanar()) {
srcp += pitch; // y=1
for(y=1; y <= y_end; y += step) {
for(x=1; x <= x_end; x += step) { // x step appropriate for slices
const int a=srcp[x]; // x , y
const int b=srcp[x+1]; // x+1, y
const int c=srcp[x+pitch]; // x , y+1
const int d=srcp[x+1+pitch]; // x+1, y+1
dif += abs(a-b); // x,y <-> x+1,y
dif += abs(a-c); // x,y <-> x,y+1
dif += abs(b-d); // x+1,y <-> x+1,y+1
dif += abs(c-d); // x,y+1 <-> x+1,y+1
}
srcp += ystride; // y step appropriate for slices
}
} else if(vi.IsYUY2()) {
srcp += pitch; // y=1
const int xstride = step * 2; // double up for YUY2 stepping (*2)
const int x_lim = x_end * 2; // luma samples 2 apart for YUY2
for(y=1; y <= y_end; y += step) {
for(x=2; x <= x_lim; x += xstride) { // x=2 is offset to YUY2(1).Y
const int a=srcp[x]; // x , y
const int b=srcp[x+2]; // x+1, y
const int c=srcp[x+pitch]; // x , y+1
const int d=srcp[x+2+pitch]; // x+1, y+1
dif += abs(a-b); // x,y <-> x+1,y
dif += abs(a-c); // x,y <-> x,y+1
dif += abs(b-d); // x+1,y <-> x+1,y+1
dif += abs(c-d); // x,y+1 <-> x+1,y+1
}
srcp += ystride; // y step appropriate for slices
}
} else if(vi.IsRGB()) {
// only bother to check matrix if RGB
if(matrix < 0 || matrix > 3) env->ThrowError("%sIllegal Matrix %d (0->3)",myName,matrix);
// RGB to YUV-Y Conversion
// Matrix: REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
double Kr,Kb;
int Sy,offset_y;
if(matrix & 0x01) {Kr = 0.2126; Kb = 0.0722;} // 709 1 or 3
else {Kr = 0.2990; Kb = 0.1140;} // 601 0 or 2
if(matrix & 0x02) {Sy = 255 ; offset_y = 0;} // PC 2 or 3
else {Sy = 219 ; offset_y = 16;} // TV 0 or 1
const int shift = 15;
const int half = 1 << (shift - 1);
const double mulfac = double(1<<shift);
double Kg = 1.0 - Kr - Kb;
const int Srgb = 255;
const int Yb = int(Sy * Kb * mulfac / Srgb + 0.5); //B
const int Yg = int(Sy * Kg * mulfac / Srgb + 0.5); //G
const int Yr = int(Sy * Kr * mulfac / Srgb + 0.5); //R
const int OffyPlusHalf = (offset_y<<shift) + half;
//
// RGB is weird upside down frame
srcp += ((vi.height-1) * pitch); // Upside down RGB, height-1 is top line (y=0)
srcp -= pitch; // y=1, (could of course be combined with above, NOTE subtract)
const int xstep = (vi.IsRGB24()) ? 3 : 4;
const int xstride = step * xstep; // scale step for RGB24/32
const int x_lim = x_end * xstep; // scale x_end for RGB24/32
for(y=1; y <= y_end; y += step) {
for(x=xstep; x <= x_lim;x += xstride) { // x stride appropriate for slices
int a = (srcp[x] * Yb + srcp[x+1] * Yg + srcp[x+2] * Yr + OffyPlusHalf) >> shift; // x ,y
int x2= x + xstep;
int b = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x+1, y
x2= x - pitch; // NOTE - pitch
int c = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x ,y+1
x2= x + xstep - pitch; // NOTE - pitch
int d = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x+1,y+1
dif += abs(a - b); // x,y <-> x+1,y
dif += abs(a - c); // x,y <-> x,y+1
dif += abs(b - d); // x+1,y <-> x+1,y+1
dif += abs(c - d); // x,y+1 <-> x+1,y+1
}
srcp -= ystride; // y step appropriate for slices, NOTE subtract
}
} else {
env->ThrowError("%sUnknown Colorspace",myName);
}

int count = (!slices) ? ((W-3) * (H-3)) : (W/4 * H/4);
double value = double(dif) / (4*count); // (4 * count) will be coerced to double during division

AVSValue ret = value; // create type variant AVSValue and init with double (converted to float for AVSValue)
return ret;
// return value; // This would also work, value is implicitly converted to type AVSValue
}

// Avisynth v2.5 plugin initializer, requires AVISYNTH_INTERFACE_VERSION 3 Avisynth.h
extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
// Tell Aviynth name and args and which function to call
env->AddFunction("AvgLumaDif", "c[slices]b[n]i[matrix]i", AvgLumaDif, 0);
return "`AvgLumaDif' AvgLumaDif plugin"; // A freeform name of the plugin.
}


Added Matrix arg as RGB capable.
Defaulted n to current_frame, changeable in source.

About 50FPS on UT_Video (EDIT: YV12) Compressed AVI DVD Size
EDIT: Above 50FPS in error, should have been higher.

[Forensic]

Gavino: I created a mess by trying to handle both states of "slices" in the same loop (sorry StainlessS). From how I read your version:
If slices is TRUE then your method only gets half of four possible central pixel pairs per 4x4 pixel block, and the last two loops (last row & last column) cross through all the slice boundaries. If slices is FALSE then did you mean -2 and not -3 for the last two loops?

I think that this version solves everything.


Function AvgLumaDif_MOD(Clip c, bool "slices", int "n") { # Suggested implementation, same result as modified logic above
c
n = Default(n,0) # default current_frame = 0
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0
GScript("""
if (Defined(slices) && slices) {
for (y=1,H-3,4) {
for (x=1,W-3,4) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1)
}
}
count = W * H /4
}
else {
for (y=1,H-2,1) {
for (x=1,W-2,1) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
}
}
value = value // now deal with the bottom right non-perimeter pixel
\ + RT_YDifference(c,n,delta=0, x=W-1, y=H-1, x2=W-1, y2=H-2, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=W-1, y=H-1, x2=W-2, y2=H-1, w=1, h=1)
count = (W-2) * (H-2) *2
}
value = value / count
""")
return value
}

[StainlessS]

Post #1 of 2

Forensic, think big G is correct, have implemented so.


/*
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
*/

#include <windows.h>
#include <stdio.h> // dprintf
#include "Avisynth.h"

#define DEFAULT_CURRENT_FRAME // comment out to default n to 0

//#define BUGPRINT // uncomment to print something to debugview


#ifdef BUGPRINT
// Perhaps useful for debug purposes
int dprintf(char* fmt, ...) {
char printString[2048];
char *p=printString;
va_list argp;
va_start(argp, fmt);
vsprintf(p, fmt, argp);
va_end(argp);
for(;*p++;);
--p; // @ null term
if(printString == p || p[-1] != '\n') {
p[0]='\n'; // append n/l if not there already
p[1]='\0';
OutputDebugString(printString);
} else {
OutputDebugString(printString);
}
return p-printString; // strlen printString
}
#endif

#ifdef DEFAULT_CURRENT_FRAME
// helper function
AVSValue GetVar(IScriptEnvironment* env, const char* name) {
try {return env->GetVar(name);} catch (IScriptEnvironment::NotFound) {} return AVSValue();
}
#endif

// AvgLumaDif function
AVSValue __cdecl AvgLumaDif(AVSValue args, void* user_data, IScriptEnvironment* env) {
const char *myName="AvgLumaDif: "; // In case of error, we have pointer to our own name handy

// args is an array of type AVSValue (variant) holding calling args

// Below not necessary as clip is compulsory arg (ie avisynth already made sure clip was supplied)
// if(!args[0].IsClip()) env->ThrowError("%sMust have a source clip",myName);

PClip child = args[0].AsClip(); // Clip, compulsory arg, no default
const bool slices = args[1].AsBool(false); // slices, default false

int n;

// Allows selection of default for n,current_frame or 0

#ifdef DEFAULT_CURRENT_FRAME
if(args[2].IsInt()) {n = args[2].AsInt(); } // Frame n
else {
AVSValue cn = GetVar(env,"current_frame");
if (!cn.IsInt()) env->ThrowError("%s'current_frame' only available in runtime scripts",myName);
n = cn.AsInt(); // current_frame
}
#else
n = args[2].AsInt(0); // n, default 0
#endif

// Get reference to VideoInfo
const VideoInfo &vi = child->GetVideoInfo();

const int matrix = args[3].AsInt(vi.height<600?2:3); // matrix, default PC range, 601/709 dependent upon height
// Matrix: REC601 : 1=REC709 : 2 = PC601 : 3 = PC709

#ifdef BUGPRINT
// Output something interesting when debugging
dprintf("%s slices=%s n=%d matrix=%d",myName,slices?"True":"False",n,matrix);
#endif

// width=0 means no video, most plugins dont bother with below check (I dont either usually, but probably should)
if(vi.num_frames <= 0 || vi.width==0) env->ThrowError("%sClip must have video",myName);

n = (n < 0) ? 0 : (n >= vi.num_frames) ? vi.num_frames - 1 : n; // range limit n to valid frames

const int step = (slices) ? 4 : 1;

// Ignore RHS odds and ends, FULL 4x4 pixel blocks only
const int W = vi.width & (~0x03); // clear bits 0 and 1, ie make mod 4
const int H = vi.height & (~0x03);
if(W<=0 || H <=0) env->ThrowError("%sIllegal Frame Size (at least 4x4)",myName);

PVideoFrame src = child->GetFrame(n,env); // pointer to video frame
const int pitch = src->GetPitch(PLANAR_Y); // PLANAR_Y no effect on non-Planar (equates to 0)
const BYTE *srcp = src->GetReadPtr(PLANAR_Y);
const int ystride = pitch*step; // how much to step data pointer for each y iteration

int dif = 0; // init, dif sum as int (not double)
const int x_end=W-3, y_end=H-3; // end limits
int x,y;
if(vi.IsPlanar()) {
srcp += pitch; // y=1
if(slices) {
for(y=1; y <= y_end; y += 4) {
for(x=1; x <= x_end; x += 4) { // x step appropriate for slices
const int tl=srcp[x]; // x , y
const int tr=srcp[x+1]; // x+1, y
const int bl=srcp[x+pitch]; // x , y+1
const int br=srcp[x+1+pitch]; // x+1, y+1
dif += abs(tl-tr); // x,y <-> x+1,y
dif += abs(tl-bl); // x,y <-> x,y+1
dif += abs(tr-br); // x+1,y <-> x+1,y+1
dif += abs(bl-br); // x,y+1 <-> x+1,y+1
}
srcp += ystride; // y step appropriate for slices
}
} else {
for(y=1; y <= y_end; ++y) {
for(x=1; x <= x_end; ++x) { // x step appropriate for slices
const int tl=srcp[x]; // x , y
const int tr=srcp[x+1]; // x+1, y
const int bl=srcp[x+pitch]; // x , y+1
dif += abs(tl-tr); // x,y <-> x+1,y
dif += abs(tl-bl); // x,y <-> x,y+1
}
// rhs column, x = W - 2
const int tl=srcp[x]; // x = W - 2, y
const int bl=srcp[x+pitch]; // x = W - 2, y+1
dif += abs(tl-bl);
srcp += ystride; // y step appropriate for slices
}
// y = H - 2
for(x=1; x <= x_end; ++x) { // bottom row
const int tl=srcp[x]; // x , y = H - 2
const int tr=srcp[x+1]; // x+1, y = H - 2
dif += abs(tl-tr);
}
}
} else if(vi.IsYUY2()) {
srcp += pitch; // y=1
const int x_lim = x_end * 2; // luma samples 2 apart for YUY2
if(slices) {
for(y=1; y <= y_end; y += 4) {
for(x=2; x <= x_lim; x += (4*2)) { // x=2 is offset to YUY2(1).Y
const int tl=srcp[x]; // x , y
const int tr=srcp[x+2]; // x+1, y
const int bl=srcp[x+pitch]; // x , y+1
const int br=srcp[x+2+pitch]; // x+1, y+1
dif += abs(tl-tr); // x,y <-> x+1,y
dif += abs(tl-bl); // x,y <-> x,y+1
dif += abs(tr-br); // x+1,y <-> x+1,y+1
dif += abs(bl-br); // x,y+1 <-> x+1,y+1
}
srcp += ystride; // y step appropriate for slices
}
} else {
for(y=1; y <= y_end; ++y) {
for(x=2; x <= x_lim; x += 2 ) { // x=2 is offset to YUY2(1).Y
const int tl=srcp[x]; // x , y
const int tr=srcp[x+2]; // x+1, y
const int bl=srcp[x+pitch]; // x , y+1
dif += abs(tl-tr); // x,y <-> x+1,y
dif += abs(tl-bl); // x,y <-> x,y+1
}
// rhs column, x = W - 2
const int tl=srcp[x]; // x = W - 2, y
const int bl=srcp[x+pitch]; // x = W - 2, y+1
dif += abs(tl-bl);
srcp += ystride; // y step appropriate for slices
}
// y = H - 2
for(x=2; x <= x_lim; x+=2) { // bottom row
const int tl=srcp[x]; // x , y = H - 2
const int tr=srcp[x+2]; // x+1, y = H - 2
dif += abs(tl-tr);
}
}
} else if(vi.IsRGB()) {
// only bother to check matrix if RGB
if(matrix < 0 || matrix > 3) env->ThrowError("%sIllegal Matrix %d (0->3)",myName,matrix);
// RGB to YUV-Y Conversion
// Matrix: REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
double Kr,Kb;
int Sy,offset_y;
if(matrix & 0x01) {Kr = 0.2126; Kb = 0.0722;} // 709 1 or 3
else {Kr = 0.2990; Kb = 0.1140;} // 601 0 or 2
if(matrix & 0x02) {Sy = 255 ; offset_y = 0;} // PC 2 or 3
else {Sy = 219 ; offset_y = 16;} // TV 0 or 1
const int shift = 15;
const int half = 1 << (shift - 1);
const double mulfac = double(1<<shift);
double Kg = 1.0 - Kr - Kb;
const int Srgb = 255;
const int Yb = int(Sy * Kb * mulfac / Srgb + 0.5); //B
const int Yg = int(Sy * Kg * mulfac / Srgb + 0.5); //G
const int Yr = int(Sy * Kr * mulfac / Srgb + 0.5); //R
const int OffyPlusHalf = (offset_y<<shift) + half;
//
// RGB is weird upside down frame
srcp += ((vi.height-1) * pitch); // Upside down RGB, height-1 is top line (y=0)
srcp -= pitch; // y=1, (could of course be combined with above, NOTE subtract)
const int xstep = (vi.IsRGB24()) ? 3 : 4;
const int xstride = step * xstep; // scale step for RGB24/32
const int x_lim = x_end * xstep; // scale x_end for RGB24/32
if(slices) {
for(y=1; y <= y_end; y += step) {
for(x=xstep; x <= x_lim;x += xstride) { // x stride appropriate for slices
int tl = (srcp[x] * Yb + srcp[x+1] * Yg + srcp[x+2] * Yr + OffyPlusHalf) >> shift; // x ,y
int x2= x + xstep;
int tr = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x+1, y
x2= x - pitch; // NOTE - pitch
int bl = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x ,y+1
x2= x + xstep - pitch; // NOTE - pitch
int br = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x+1,y+1
dif += abs(tl - tr); // x,y <-> x+1,y
dif += abs(tl - bl); // x,y <-> x,y+1
dif += abs(tr - br); // x+1,y <-> x+1,y+1
dif += abs(bl - br); // x,y+1 <-> x+1,y+1
}
srcp -= ystride;
}
} else {
for(y=1; y <= y_end; y += step) {
for(x=xstep; x <= x_lim;x += xstride) { // x stride appropriate for slices
int tl = (srcp[x] * Yb + srcp[x+1] * Yg + srcp[x+2] * Yr + OffyPlusHalf) >> shift; // x ,y
int x2= x + xstep;
int tr = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x+1, y
x2= x - pitch; // NOTE - pitch
int bl = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x ,y+1
dif += abs(tl - tr); // x,y <-> x+1,y
dif += abs(tl - bl); // x,y <-> x,y+1
}
// rhs column, x = W - 2
const int tl = (srcp[x] * Yb + srcp[x+1] * Yg + srcp[x+2] * Yr + OffyPlusHalf) >> shift; // x = W - 2, y
int x2= x - pitch; // NOTE - pitch
const int bl = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x = W - 2, y+1
dif += abs(tl-bl);
srcp -= ystride; // y step appropriate for slices, NOTE subtract
}
// y = H - 2
for(x=xstep; x <= x_lim; x+=xstride) { // bottom row
const int tl = (srcp[x] * Yb + srcp[x+1] * Yg + srcp[x+2] * Yr + OffyPlusHalf) >> shift; // x , y = H - 2
int x2= x + xstep;
const int tr = (srcp[x2] * Yb + srcp[x2+1] * Yg + srcp[x2+2] * Yr + OffyPlusHalf) >> shift; // x , y = H - 2
dif += abs(tl-tr);
}
}
} else {
env->ThrowError("%sUnknown Colorspace",myName);
}
int count = (slices) ? (W/4 * H/4) * 4 : 2*((W-3)*(H-3)) + (H-3) + (W-3);
double value = double(dif) / count; // count will be coerced to double during division

AVSValue ret = value; // create type variant AVSValue and init with double (converted to float for AVSValue)
return ret;
// return value; // This would also work, value is implicitly converted to type AVSValue
}

// Avisynth v2.5 plugin initializer, requires AVISYNTH_INTERFACE_VERSION 3 Avisynth.h
extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
// Tell Aviynth name and args and which function to call
env->AddFunction("AvgLumaDif", "c[slices]b[n]i[matrix]i", AvgLumaDif, 0);
return "`AvgLumaDif' AvgLumaDif plugin"; // A freeform name of the plugin.
}

[StainlessS]

Post #2 of 2

Test script

Function AvgLumaDif_1(Clip c, bool "slices", int "n",int "Matrix") { # count, Check version
c
slices = Default(slices,False)
n = Default(n,0) # default current_frame = 0
Matrix = Default(Matrix, Height < 600 ? 2 : 3) # REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0 count = 0
GScript("""
if(slices) {
for (y=1,H-3,4) {
for (x=1,W-3,4) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
count = count + 4
}
}
} else {
for (y=1,H-3) {
for (x=1,W-3) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
count = count + 2
}
}
for (y=1,H-3) { # do last column
value=value + RT_YDifference(c,n,delta=0, x=W-2, y=y, x2=W-2, y2=y+1, w=1, h=1)
count = count + 1
}
for (x=1,W-3) { # do last row
value=value + RT_YDifference(c,n,delta=0, x=x, y=H-2, x2=x+1, y2=H-2, w=1, h=1)
count = count + 1
}
}
value = value / (count)
""")
return value
}

Function AvgLumaDif_MOD(Clip c, bool "slices", int "n",int "Matrix") { # dll check version
c
slices = Default(slices,False)
n = Default(n,0) # default current_frame = 0
Matrix = Default(Matrix, Height < 600 ? 2 : 3) # REC601 : 1=REC709 : 2 = PC601 : 3 = PC709
W = (Width / 4) * 4 # Ignore RHS odds and ends, FULL 4x4 pixel blocks only
H = (Height / 4) * 4 #
Assert(W>0 && H>0,"AvgLumaDif_MOD: Illegal Frame Size")
value=0.0
GScript("""
if(slices) {
for (y=1,H-3,4) {
for (x=1,W-3,4) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x+1, y=y, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
\ + RT_YDifference(c,n,delta=0, x=x, y=y+1, x2=x+1, y2=y+1, w=1, h=1,matrix=Matrix)
}
}
count = (W/4 * H/4) * 4
} else {
for (y=1,H-3) {
for (x=1,W-3) {
value=value
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1)
\ + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1)
}
value=value + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x, y2=y+1, w=1, h=1) # x = W - 2, Last column
}
for (x=1,W-3) {
value=value + RT_YDifference(c,n,delta=0, x=x, y=y, x2=x+1, y2=y, w=1, h=1) # y = H - 2, Last row
}
count = 2*((W-3)*(H-3)) + (H-3) + (W-3)
}
value = value / count
""")
return value
}

W=512
H=64
T=True
Avisource("D:\avs\test.avi").ConvertToYUY2().BilinearResize(W,H)
A=AvgLumaDif_1(T)
B=AvgLumaDif_MOD(T)
C=AvgLumaDif(T,n=0) # dll defaults to current_frame rather than 0, can change in source.
RT_DebugF ("%f %f %f",A,B,C)
RT_SubTitle("%f %f %f",A,B,C)
return last


Previous FPS in error, should be about 200+ FPS in ScriptClip test script.

EDIT: Slices=False: Have used value of for() loop variable on exit from loop, ie end condition value + 1

[Forensic]

Gavino & StainlessS are you sure that "for (y=1,H-3) { # do last column" and "for (x=1,W-3) { # do last row" shouldn't use "2" instead of "3" to actually work on the last row/column (ignoring the perimeter)?

[StainlessS]

Yep, W-1 is last pixel, W-2 leaves 1 pixel perimeter untouched (same as at start).

EDIT: On say the bottom row, it starts column 1 to 2 dif, goes on to column W-3 to W-2 dif, leaving untouched W-1 pixel
(W pixel not exist, ie range 0 -> W-1).
W-3 end condition position, differences W-3 -> W-2.

[Forensic]

Okay, so then it seems I am right that the "for (y=1,H-3) { # do last column" and "for (x=1,W-3) { # do last row" lines SHOULD both use "2" not "3" to get the last non-perimeter row and column for when slices=false. Thank you both for your help with this. StainlessS, now that mediafire.com/StainlessS is no longer valid, where/when is the new RT_Stats build with this function?

[Gavino]

If slices is TRUE then your method only gets half of four possible central pixel pairs per 4x4 pixel block, and the last two loops (last row & last column) cross through all the slice boundaries.
The method I described was intended only for the case slices=false, since that's the case that was wrong; it was not meant to be the complete code.

Okay, so then it seems I am right that the "for (y=1,H-3) { # do last column" and "for (x=1,W-3) { # do last row" lines SHOULD both use "2" not "3" to get the last non-perimeter row and column for when slices=false.
No, that's not right - you are making a fencepost error (http://en.wikipedia.org/wiki/Fencepost_error#Fencepost_error).
Sure, the pixels of the last (relevant) column have x=W-2 and y running from 1 to H-2.
However, there are only H-3 comparisons, with each pixel in the range y=1 to H-3 being compared with the one below it.
Similarly, in the last row, the comparisons run from x=1 to W-3.

[Forensic]

Gavino, Thank you so much for your wisdom and correcting the errors of my way.
StainlessS, Please let me know where to find the compiled version once it is available. I do have the ability to compile it myself, but discovered that anything I compile using VC++ 2012 will only work on computers that have that installed. An unlikely scenario for my forensic peers. Besides, it seems you are building this into RT_Stats.

[Guest]

anything I compile using VC++ 2012 will only work on computers that have that installed Do some research on runtime library linkage, it doesn't have to be that way.

[Forensic]

Will do. As always, thank you neuron2.

[Guest]

Simplest:

In Properties/C&C++/Code generation, set runtime library to /MT, not /MD.

[StainlessS]

now that mediafire.com/StainlessS is no longer valid, where/when is the new RT_Stats build with this function?

Sorry bout dat, have submitted problem report to MediaFire, was not aware of the Page Not Found error. They changed page layout recently (mediafire) and probably missing since then.
Will add to RT when established to be as required.

[Forensic]

neuron2: THANK YOU!!!!!!
For others looking to do the same in VS Express 2012, you press Alt-F7 and choose "Configuration Properties" - "C/C++" - "Code Generation" and then change the "RunTime Library" to "Multi-threaded Debug (/MTd)". I tested this by compiling the source code of several existing valid AviSynth codes and they all work without needing any special pre-installed software (at least from my limited testing). The only caveat is that the builds are larger, but that may be as a result of the windows.h & stdio.h VS Express uses -or- the use of Debug option in the RunTime library (which appears to be required if you VS2012 is to build a valid DLL) . Now I just need to learn proper C++ coding.

[StainlessS]

only caveat is that the builds are larger

Has 'static' built in library routines rather than external runtime dll.

[Forensic]

Has 'static' build in library routines rather than external runtime dll.

Is that a functional problem (other than size)?

[Guest]

Is that a functional problem (other than size)? No, he's just explaining why it is larger.

For your released filters you will want to build in release mode, not debug mode. That will affect size, as you speculated, but also performance. So select /MT for your release configuration, rather than /MTd.

You don't actually have to get a lot of C++ and object orientation mastered. Just get source for a filter similar to what you want to make and start modifying it. Then you get more into a C style of coding, e.g., just modifying GetFrame(), etc. It's not really that difficult.

[Forensic]

StainlessS. I compiled and am using your CPP file. Is your code based on Gavino's version (he was correct)?

[StainlessS]

StainlessS. I compiled and am using your CPP file. Is your code based on Gavino's version (he was correct)?

Yep, with a couple of minor shortcuts. See accompanying avs script, dll version script function.
I incorporated the end column into the Y loop, immediately after the X loop exit, where x var is W-3, + 1, ie W-2 (and y from the Y loop) , instead of doing it in a loop of it's own. Did same sort of thing after the Y loop for the last row using y loop counter value eg H-3 + 1, ie H-2.

EDIT: The Count check version script function is is pretty much the intended Gavino full script (G only gave bare bones slice =false section),
but with the counts done separately with each iteration. dll check script version implements pretty much exactly same as dll. Different
versions just to make sure all versions produce identical results. The speed difference of the 'shortcuts' would not be of any significance
in script but would be a little quicker in the dll cpp func.

[Forensic]

Thank you. I will pass this along to my peers.

[martin53]

Forensic,
I am interested to know which development environment you chose - I assume Visual Studio 2012 or 2013 - and if you were able to rebuild other plugins 1:1

[StainlessS]

@Forensic,
Do you think that AvgLumaDif(slices=false) / AvgLumaDif(slices=true) [or similar] could be used as some kind of
measure of 'blockiness' ?

EDIT: Also, further to M53 question: Think Avisynth+ decided upon VS2010 as standard compiler for both Avisynth+
and plugins. Think 2012 only compilable on W7+ (maybe vista but who cares bout dat). Also think VS2010 compilable
on XP.

EDIT: In your other thread, I said that converting avs script to C took little more than liberally scattering
semicolons into source, I was mostly referring to the script samples in RT_Stats AVS/Converted2Plugins/ folder,
if compared, you should see that there was not much else that was needed.

EDIT: Reading/writing frames directly rather than using some other function to do it would be somewhat different
to the script versions. The Converted2Plugins funcs call other RT_Stats funcs rather than accessing frames
directly.

[Forensic]

@martin53: I use "Microsoft Visual Studio 2012 Express" and also installed "Microsoft Visual C++ Toolkit 2003" and "Microsoft Platform SDK for Windows Server 2003 R2". Just be sure to set your Windows path to include the Express BIN folder. When you want to build a DLL, press ALT-F7 in Express and the 'Runtime Library' to "MTd" (located under Configuration Properties - C/C++ - Code Generation). It works like a charm. NOTE: I tried MT but the builds fail, so I use MTd which makes them slightly larger as they include the header.

@StainlessS: If you compare the false to true usage, significant value differences does IMPLY blockiness, but you can't set a standard threshold due to all the video variances (at least in the security surveillance field). Your concept may work for hi-res movies/video, but I have no quantitative values on that.

[Groucho2004]

I use "Microsoft Visual Studio 2012 Express" and also installed "Microsoft Visual C++ Toolkit 2003" and "Microsoft Platform SDK for Windows Server 2003 R2". Just be sure to set your Windows path to include the Express BIN folder. When you want to build a DLL, press ALT-F7 in Express and the 'Runtime Library' to "MTd" (located under Configuration Properties - C/C++ - Code Generation). It works like a charm. NOTE: I tried MT but the builds fail, so I use MTd which makes them slightly larger as they include the header.

Are you using the VC2012 with the 2K3R2 SDK? Why did you install VC7.1 when you use VC2012?
It also would be helpful if you told us why the release builds fail.

Only being able to build debug versions is not what I'd call "working like a charm". :rolleyes:

[StainlessS]

Just be sure to set your Windows path to include the Express BIN folder

IDE uses 1st compiler that it comes across in the list of paths to BIN, compiler uses 1st header that it comes across in list of INCLUDE directories that have the particular header file, same for LIB libraries when linking. So, order is important, set the SDK paths 1st in lists, then the path to the desired compiler eg VS2012 or TKit 2003 (EDIT: also for include and libs when switching compiler) .
If you cannot create a non debug version, it may be a mismatch in paths order for Bin/Include/Libs.

EDIT: TKit 2003, does not have debug libraries.

[Forensic]

To me "Worked like a charm" referred to the fact I could finally build working AviSynth DLLs, even if they were significantly larger. Obviously, I prefer to make my builds fully compliant and appreciate any guidance. My environmental paths start with:
C:\Program Files\Microsoft SDKs\Windows\v7.1A\Include;C:\Program Files\Microsoft Visual Studio 11.0\VC\bin;
When I try to use MT (versus MTd) I get two errors {I used StainlessS's RGBamplifier source code for the test since I understand its behavior and can easily test the results}
Error 1 error LNK2019: unresolved external symbol __CrtDbgReportW referenced in function "public: class AVSValue const & __thiscall AVSValue:: operator[](int)const " (??AAVSValue@@QBEABV0@H@Z) c:\...\RgbAmplifier.obj RGBamplifier
Error 2 error LNK1120: 1 unresolved externals c:\...\Debug\RGBamplifier.dll 1 1 RGBamplifier
NOTE: The (...) are just to not annoy you with the full directory paths

[Groucho2004]

To me "Worked like a charm" referred to the fact I could finally build working AviSynth DLLs, even if they were significantly larger. Obviously, I prefer to make my builds fully compliant and appreciate any guidance.
For full compliance you would have to use VC6 unless you're using Avisynth+.
Looking at your error message it seems that you configured your project file to output a Unicode binary which is not supported by Avisynth.

[Forensic]

For full compliance you would have to use VC6 unless you're using Avisynth+.
Looking at your error message it seems that you configured your project file to output a Unicode binary which is not supported by Avisynth.
I recognize that the MT build does not work or even build (you identified the cause as Unicode), but the MTd version does both. I would happily use VC6, but can not find a copy. Do you know of an active download link? If I cannot get access to VC6, then I have no choice but to use what I have.

[Groucho2004]

I recognize that the MT build does not work or even build (you identified the cause as Unicode), but the MTd version does both. I would happily use VC6, but can not find a copy. Do you know of an active download link? If I cannot get access to VC6, then I have no choice but to use what I have.
The compatibility issues are mostly about different exception handling introduced in newer compilers. For example, it might just crash instead of throwing a nice error message before gracefully terminating.
There's a good chance that you'll never have any problems using VC11 for your plugin.
Please have a look at this (http://www.avisynth.nl/index.php/Filter_SDK/Compiling_instructions) page, it should give you enough info to build your DLL. The instructions for VS2008 should be more or less valid for VS2010/2012.

[TurboPascal7]

I would happily use VC6, but can not find a copy.
Don't. Just don't. Visual studio 2012 is a nice and correct way of building avisynth plugins as long as you handle all exceptions yourself and don't do dumb stuff no one is doing anyway. Don't even think about using VC6 in almost 2014.

[StainlessS]

we luddites would suggest using least modern that you can get away with :)

[Forensic]

Build->Configuration Manager, select "Release" in the left drop-down
Configuration Properties, General, Character Set "Multi-Byte Character Set"
Configuration Properties, General, Configuration Type "Dynamic Library (.dll)"

Now my builds are MT, work, and are very very small. :) Thank you.

[jmac698]

ok I can't get planetcrop to work :(
http://www.sendspace.com/file/h3x698

It just says planet can't be found, but it's very distinct, about 20 pixels across, luma from 30 to 160, and the background is under 20.

Show the detect clip it's very clear, the first frame is from x=380 to x=401

edit: I think the problem is that a few frames have no planet. Can you make it so that those frames are ignored?

There was another problem, a few frames were very large and made the whole sequence large, I'd like those ignored also. So ideally, find the most consistent size and use that.

[StainlessS]

Well I dont know what I was supposed to do with that bunch of pics.
The 3rd one does not look like it was taken on same night nor location in the sky, and has not just 1 but at least 3
luminous bodies (non of them being the body in the other pics) spanning almost entire frame width.
The forth pic looks like it could be the same planet as in 1st two pics, but definitely not in time lapse sequence.
You pic these pics out at random ?
If you delete/remove the 3rd pic (from another time/place) and just process the remaining 3 it works quite fine.
For the first few frames that have no planet (just black sky, not included in zip), suggest you delete them :)

EDIT: Actually, I'm quite amazed at how well it works when you remove the unrelated frame.

Curious, what was it (planet) ?

EDIT:
Jmac, I dont actually expect you to [rule 4], but you never did acknowledge receipt of previous requested
mods to PlanetCrop, nor to eg ShowChannels where I spent about a month or more banging my head against a wall
and despite pointing out the existence of mods got zero response. In that situation, all I can think is I wasted
my time (and a lot of it, not to mention the bruises). All you have to do to placate me is say yes, I saw the post
then I am at least aware that you have seen it (whether you thought it well accepted, or not).
I am quite frankly not in the mood to spend time especially when I end up not knowing whether it was completely
wasted or not ("I saw it, but it was shite." would be not ideal but appreciated).

P.S., if you did wanna [rule 4]

[jmac698]

Yep you're right, if I manually remove bad frames it works quite awesome, and I must say good job because in an animation, it looks virtually steady. They are all in the same sequence, but here's what happens: I bump the tripod, making it appear in a different location, or it moves out of frame after a few minutes. As I try to re-center it, I might catch a bit of something else like street lights.
I think most of these can be easily removed if I look through the beginning and end, so it's not a big problem after all.

I managed to process 85 frames to a noise-free image, unfortunately it was just a smooth blob. It's supposed to be Jupiter, but I'm thinking I pointed at a bright star by mistake, as there's no detail.

I'll try it on a quarter moon and see if it can center that.

It's quite amazing really and thanks for making it :)

[StainlessS]

I also made an edit Jmac, forgive me, bit pissed off at the moment.

[wiseant]

@StainlessS

Hi StainlessS

I said it before and I'll say it again:

You have to be one of the more underappreciated members here [how ironic underappreciated is deemed to be misspelled or not a recognized word].

What I see is someone who so generously gives their time to anyone that asks for your assistance. I wish that I had more time to look at all the neat stuff that you have created. I hope that you don't get discouraged and that you will keep on keeping on. The amount of information you have on MediaFire since the start of this year is amazing.

[vampiredom]

I agree 100%. Thank you, StainlessS.

[Forensic]

Your contributions have allowed my forensic peers to have customizable video enhancement resources beyond the limitations of industry specific software. And, as a mentor, you have helped me advance from scripting to DLL coding, so that I can pay it forward. You help anyone in need and never ask for anything in return. I believe that is the definition of a hero. Thank you.

[StainlessS]

Way too many sycophants on this site, fortunately Jmac aint one of them :)

Will not be active until mods to RT_ Array + DBase completed.

[wiseant]

Hmm . . . surely you jest - you get three supportive comments - not insincere nor excessive and certainly not self-serving

However, if you can, StainlessS, please try to find another place to upload your files - the pop-ups are horrendous

[StainlessS]

v1.30, 19 Dec 2013, Added DBase Functions, Increased Array attributes to 256, improved error messages.

See mediafire in sig

Script ReDAR_DGIT.avs, may be of some use:-


/*
ReDAR_DGIT.avs v0.1 - DGIndex AVS Template - By StainlessS.
Requires GSCript, RT_Stats v1.30+, RoboCrop, TIVTC, FFT3DFilter, NicAudio plugins.
Requires MediaInfo CLI version.

Progressive, or NTSC FILM with soft pulldown.
Purpose:- Crop letterboxing, with additional crop/resize to LESS wide screen DAR, CANNOT make more wide (DAR = Display Aspect Ratio).
Useful to crop to eg 16:9 on 16:9 hand held device.
Can also resize to SAR 1:1 [Sample (pixel) Aspect Ratio] for devices/players that cannot deal with non 1:1 clips.
Can Also call TFM and TDecimate to remove soft pulldown from NTSC FILM when Honour Pulldown Flags was used in DGIndex.
Signals resultant DAR for MeGUI in Global vars MeGUI_DarX and MeGUI_DarY.
*/

SHOW=False # Show Info on frame.
FRAME_DAR =RT_Undefined # Float, Input DAR of frame before RoboCrop(including letterboxing). RT_Undefined auto interrogates d2v file.
OUT_DAR =2.21/1.0 # Float, 0.0 = Use RoboCrop'ed DAR : Other = Additional crop to output DAR, eg FRAME 16:9, Robocrop'ed 2.4:1, OUT 16:9.
OUT_DAR_IS_LIMIT=True # If False, then OUT_DAR is required output DAR. (Error if OUT_DAR > RoboCrop'ed DAR, Cannot widen aspect ratio)
# If True and OUT_DAR != 0.0(0.0=RoboCrop'ed DAR), then reduce DAR only if RoboCrop'ed DAR is greater than OUT_DAR.
# Allows setting of maximum allowable output DAR(feel less like a peeping Tom for 2.4:1, eg limit to 16:9 on 4:3 device).

OUT_HEIGHT=0 # Int, 0 = Use RoboCrop'ed Height, Otherwise vertical resize to OUT_HEIGHT
OUT_WIDTH =0 # Int, 0 = Use OUT_SAR : Otherwise ignore OUT_SAR and use explicit OUT_WIDTH
OUT_SAR =0.0 # Float, Used if OUT_WIDTH=0 : 0.0 = Original SAR(crop only if OUT_HEIGHT=0) : 1.0 = 1:1 SAR
#
STRENGTH =0.75 # Float, Luma Auto Level strength (0.0 -> 1.0, 0.0=off)
UPCONV=True # Bool, False = no upsample(Progressive Encoded), True=upsample to YUY2(Progressive encoded as Interlaced), Can leave True.
###### Less often Changed
SAMPLES=32 # Int, Frames to sample, used by both RoboCrop and Auto Luma Levelling.
THRESH=-32 # int, -32 = RoboCrop AUTO Thresh
CROPMODE=2 # int, RoboCrop CropMode. 1=CropLess, 2= CropMore(Default)
TFM=True # If True, and FrameRate > 29.0 and MPEG-2 and DGIndex 'Honour Pulldown Flags' then Call TFM and TDecimate.
TFM_ANIME=False # True if Anime, For TDecimate
XMOD=0 # Int, 0=AUTO For crop/resize.
WMOD=4 # Int, 0=AUTO For crop/resize. (4 avoids problems on eg mp4 encodes displayed in MPC-HC)
MEDIAINFO="C:\BIN\MEDIAINFO.EXE" # To Check Aspect Ratio in IFO and compare with VOB : If conflict use IFO_MODE
IFO_MODE= 0 # 0 Error Alert on VOB/IFO DAR mismatch, 1 = Use IFO Frame Aspect Ratio
# List of programs that DONT like WMOD=2 for eg YUV.
# WMOD=0(AUTO) For YV12 will use eg 2 but switch to 4 if Process Name in WMOD4_PROGS list below (can append other names as in Task Manager).
WMOD4_PROGS="
VirtualDubMod.exe # VirtualDub Current OK
AVSEdit.exe # Built in Player
mplayer2.exe
wmplayer.exe
mpc-hc.exe
"
ROBOCROP_DEBUG=False
AUTOLEVEL_DEBUG=False
######################
### Filled in by DGIndex when using as DGIndex Template
VideoFileName ="__vid__"
AudioFileName =LCase("__aud__")
AudioDelay =Value("__del__")
######################
MPEG2Source(VideoFileName,UpConv=(UPCONV)?1:0) # Upconv to convert to YUY2, unmangle chroma if encoded as interlaced.

AudioExt=RT_GetFileExtension(AudioFileName)

Audio= (AudioExt==".ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
\ : (AudioExt==".mpa"||AudioExt=="mp1"||AudioExt==".mp2"||AudioExt==".mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
\ : (AudioExt==".wav") ? RaWavSource(AudioFileName)
\ : (AudioExt==".dts") ? NicDTSSource(AudioFileName)
\ : 0

Assert(!isInt(Audio),"NO AUDIO")

(!isInt(Audio)) ? AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0) : NOP # Trim, chop/add audio to length

(!isInt(Audio) && AudioRate() <> 44100) ? ResampleAudio(44100) : NOP
#(!isInt(Audio) && AudioRate() <> 48000) ? ResampleAudio(48000) : NOP
#(!IsInt(Audio) && (AudioRate() <> 44100 && AudioRate() <> 48000))?ResampleAudio(44100) : NOP
#(!IsInt(Audio))? ConvertToMonO().ResampleAudio(44100) : NOP

######
GSCript("""
myName="ReDAR_DGIT: "
D2VVOBS_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=1)
D2V_START=RT_NumberValue(D2VVOBS_S) + 3
D2V_LINES = 12
SS0=""
if(TFM && FrameRate>29.0) {
MPEG_LINE = RT_FileFindStr(VideoFileName,"MPEG_Type",Sig=False,start=D2V_START,lines=D2V_LINES)
MPEG_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=MPEG_LINE)
MPEG2 = (RT_FindStr(MPEG_S,"=2",Pos=9) >= 0)
if(MPEG2) {
HONOUR_LINE = RT_FileFindStr(VideoFileName,"Field_Operation",Sig=False,start=D2V_START,lines=D2V_LINES)
HONOUR_S=RT_ReadTxtFromFile(VideoFileName,lines=1,Start=HONOUR_LINE)
HONOUR_PULLDOWN_FLAGS=(RT_FindStr(HONOUR_S,"=0",Sig=False,Pos=15) > 0)
if(HONOUR_PULLDOWN_FLAGS) {
FIN_LINE = RT_FileFindStr(VideoFileName,"FINISHED",Sig=False)
Assert(FIN_LINE>=0,myName+"'FINISHED' NOT FOUND in d2v file")
FIN_S = RT_ReadTxtFromFile(VideoFileName,lines=1,Start=FIN_LINE)
IsFILM = (RT_FindStr(FIN_S,"FILM",Sig=False,Pos=9) > 0)
FILM_PERC = RT_NumberValue(FIN_S,Pos=9)
FILM_PERC = (IsFILM) ? FILM_PERC : 100-FILM_PERC
if(FILM_PERC >= 95) {
SS0=RT_String("Calling TFM and TDecimate")
RT_DebugF(SS0,name=myName)
SS0=SS0+Chr(10)
TFM(d2v=VideoFileName)
TDecimate(mode=(TFM_ANIME)?1:0)
}
}
}
}
# Cannot eg Trim/Add frames before here, will interfere with TFM and TDecimate
if(!Defined(FRAME_DAR)) { # Interrogate d2v for FRAME Display Aspect Ratio (incl letterboxing)
FDAR_LINE=RT_FileFindStr(VideoFileName,"Aspect_Ratio=",Sig=False,start=D2V_START,lines=D2V_LINES)
Assert(FDAR_LINE>=0,myName+"'Aspect Ratio' NOT FOUND in d2v file")
FDAR_S=MidStr(RT_ReadTxtFromFile(VideoFileName,lines=1,start=FDAR_LINE),14)
FNDS=RT_String("16:9,625\n16:9,525\n4:3,625\n4:3,525") # DGIndex version of MPEG1 PAR (y/x)
REPS=RT_String("0.7031\n0.8437\n0.9375\n1.1250") # Actual numbers replacing (MPEG1 PAR=y/x)
FDAR_S=RT_StrReplaceMulti(FDAR_S,FNDS,REPS) # Convert MPEG1 625(PAL),525(NTSC) Ratio to number
FDAR_N=Value(FDAR_S) # May be Float
COLON=RT_FindStr(FDAR_S,":") # Ratio ?
FDAR_D=(COLON==0) ? 1 : RT_NumberValue(FDAR_S,Pos=COLON+1)
Assert(FDAR_N>0 && FDAR_D>0,myName+"Error Reading d2v 'Aspect Ratio' (N="+String(FDAR_N)+",D="+String(FDAR_D)+")")
# Get Frame DAR from:- 1:1 SAR, or MPEG2 DAR, or MPEG1 PAR(y/x)
FRAME_DAR=(FDAR_N==1 && FDAR_D==1) ? Float(Width)/Height : (COLON!=0) ? Float(FDAR_N)/FDAR_D : (Float(Width)/Height)/FDAR_N
FRAME_DAR = Round(FRAME_DAR * 1000.0) / 1000.0
If(MEDIAINFO != "" && Exist(MEDIAINFO)) {
MPEG_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=2)
MPEG_S= RT_TxtGetLine(MPEG_S) # remove newline
M_EXT = RT_GetFileExtension(MPEG_S)
if(M_EXT == ".VOB") {
IFO = RT_FilenameSplit(MPEG_S,7)
IFO = LeftStr(IFO,Strlen(IFO)-1)
IFO = IFO + "0.IFO"
if(Exist(IFO)) {
RT_DebugF("File Is VOB and IFO exists",name=myName)
TMP=RT_GetFullPathName("~"+RT_LocalTimeString + ".txt")
RT_DebugF("Calling MediaInfo on IFO '%s'",IFO,name=myName)
# write Aspect Ratio in form "1.333" to file TMP
result=RT_Call(
\ RT_QuoteStr(MEDIAINFO) +
\ " --LogFile=" +
\ RT_QuoteStr(TMP) +
\ " --Inform=Video;%DisplayAspectRatio% " +
\ RT_Quotestr(IFO)
\ , hide=True,Debug=True
\ )
if(result==0 && Exist(TMP)) {
MI_TXT = RT_TxtGetLine(RT_ReadTxtFromFile(TMP))
RT_FileDelete(TMP)
Val = Value(MI_TXT)
RT_DebugF("MediaInfo Text = %s",MI_TXT,name=myName)
if(Val != FRAME_DAR) {
RT_DebugF("IFO(%f) and d2v(%f) Aspect Ratios DISAGREE, Using IFO AR = %f",Val,FRAME_DAR,Val,name=myName)
Assert(IFO_MODE==1,RT_String("IFO(%f) and d2v(%f) Aspect Ratios DISAGREE\nSet FRAME_DAR manually"
\ ,Val,FRAME_DAR))
FRAME_DAR=Val
} Else {
RT_DebugF("IFO and d2v Aspect Ratios AGREE on %f",FRAME_DAR,name=myName)
}
} Else {
RT_DebugF("MediaInfo Fails, result = %d",result,name=myName)
}
}
}
}
}

PROCESSNAME=RT_GetProcessName()
XMOD = (XMOD!=0) ? XMOD : RT_ColorSpaceXMod()
WMOD = (WMOD!=0) ? WMOD : (IsYUV() && RT_TxtFindStr(WMOD4_PROGS,PROCESSNAME,Sig=False)>=0) ? 4 : RT_ColorSpaceXMod
HMOD = Max(RT_ColorSpaceYMod(Laced=false),2)
RT_DebugF("ProcessName = %s WMOD=%d HMOD=%d",PROCESSNAME,WMOD,HMOD,name=myName)
OUT_DAR =Float(OUT_DAR)
OUT_SAR =Float(OUT_SAR)
SAR=RT_GetSAR(FRAME_DAR) # Sample (pixel) aspect ratio of input frame
SS1=RT_String("INPUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",Width,Height,FRAME_DAR,SAR,Float(Width)/Height)
RT_DebugF(SS1,name=myName)
SS1=SS0+SS1
RoboCrop(samples=SAMPLES,thresh=THRESH,cropmode=CROPMODE,debug=ROBOCROP_DEBUG,hmod=HMOD,Laced=false)
DAR=RT_GetDAR(SAR) # Image DAR after RoboCrop'ed letterboxing
SS2=RT_String("RoboCrop: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",Width,Height,DAR,SAR,Float(Width)/Height)
RT_DebugF(SS2,name=myName)
if(OUT_DAR_IS_LIMIT && OUT_DAR > DAR) {
OUT_DAR = DAR # Do not report Error, we are just limiting DAR
RT_DebugF("Ignoring OUT_DAR, (RoboCrop'ed DAR is Less))",name=myName)
}
Assert(OUT_DAR <= DAR,"OUT_DAR("+String(OUT_DAR,"%.3f") + ") Must be <= DAR("+String(DAR,"%.3f")+")")
OUT_SAR = (OUT_SAR == 0.0) ? SAR : OUT_SAR
OUT_DAR = (OUT_DAR == 0.0) ? DAR : OUT_DAR
OUT_HEIGHT=OUT_HEIGHT/HMOD*HMOD
OUT_WIDTH =OUT_WIDTH /WMOD*WMOD
H = (OUT_HEIGHT == 0) ? Height : OUT_HEIGHT # RoboCrop'ed OR user supplied resize height
# If OUT_WIDTH==0 Then use H(Height or user OUT_HEIGHT) & OUT_SAR(SAR or user OUT_SAR) to calc output width ELSE user OUT_WIDTH.
W = (OUT_WIDTH==0) ? Int(H * OUT_DAR / Float(OUT_SAR) / WMOD) * WMOD : OUT_WIDTH
if(OUT_WIDTH==0 && H==Height && OUT_SAR==SAR) { # Output RoboCrop'ed Width,Height, No resize, non-fractional cropping
CROP_WIDTH = W # Crop Only, maintain SAR, with approx OUT_DAR (occurs when original OUT_WIDTH==0 and OUT_HEIGHT==0 and OUT_SAR==0.0)
CROP_LEFT = Int((Width - CROP_WIDTH) * 0.5 / XMOD + 0.5) * XMOD # Crop centering, CROP_WIDTH and CROP_LEFT are Int
} Else { # Crop/Resize with fractional cropping to OUT_DAR
CROP_WIDTH = Width * (Float(OUT_DAR) / DAR)
CROP_LEFT = (Width - CROP_WIDTH) * 0.5 # Crop centering, CROP_WIDTH and CROP_LEFT are Float
}
# Now decide none, crop only, or crop/resize. CROP_WIDTH and CROP_LEFT may be Int or Float.
if(CROP_WIDTH==Width && CROP_LEFT==0 && W==Width && H==Height) {
SS3=RT_String("NO CROP/RESIZE, %dx%d DAR=%.3f",Width,Height,DAR)
} Else if(Int(CROP_LEFT/XMOD)*XMOD == CROP_LEFT && CROP_WIDTH==W && H==Height) { # Crop Only
CROP_LEFT=Int(CROP_LEFT)
CROP_WIDTH=Int(CROP_WIDTH)
Crop(CROP_LEFT,0,CROP_WIDTH,-0) # Cropping does not alter SAR
DAR=RT_GetDAR(SAR) # Calc resultant DAR after crop
SS3=RT_String("Crop(%.d, %d, %d, %d)",CROP_LEFT,0,CROP_WIDTH,-0)
} Else {
DAR=RT_GetCropDAR(DAR,CROP_LEFT,0,CROP_WIDTH,-0) # Pre-calc resultant DAR of Last clip, after possible fractional cropping
Spline36Resize(W,H, CROP_LEFT,0,CROP_WIDTH,-0)
SAR=RT_GetSAR(DAR) # Calc new SAR due to Resizing
SS3=RT_String("Resize(%d,%d, %.1f, %d, %.1f, %d)",W,H, CROP_LEFT,0,CROP_WIDTH,-0)
}
RT_DebugF(SS3,name=myName)
SS9=""
if(STRENGTH>0.0) { # Auto Luma Levelling
Eval(RT_QueryLumaMinMax(samples=SAMPLES,debug=AUTOLEVEL_DEBUG)) # Get luma min/max over SAMPLES frames
if(IsRGB()) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
SS9=RT_String("Levels(%d,1.0,%d,%d,%d,Coring=False)",ALMin,AlMax,CSMin,CSMax)
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring
}
}
RT_DebugF ("%s\nOUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f ",SS9,Width,Height,DAR,SAR,Float(Width)/Height,name=myName)
(SHOW) ? RT_Subtitle("%s\n%s\n%s\n%s\nOUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",SS1,SS2,SS3,SS9,Width,Height,DAR,SAR,Float(Width)/Height) : NOP
RT_SignalDar(OUT_DAR) # Tell MeGUI the Exact required Display Aspect Ratio rather than approximate.
RT_DebugF("Signalled MeGUI DAR=%f MeGUI_DarX=%d MeGUI_DarY=%d",OUT_DAR,MeGUI_DarX,MeGUI_DarY,name=myName)
ConvertToYV12
# FFT3DFilter(Sigma=1.6,plane=0,Sharpen=0.01)
""")
Return Last


EDITED:

[johnmeyer]

Many thanks for the security cam detection script:

SecurityCamExtractMotion.AVS (http://forum.doom9.org/showthread.php?p=1628675#post1628675)

It works much better than using Scene Detection in MVTools2, or YDiffxxx, etc. In fact, it is just plain great!

Quick question.

I was able to get it to work when using SetMTMode, but I'm still looking for more performance, since my security camera captures are twelve hours each at 30 fps. I've used the cropping parameters to reduce the pixels that must be evaluated, but can you suggest any other things I might do to get even faster performance? I'm able to get 80 fps on 30 fps 1440x1080 AVCHD footage. This is obviously pretty darned good, and I'm very impressed with how fast this works. So, I'm not complaining; I'm just greedy.

Thanks!

[StainlessS]

Hmm . . . surely you jest - you get three supportive comments - not insincere nor excessive and certainly not self-serving

However, if you can, StainlessS, please try to find another place to upload your files - the pop-ups are horrendous

Yes of course I jest (hence the smiley :) )
You should never take anything I say as serious, I dont (even without smiley).

Mediafire does still seem to be preferred file host on D9, I dont usually see much in the way of pop-ups from there (or anywhere else). Suggest that you switch on/install pop-up blocker, ad-blocker and flash-blocker, also FireFox is a good idea and way safer from malware than IE (EDIT: every hackers favorite target).

@John,
In the older script version you linked, I think I tried to keep all
frame in areas of movement even if no movement in particular frames, Forensic seemed to give more importance to getting rid of as many frames as possible, so I think I changed behaviour
to match (in later versions of script in RT_Stats zip).
Do you have any preference, least frames/ keep alll frames in movement area ?

I have been revisiting scripts supplied in RT_Stats.zip and will again look at this one, give me some time, I'm not spending as much time as previously on Avisynth, found other things to occupy myself.

EDIT: I'm not in NTSC land and only added the pulldown removal stuff to script in previous post as a sort of
example for others to expand on. I only have 1 single NTSC VOB (rescued from a DVD that is now almost totally corrupt)
and so have only 1 sample to play with. If anyone does expand on NTSC handling, you could post results here.

[Groucho2004]

Mediafire does still seem to be preferred file host on D9
Mediafire lately seems to have been embracing the dark side. I suggest you move your stuff to dropbox.com, much lighter and no popups. I did. :)

[StainlessS]

Thanks Grouchy, I shall look into that. :)

Here, DBase funcs as finally added to RT_Stats v1.30.


*****************************************************
********** DBASE FUNCTIONS ***********
*****************************************************

RT_Stats DBASE functions allow fast access to a file based DataBase where each record can have up to 256 fields of variable type,
Bool, Int, Float, String, or BIN. BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in the DBase field
as an unsigned 8 bit Int, upper 24 bits ignored.
The maximum possible file size is about 2GB, and the number of records is restricted by ($7FFFFFFF - Header_Size) / Record_Size
where $7FFFFFFF is maximum +ve integer, Header_Size is 4096 and Record_Size is sum of all Field sizes.
Field types Bool and BIN require 1 byte, Int and Float 4 bytes, and String is of fixed maximum length (user chosen) up to
256KB ((256 * 1024), no nul terminating character is stored in a DBase String).
In addition to the DBase records, a DBase file allows up to 256 Attributes of types Int or Float, which can be used for
whatever your heart desires, these attributes are stored in the DBase file header.
You could use the DBase functions to return multiple results from a script function to the caller via a caller supplied DBase
filename. The DBase functions also make it more likely that script functions could be written to eg find where edits between
two clips have been made, or previously impossible/implausible tasks performed.
Currently, the DBase functions are just primitive access functions, it is possible that some future update may have functions
added for eg sorting or searching or selecting records, who knows. :)


***
***
***

RT_DBaseAlloc(String Filename,Int Records,String TypeString,Int "StringLenMax"=256)
Creates a file to hold DataBase records. Each record can have up to 256 fields of varying type, Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in DBase field as an unsigned 8 bit int, upper 24 bits ignored.

Filename, DBASE Filename, no default.

Records, int zero or more. Number of records to create, initalized to zeros (initialized to zeros, Bool=False,Float=0.0,String="").

TypeString, string holding record construction data. One character for each field, valid characters are "bifsn" or Uppercase.
'b' = BOOL, 'i' = INT, 'f' = FLOAT, 's' = STRING, 'n' = BIN.
The 's' STRING construction character can be followed by numeric digits describing fixed maximum string length (excluding
any nul term character), otherwise StringLenMax is used by default instead.
For no particular reason, I have limited the maximum string length to 256K, ie 256 * 1024, throws an error if greater.

StringLenMax Default=256. Default length of String fields if no explicit string length defined following TypeString S constructor char.

Example TypeString:-
"ifffs1024bsn", 1st field Int, 3 Float fields, 1 string field of length=1024, 1 Bool field, 1 string field using length of
StringLenMax, and 1 BIN field. 8 Fields in all (0->7).
Returns number of records created.
All RT_DBaseXXXX() errors produce an error alert.

***
***
***

RT_DBaseRecords(String Filename)
Filename, DBASE Filename, no default.
Returns current number of records in Filename database.

***
***
***

RT_DBaseRecordSize(String Filename)
Filename, DBASE Filename, no default.
Returns record size, ie sum of field sizes.

***
***
***

RT_DBaseRecordsMax(String Filename)
Filename, DBASE Filename, no default.
Returns maximum theoretical number of records that could be stored in file based on (Maximum +ve int - header size) / Record Size.
(where record size is sum of all field sizes).

***
***
***

RT_DBaseFields(String Filename)
Filename, DBASE Filename, no default.
Return number of fields per record.

***
***
***

RT_DBaseFieldType(String Filename,Int Field)
Filename, DBASE Filename, no default.
Field, range 0 -> RT_DBaseFields(Filename) - 1
Return type of Field. 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.

***
***
***

RT_DBaseFieldSize(String Filename,Int Field)
Filename, DBASE Filename, no default.
Field, range 0 -> RT_DBaseFields(Filename) - 1
Return size of Field in bytes. Bool=1, Int=4, Float=4, String length as set by RT_DBaseAlloc(), BIN=1.

***
***
***

RT_DBaseGetField(String Filename,Int Record,Int Field)
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Field, range 0 -> RT_DBaseFields(Filename) - 1
Returns contents of a field belonging to the particular record.

***
***
***

RT_DBaseSetField(String Filename,Int Record,Int Field, FieldVar)
Sets a single Field in a Record to FieldVar.
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Field, range 0 -> RT_DBaseFields(Filename) - 1
FieldVar, variable of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseSet(String Filename,Int Record, FieldVar_1, ... , FieldVar_n)
Simultaneously sets all fields of a record. All field variables must be supplied.
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
FieldVar_1, ... , FieldVar_n. A variable for each field of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseExtend(String Filename,Int "Add"=1)
Adds a number of blank records to Filename Database. (initialized to zeros, Bool=False, Float=0.0, String="").
Filename, DBASE Filename, no default.
Add, Int default 1, Number of blank records to add to the DataBase.
Returns the current number of records.

***
***
***

RT_DBaseAppend(String Filename, FieldVar_1, ... , FieldVar_n)
Creates a record from Field Vars and appends it to the end of the DataBase file. All field variables must be supplied.
Filename, DBASE Filename, no default.
FieldVar_1, ... , FieldVar_n. A variable for each field of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseGetAttrib(String Filename,Int ix)
There are 256 available attributes in the DataBase file header, that can be used for whatever purpose you like,
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 255.
Returns the value of the user set attribute Int or Float (or type Int 0 if not yet set).

***
***
***

RT_DBaseSetAttrib(String Filename,Int ix, Var)
There are 256 available attributes in the DataBase file header, that can be used for whatever purpose you like,
RT_DBaseAlloc() initializes all Attributes to type Int, 0.
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 255.
Value, Int or Float, The value that Attrib(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.


Also, number of Array attributes increased to 256.

[johnmeyer]

In the older script version you linked, I think I tried to keep all frame in areas of movement even if no movement in particular frames, Forensic seemed to give more importance to getting rid of as many frames as possible, so I think I changed behaviour to match (in later versions of script in RT_Stats zip).

Do you have any preference, least frames/ keep alll frames in movement area ?

I have been revisiting scripts supplied in RT_Stats.zip and will again look at this one, give me some time, I'm not spending as much time as previously on Avisynth, found other things to occupy myself.No need to make any changes. I am using the script to generate frame numbers that I then import into my NLE (Vegas). This gives me markers, and I can then simply go to each marker. I "thin out" the markers in Excel, prior to importing to Vegas, so I don't need any logic in the script itself to only keep the first/middle/last frame during a motion sequence.

So, the script works brilliantly and perfectly; my only goal is to improve the performance, even though it is already very fast. To do this, I'm simply looking for settings that might bypass certain operations. I'm not asking for you to do any programming or changes.

Once again, many thanks!

[StainlessS]

Oooops, sorry John, I meant to come back to this but forgot until I saw you online a little earlier.
I've just looked at the script and only suggestions I can make are to increase block size (should
not be so important for HD content, I think [governs small localized movement]), and an ad hoc fix.
Ad hoc fix being to set a Global var name eg LACED=true/False to control args to all instances
of RT_LumaMovement, and add to each instance of RT_LumaMovement an 'Interlaced=LACED' arg.
Increasing block size should not matter too much for HD and although testing only every other row
of pixels would make a difference to sensitivity, I'm guessin that it will not matter too much.
Reducing EDIT: Should read Increasing block size should reduce overhead (eg checking coords) for each block, and the laced thing
could perhaps yield up to double speed, but you'll have to suck it and see to find out for sure,
let us know how it goes, good luck.

I'll add the laced thing in next version, peace bro.

EDIT: 'yield up to double speed', relates to the script, eg no effect on source filter, reading hard drive etc.
EDIT: Will likely require changes to thresholds (if changing block size).
EDIT: Try ad hoc fix first, and play with block size after that. If you seem to lose sensitivity after LACED thing,
you might even want to decrease block size.
EDIT: I dont use MT, and so have no suggestions concerning that.

[Forensic]

I am trying to detect if a specific file already exists in the same directory as my script and, if not, do stuff. I tried

try { rt_temp = RT_GetFileTime ("test.txt") }
catch (err) { DO STUFF }

Something about that syntax is wrong. Any advice or a better way to test for this?

[StainlessS]

Try built-in Exist().

From Docs:-

Exist | v2.07 | Exist(filename)
Tests if the file specified by filename exists.
Examples:

filename = ...
clp = Exist(filename) ? AviSource(filename) : Assert(false, "file: " + filename + " does not exist")


EDIT: From RT Docs

RT_GetFileTime(string filename,int item)
Returns string, one of three times associated with filename file.
Error if cannot access file. (debugview for error string).
Item, int, (0 -> 2). 0=Creation Time, 1=Last Write/Modified Time 2=Last Accessed Time,
The return string is always of format "YYYY-MM-DD HH:MM:SS.mmm" and so can be used to
compare whether one file time is later than another using string comparison.
The time strings returned are in UTC (Coordinated Universal Time), not local time.
v1.28Beta5, added milliseconds to string.


You were missing compulsory 'Item' arg.

[Forensic]

Thank you.

[StainlessS]

Posted in response to a Reel.Deel post here: http://forum.doom9.org/showthread.php?p=1681509#post1681509


Extracts all filter/function names and arguments contained in an autoloaded dll, writes results to a text file.
Target dll required to be in Plugins dir. Hack of something I did to list RT_Stats funcs.

# Make_FunctionList.AVS by StainlessS, Requires RT_Stats v1.30+, and GScript.
# Extracts names and args of Filters and Functions contained in an autoloaded dll, results written to text file.
# Plugins need to be auto loaded in Plugins Dir.

ORDER=True

HEAD="""
There follows a list of all function names together with CPP style argument specifiers that inform
Avisynth the argument types and optional names. Optional arguments have square brackets surrounding
their name as in [name] and are followed by a type specifier character that gives the type.
Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args,
followed by optional 'arg1' of type bool and optional 'arg2' of type int.

# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
###################################
"""
Assert(RT_FunctionExist("GScript"),"You Need GScript To Run This Script"+Chr(10)+"http://forum.doom9.org/showthread.php?t=147846")
FSEL_TITLE="Select DLL file"
FSEL_DIR=RT_PluginDir()
FSEL_FILT="DLL files|*.dll"
FSEL_MULTI=False
DLLFILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(DLLFILE_LIST.IsString,"RT_FSelOpen: Error="+String(DLLFILE_LIST))
DLLFN=RT_TxtGetLine(DLLFILE_LIST,0)
PREFIX = RT_FilenameSplit(DLLFN,4) + "_"
PREFIX_LEN=StrLen(PREFIX)
ALLPLUGS = RT_StrReplace(RT_PluginFunctions," ",Chr(10))
Assert(RT_TxtFindStr(ALLPLUGS,PREFIX,Sig=False,Start=0)>=0,PREFIX+" Not found, Not an AutoLoad plugin(or more than 50 plugs)")
LINES = RT_TxtQueryLines(ALLPLUGS)
PLUGS = ""
PARMS = ""
MaxLen = 0
DQUOT=34 # Double Quote Chr(34)
GSCript("""
for(i=0,LINES-1) {
i=RT_TxtFindStr(ALLPLUGS,PREFIX,Sig=False,Start=i)
if(i >= 0) {
PLG=RT_TxtGetLine(ALLPLUGS,Line=i)
if(RT_FindStr(PLG,PREFIX,Sig=False)==1) { # match at start ?
SHORTPLG=MIDStr(PLG,PREFIX_LEN+1)
MaxLen=Max(MaxLen,StrLen(SHORTPLG))
PLUGS=RT_TxtAddStr(PLUGS,SHORTPLG)
PARMS=RT_TxtAddStr(PARMS,RT_String(" %c%s%c",DQUOT,RT_PluginParam(PLG),DQUOT))
}
} Else {
i = LINES # break, no more to find
}
}
LINES = RT_TxtQueryLines(PLUGS)
PLUGS_AND_PARMS=""
for(i=0,LINES-1) {
PLUGS_AND_PARMS=RT_TxtAddStr(PLUGS_AND_PARMS,RT_StrPad(RT_TxtGetLine(PLUGS,Line=i),MaxLen)+RT_TxtGetLine(PARMS,Line=i))
}
""")
PLUGS_AND_PARMS = (ORDER) ? RT_TxtSort(PLUGS_AND_PARMS) : PLUGS_AND_PARMS # Order Alphabetical
TXT = RT_String("%s%sFunction_List",PREFIX,(ORDER)?"ORDERED_":"")
FN=TXT+".TXT"
S=RT_String("\n %s \n\n%s\n\n%s",TXT,HEAD,PLUGS_AND_PARMS)
RT_Debug(S)
RT_TxtWriteFile(S,FN)
S=RT_TxtAddStr(S,RT_String("\n\n%s\n\nCreated in current directory\n",FN))
W=1280 H=640 LINES=RT_TxtQueryLines(S)
L=LINES*20+H+100
BlankClip(Width=W,Height=H,Length=L).ScriptClip("""RT_Subtitle("%s",S,x=10,y=height+100-current_frame,expx=true,expy=true)""")

EDITED: Throws Error if not an autoloaded plugin.

EDIT: If I remember correctly, Avisynth only autoloads the first 50 plugins it comes across in the plugins dir, so the above script may not work for all autoload plugs if more than 50 in plugins dir.

[Reel.Deel]

Posted in response to a Reel.Deel post here: http://forum.doom9.org/showthread.php?p=1681509#post1681509

Gave it a try and it works like a charm, thanks StainlessS. :)

[StainlessS]

RT_Stats v1.40 Beta 01
Post #1 of #2 (docs too big for 16KB limit)


v1.40Beta01

RT_WriteFile(String FileName, string format,dat1,...,datn,bool "Append"=False)
Writes a formatted string to FileName file. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the
text string that is written, uses C/CPP printf() style formatting.
Format: compulsory string controlling format and describing the datn type args that are expected.
datn: Variable number of data args of any type (excluding clip).
Append, Default False, optional and MUST be supplied with name (eg Append=True) as we do not know where optional data args end,
Appends to file if already exists, else opens new FileName.

printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned (o=octal, x=Hex).
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

Formatting Insertion point is marked with '%' character in the format string (as in Avisynth String function), if you wish to use
a percent character within the output string, it should be inserted twice in the format string ie '%%' will produce a single '%'.
The data datn arg strings do not require a double '%' character.

A Backslash character '\' introduces escape sequences, to insert a backslash character itself, you must supply a double
backslash sequence ie '\\'.
Converts embedded escape character sequences (Case Significant):-
'\\' Converted to '\' Single Backslash
'\a' Converted to Chr(7)
'\b' Converted to Chr(8)
'\t' Converted to Chr(9)
'\n' Converted to Chr(10) NewLine
'\r' [and Chr(13)] Converted to Chr(10) NewLine
'\v' Converted to Chr(11)
'\f' Converted to Chr(12)
'\x', where x is ANY OTHER CHARACTER not included above, will be copied verbatim, ie '\x'.

Escape sequences are replaced only when occurring in the format string, if you need escapes in data string then use RT_String.

Example:
RT_WriteFile(FileName,"Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013,Append=False)

***
***
***

RT_VarType(var)
Returns type of var arg as an int.
Returns,
0 = Type BOOL
1 = Type INT.
2 = Type FLOAT
3 = Type STRING
4 = Type CLIP
-1 = Type UNDEFINED
Avoids multiple tests in script to establish a variable type.
eg, RT_VarType(66.6) returns 1, ie FLOAT.
NOTE, If called without an arg, it will return type of implicit Last (Undefined if no clip implicitly/explicitly assigned to Last).

***
***
***

RT_VarIsSame(var Var1, ...,var Varn)
Accepts two or more variables of any type. NOTE, if only 1 arg supplied, will use implicit Last as first arg.
Returns Bool, False if args are not all of same type AND/OR value.
Return True, if all args same type and value.
Type String, compares strings exactly, same length and casing.
Type Clip, compares ONLY video attributes, frame Width, Height, ColorSpace.
If first clip has no video (HasVideo==False) then all other clips must also have no video (other attributes not tested).
Clip audio, compares Number of Audio Channels, Sampe Rate, Sample Type (int16, Float etc).
If first clip has no audio (HasAudio==False) then all other clips must also have no audio (other attributes not tested).
No other attributes are taken into account, (ie FrameRate, FrameCount etc).

***
***
***

RT_ChanAve(clip c,int "n",string "Prefix"=RCA_")
Sets channel averages for clip c frame n, as local variables, 0.0 -> 255.0. RGB24/32, YUY2, Planar.
Returns the number of channels in clip ie 3 unless Y8 where returns 1.
Default Prefix is "RCA_" so for YV12, would return 3 (3 channels) and set
RCA_0= Luma Ave, RCA_1= U Ave, RCA_2 = V Ave.
For RGB, RCA_0= Red Ave, RCA_1= Grn Ave, RCA_2 = Blue Ave.
For Y8, RCA_2 and RCA_3 will be set at 128, but will return a value of 1 for number of channels.


***
***
***

RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0)
Returns difference between clip c frame n and clip c2 frame n2 (0.0 -> 255.0).
ChromaWeight range 0.0 -> 1.0.
For RGB, returns same as RGBDifference.
For Y8 returns same as LumaDifference.
For non Y8 Planar returns Lumadif + Chromadif with chroma weighted by ChromaWeight arg.
Returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
EXPERIMENTAL: Also currently accepts ChromaWeight -ve (eg -1.0) where will return
Lumadif + (1.0 - Lumadif/255.0) * ((Udif + Vdif)/2.0)
This may well be removed at some future date (although would do no harm to leave in-situ).

***
***
***

RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=current_frame,int "x2"=0,int"y2"=0,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns number of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Note, returns the number of pixels whose difference is greater than Thresh, so ranges 0 -> (Width*Height).
Thresh Default 0, returns number of pixles that are not exactly the same.

***
***
***

RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elased in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only ting that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str="Test Start="+String(s,"%.3f")+" End="+String(e,"%.3f")+" Time="+String(e-s,"%.6")+"secs"
SubTitle(str)

***
***
***

RT_SignalDAR2(int darx,int dary)
As for RT_SignalDAR() except it allows setting of numerator and denominator individually.
Signal DAR to MEGUI. Sets Global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

***
***
***

[StainlessS]

RT_Stats v1.40Beta01
Post #2 of 2


RT_QwikScanCreate(clip c,string DB,string "PrevDB"="",string "NextDB"="",int "matrix"=(Width>1100||Height>600?3:2),bool "Debug"=False)
Creates a DataBase set used to quickly scan a clip for frame matching, DataBases used by RT_QwikScanGetNear.
c, clip used to create database frame fingerprint info.
PrevDB, DBase filename, used to find frames prior to a 'start frame'.
NextDB, DBase filename, used to find frames after a 'start frame'.
Either PrevDB or NextDB (or both) must be given.
Matrix, used to convert RGB to Luma_Y for Fingerprint creation.
Debug, True sends a little info to DebugView window (google).
Maximum PrevDB/NextDB size is restricted to 524283 records(frames).

***
***
***

RT_QwikScanGetNear(clip DBClip,string DB,string PNDB,int StartFrame,int FindFrame, \
clip "FindClip"=NULL,int "Maxdistance"=DBClip.FrameCount-1,bool "Inclusive"=True,Float "LumaTol"=0.0, \
int "Flags"=0, Float "LD"=0.0,Float "LC"=0.0,Float "FD"=0.0,int "PC"=0,int "XP"=1,string "Prefix"="QSGN_",Float "ChromaWeight"=1.0/3.0)

*** EXPERIMENTAL ***

Developer function only, not for the casual scriptor.
Function that scans a pair of database files to find a sequence of candidate frames that could match a target FindFrame.
Returns frame number of possible matching frame, or -1 meaning NOT FOUND.
DBClip, the clip to search, and used as arg to RT_QwikScanCreate to create DB, PrevDB and NextDB database files that hold
"FingerPrint" and frame location data to assist in rapid location of candidate frames.
DB, Filename of the "FingerPrint" Dbase. Created via RT_QwikScanCreate.
PNDB, Filename of the frame locator Dbase. Created via RT_QwikScanCreate as either PrevDB or NextDB.
Use the PrevDB if searching frames prior to StartFrame in DBClip, or the NextDB if search frames after StartFrame.
StartFrame, frame in DBClip where searching will start, see also Inclusive.
FindFrame, Frame number of the frame that is to be matched (in FindClip if supplied or DBClip if not supplied).
FindClip, clip holding frame to match, defaults to DBClip if not supplied.
MaxDistance, maximum number of frames to search, defaults DBClip.FrameCount -1, but silently limited to the maximum possible
distance depending upon where StartFrame is and whether searching forwards or backwards. A MaxDistance of 1 will search only
the StartFrame (if Inclusive) and the frame next to it.
Inclusive. If true (default) then also tests StartFrame.
LumaTol, +- Luma Tolerance (0.0 -> 16.0). Allow for mismatch in luma between DBClip and FindClip. If eg the FindClip pixels can vary
by as much as +- 4 pixel levels, then set LumaTol to 4.0, the bigger this number the slower the function, but it should be still
quite fast with most clips even up to a LumaTol of maximum 16.0, the bigger this number, the more frames will fit the fingerprint
and so there will be more candidate frames to check.
Flags, Bit flags, sets the additional Exit Conditions of the function.
Bit 0 set eg 0x01, Sets the LumaDifference Exit Condition.
Bit 1 set eg 0x02, Sets the LumaCorrelation Exit Condition.
Bit 2 set eg 0x04, Sets the FrameDifference Exit Condition.
Bit 3 set eg 0x08, Sets the PixelsDifferentCount Exit Condition.
LD, LumaDifference Exit Condition Threshold.
LC, LumaCorrelation Exit Condition Threshold.
FD, FrameDifference Exit Condition Threshold.
PC, PixelsDifferentCount Exit Condition Threshold (int).
XP, Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=continue searching while next frame is a better Exit Condition match.
2=Continue searching while next frame is a better or equal Exit Condition match.
Prefix, Default "QSGN_". Prefix for names of local variables returned to give status of exit conditions.
Below only set if Flags non zero.
QSGN_Exit_Flags, bit flag, condition that was satisfied to cause exit (same as Flags arg, only 1 bit will be set).
QSGN_Exit_LD, LumaDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x01.
QSGN_Exit_LC, LumaCorrelation for the exit frame, Only set if QSGN_Exit_Flags == 0x02.
QSGN_Exit_FD, FrameDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x04.
QSGN_Exit_PC, PixelsDifferentCount for the exit frame, Only set if QSGN_Exit_Flags == 0x08.
QSGN_Exit_XP, Count of additional frames traversed if found to be a better match. Only set if both XP arg is non zero
and exit condition occurred ie QSGN_Exit_Flags non zero.
ChromaWeight, Argument to the FD FrameDifference Exit Condition (YUV only).

The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
The PNDB Frame Locator Dbase holds records with 1024 fields per record (frame). The first 256 of these fields give frame numbers
of the next nearest frame (previous[PrevDB] or next[NextDB]) that roughly complies with the first component FingerPrint value.
Eg, field 42 holds the frame number of the nearest frame whose first component FingerPrint value is greater or equal to 42.0
and less than 43.0. Fields 256 to 511 hold frame info for the 2nd FingerPrint component, etc.

This is roughly how the function works without Exit Conditions (Flags=0):-
It first establishes a range of component FingerPrint values for FindFrame, based on LumaTol.
If Inclusive, then checks StartFrame for a FingerPrint match and returns StartFrame if good match.
Next it Checks the Frame Locator Dbase (PNDB) looking for the next nearest frame that roughly complies with the FingerPrint
range. It then checks that candidate frame in the FingerPrint DBase to see if it more exactly matches the FingerPrint, if it
does match, the frame is returned to the client and if not then it moves the 'Cursor' in the Frame Locator DBase up to that
frame and repeats the searching until found or not.
So, without Exit conditions, the function returns a sequence of candidate frame numbers that may or may not be the exact frame
you are looking for, it will likely skip vast swathes of frames and then deliver a stream of frames from the same scene as
your FindFrame, where you would likely want to establish a more exact match using some other function.

With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a result to
the client. If an Exit condition is set, then that exit condition also has to be satisfied, if not satisfied, then the candidate
frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and the Exit
Condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, they are processed in bit number order ie LumaDifference condition first, the first
condition that succeeds will cause exit and no further exit conditions will be processed. If an Exit Condition is satisfied,
and the XP arg is set, the function will continue scanning adjacent frames and step forward until a better match for the Exit
Condition is NOT found. (XP=1 is better than, and XP=2 is better or equal).

Exit Conditions:-

LD, LD is the Threshold for RT_LumaDifference, if the match is less or equal to this then the Exit Condition is satisfied.

LC, LC is the Threshold for RT_LumaCorrelation, if the match is less or equal to this then the Exit Condition is satisfied.
We have 'swizzled' the values returned by RT_LumaCorrelation(), instead of the normal range -1.0 -> 1.0,
we use 0.0 -> 255.0 where 0.0 means exact match. We have done this for 2 reasons. Firstly, to make it the same (roughly)
as the other exit conditions, the closer to zero, the better the match. 2nd reason is, Avisynth Floats have only 6 digits
of decimal precision, so eg 0.999999 is about a good as it gets (closest to 1.0 without actually being 1.0) if we were
setting a threshold for normal range LumaCorrelation (-1.0 -> 1.0), but when we 'swizzle' the numbers to make 0.0 an exact
match, we can supply a more precise threshold which will be used at double precision within the function,
eg 0.0000000000000000000000001 is still a valid float (and in this case it is only 1 significant digit of precision).
RT_LumaCorrelation is slower than RT_LumaDifference but it may be more appropriate if there is a big difference in
luma beteen DBClip and FindClip.

FD, FD is the Threshold for RT_FrameDifference, if the match is less or equal to this then the Exit Condition is satisfied.
The RT_FrameDifference Exit Condition uses the ChromaWeight arg to weight chroma for YUV colorspaces.
For RGB, RT_FrameDifference is equivalent to RGBDifference. For Y8 it is equivalent to LumaDifference.
For non Y8 YUV, returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
EXPERIMENTAL: Also currently accepts ChromaWeight -ve (eg -1.0) where will return
Lumadif + (1.0 - Lumadif/255.0) * ((Udif + Vdif)/2.0)
This may well be removed at some future date (although would do no harm to leave in-situ).

PC, PC is the Threshold for RT_LumaPixelsDifferent, if the match is less or equal to this then the Exit Condition is satisfied.
PC is a pixel count, ranging 0 -> (Width * Height). Allows you to set maximum number of pixels that are in any way different
(RGB converted to Luma-Y).

Usage:-
Using PAL SD Jurassic Park with 182050 frames (UT_Video Compressed YV12) it took 15:06 to create DB and PrevDB FingerPrint
and Locator DB on a 2.13Ghz Core Duo, using this script.

AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
return RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)

And test script:
AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
#RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)
GScript("""
LUMATOL=0.0
End=FrameCount-1
For(i=3000,13000,1000) {
S=RT_TimerHP()
j = RT_QwikScanGetNear(Last,DB,PNDB,End,i,Last,maxdistance=End-i,lumatol=LUMATOL,flags=$04,fd=0.0,xp=2)
T=RT_TimerHP-S
RT_Debugf("%d] Got %d in %f secs",i,j,T)
}
""")
return Last

Results:- LumaTol = 0.0
00000005 398.69018555 RT_DebugF: 3000] Got 3000 in 0.019132 secs
00000006 398.70736694 RT_DebugF: 4000] Got 4000 in 0.017023 secs
00000007 398.73089600 RT_DebugF: 5000] Got 5000 in 0.023335 secs
00000008 398.75195313 RT_DebugF: 6000] Got 6000 in 0.020897 secs
00000009 398.76660156 RT_DebugF: 7000] Got 7000 in 0.014457 secs
00000010 398.78686523 RT_DebugF: 8000] Got 8000 in 0.020073 secs
00000011 398.80563354 RT_DebugF: 9000] Got 9000 in 0.018632 secs
00000012 398.82150269 RT_DebugF: 10000] Got 10000 in 0.015690 secs
00000013 398.85241699 RT_DebugF: 11000] Got 11000 in 0.030706 secs
00000014 398.86682129 RT_DebugF: 12000] Got 12000 in 0.014216 secs
00000015 398.88012695 RT_DebugF: 13000] Got 13000 in 0.013135 secs

Results:- LumaTol = 4.0 (many more candidate frames to test)
00000016 444.04776001 RT_DebugF: 3000] Got 3000 in 0.059216 secs
00000017 444.26177979 RT_DebugF: 4000] Got 4000 in 0.213837 secs
00000018 446.82482910 RT_DebugF: 5000] Got 5000 in 2.562858 secs
00000019 454.65585327 RT_DebugF: 6000] Got 6000 in 7.830853 secs
00000020 456.44543457 RT_DebugF: 7000] Got 7000 in 1.789409 secs
00000021 465.26766968 RT_DebugF: 8000] Got 8000 in 8.822010 secs
00000022 465.99948120 RT_DebugF: 9000] Got 9000 in 0.731642 secs
00000023 466.64910889 RT_DebugF: 10000] Got 10000 in 0.649458 secs
00000024 471.09573364 RT_DebugF: 11000] Got 11000 in 4.446451 secs
00000025 471.81393433 RT_DebugF: 12000] Got 12000 in 0.718018 secs
00000026 475.26538086 RT_DebugF: 13000] Got 13000 in 3.451269 secs


EDIT: Above usage examples scan from last frame to target frame, ie about 180,000 frames to search.
EDIT: RT_QwikScanCreate, takes some time to create the DB Dbase, the PrevDB and NextDB
are almost instantanious (maybe about 5 secs for a 200,000 frame DB that takes about 15 mins
to create, so bi-directional searches are not any kind of problem).
I'm hoping that someone will give this a beta test, works unbelievably well. (Wish I had the cash for a patent).




Get it here:- EDIT: Link Removed

[StainlessS]

RT_Stats v1.40 Beta02,

Main changes,
LumaTol change from Float to Int.
Added Exit Condition RT_LumaPixelsDifferent arg PCThresh.
Added BEST MATCH status when Function returns -1 NOT FOUND and EXIT CONDITION Flags set.

RT_QwikScanGetNear(clip DBClip,string DB,string PNDB,int StartFrame,int FindFrame, \
clip "FindClip"=NULL,int "Maxdistance"=DBClip.FrameCount-1,bool "Inclusive"=True,Int "LumaTol"=0, \
int "Flags"=0, Float "LD"=0.0,Float "LC"=0.0,Float "FD"=0.0,int "PC"=0,int "XP"=1, \
string "Prefix"="QSGN_",Float "ChromaWeight"=1.0/3.0,Int "PCThresh"=0)

*** EXPERIMENTAL ***

Developer function only, not for the casual scriptor.
Function that scans a pair of database files to find a sequence of candidate frames that could match a target FindFrame.
Returns frame number of possible matching frame, or -1 meaning NOT FOUND.
DBClip, the clip to search, and used as arg to RT_QwikScanCreate to create DB, PrevDB and NextDB database files that hold
"FingerPrint" and frame location data to assist in rapid location of candidate frames.
DB, Filename of the "FingerPrint" Dbase. Created via RT_QwikScanCreate.
PNDB, Filename of the frame locator Dbase. Created via RT_QwikScanCreate as either PrevDB or NextDB.
Use the PrevDB if searching frames prior to StartFrame in DBClip, or the NextDB if search frames after StartFrame.
StartFrame, frame in DBClip where searching will start, see also Inclusive.
FindFrame, Frame number of the frame that is to be matched (in FindClip if supplied or DBClip if not supplied).
FindClip, clip holding frame to match, defaults to DBClip if not supplied.
MaxDistance, maximum number of frames to search, defaults DBClip.FrameCount -1, but silently limited to the maximum possible
distance depending upon where StartFrame is and whether searching forwards or backwards. A MaxDistance of 1 will search only
the StartFrame (if Inclusive) and the frame next to it.
Inclusive. If true (default) then also tests StartFrame.
LumaTol, +- Luma Tolerance (0 -> 16). Allow for mismatch in luma between DBClip and FindClip. If eg the FindClip pixels can vary
by as much as +- 4 pixel levels, then set LumaTol to 4, the bigger this number the slower the function, but it should be still
quite fast with most clips even up to a LumaTol of maximum 16, the bigger this number, the more frames will fit the fingerprint
and so there will be more candidate frames to check.
Flags, Bit flags, sets the additional Exit Conditions of the function.
Bit 0 set eg 0x01, Sets the LumaDifference Exit Condition.
Bit 1 set eg 0x02, Sets the LumaCorrelation Exit Condition.
Bit 2 set eg 0x04, Sets the FrameDifference Exit Condition.
Bit 3 set eg 0x08, Sets the PixelsDifferentCount Exit Condition.
LD, LumaDifference Exit Condition Threshold.
LC, LumaCorrelation Exit Condition Threshold.
FD, FrameDifference Exit Condition Threshold.
PC, PixelsDifferentCount Exit Condition Threshold (int). See Also PCThresh.
XP, Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=Continue searching while next frame is a better Exit Condition match.
2=Continue searching while next frame is a better or equal Exit Condition match.

Prefix, Default "QSGN_". Prefix for names of local variables returned to give status of exit conditions.
All Below only set if Flags arg non zero.
QSGN_Exit_Flags, bit flag, condition that was satisfied to cause exit (same as Flags arg, only 1 bit will be set).
Below Only set if QSGN_Exit_Flags != 0 (where function returns a valid candidate frame number).
QSGN_Exit_LD, LumaDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x01.
QSGN_Exit_LC, LumaCorrelation for the exit frame, Only set if QSGN_Exit_Flags == 0x02.
QSGN_Exit_FD, FrameDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x04.
QSGN_Exit_PC, PixelsDifferentCount for the exit frame, Only set if QSGN_Exit_Flags == 0x08.
QSGN_Exit_XP, Count of additional frames traversed if found to be a better match. Only set if both XP arg is non zero
Below Only set if Function returns -1 NOT FOUND, AND QSGN_BM_Flags != 0 (QSGN_Exit_Flags will be 0 if NOT FOUND)
QSGN_BM_LD, LumaDifference of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x01.
QSGN_BM_LD_Frm, LumaDifference BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x01.
QSGN_BM_LC, LumaCorrelation of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x02.
QSGN_BM_LC_Frm, LumaCorrelation BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x02.
QSGN_BM_FD, FrameDifference of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x04.
QSGN_BM_FD_Frm, FrameDifference BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x04.
QSGN_BM_PC, PixelsDifferentCount of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x08.
QSGN_BM_PC_Frm, PixelsDifferentCount BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x08.

If the function returns -1 NOT FOUND, but an EXIT condition was set and at least one candidate frame was found but did
not satisfy any EXIT condition, then the BEST MATCH flags will be set (if none set then no candidate frames were found
that were within LumaTol of the target and so no EXIT conditions were tested). If QSGN_BM_Flags is non zero, it should be
exactly the same as Flags arg, with all selected BEST MATCH status variables returned as noted above.
Where DBClip and FindClip are quite different in eg brightness, It may be possible to have some kind of self tuning
routine. Find a single (or multiple) frames with maximum LumaTol, and EXIT conditions that are sure to fail, and use the
BEST MATCH results to tune both LumaTol and the EXIT conditions, for the remainder of the search session.
It might though be just as good to do some kind of Luma levelling on each clip, based on a sample number of frames
and using YPlaneMin and YPlaneMax followed by Levels (With Coring=False, and before DBase creation of course).

ChromaWeight, Argument to the FD FrameDifference Exit Condition (YUV only).

PCThresh, Range 0 -> 254. Argument to the PC PixelsDifferentCount Exit Condition. Difference between individual pixels has to be
greater than this to be counted. (default=0, count all differences).

The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
The PNDB Frame Locator Dbase holds records with 1024 fields per record (frame). The first 256 of these fields give frame numbers
of the next nearest frame (previous[PrevDB] or next[NextDB]) that roughly complies with the first component FingerPrint value.
Eg, field 42 holds the frame number of the nearest frame whose first component FingerPrint value is greater or equal to 42.0
and less than 43.0. Fields 256 to 511 hold frame info for the 2nd FingerPrint component, etc. A value of -1 means that there is
no frame whose fingerprint component complies.

This is roughly how the function works without Exit Conditions (Flags=0):-
It first establishes a range of component FingerPrint values for FindFrame, based on LumaTol.
If Inclusive, then checks StartFrame for a FingerPrint match and returns StartFrame if good match.
Next it Checks the Frame Locator Dbase (PNDB) looking for the next nearest frame that roughly complies with the FingerPrint
range. It then checks that candidate frame in the FingerPrint DBase to see if it more exactly matches the FingerPrint, if it
does match, the frame is returned to the client and if not then it moves the 'Cursor' in the Frame Locator DBase up to that
frame and repeats the searching until found or not.
So, without Exit conditions, the function returns a sequence of candidate frame numbers that may or may not be the exact frame
you are looking for, it will likely skip vast swathes of frames and then deliver a stream of frames from the same scene as
your FindFrame, where you would likely want to establish a more exact match using some other function.

With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a candidate frame
to the client. If an Exit condition is set, then that exit condition also has to be satisfied, if not satisfied, then the candidate
frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and the Exit
Condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, they are processed in bit number order ie LumaDifference condition first, the first
condition that succeeds will cause exit and no further exit conditions will be processed. If an Exit Condition is satisfied,
and the XP arg is set, the function will continue scanning adjacent frames and step forward until a better match for the Exit
Condition is NOT found. (XP=1 is better than, and XP=2 is better or equal).

Exit Conditions:-

LD, LD is the Threshold for RT_LumaDifference, if the match is less or equal to this then the Exit Condition is satisfied.

LC, LC is the Threshold for RT_LumaCorrelation, if the match is less or equal to this then the Exit Condition is satisfied.
We have 'swizzled' the values returned by RT_LumaCorrelation(), instead of the normal range -1.0 -> 1.0,
we use 0.0 -> 255.0 where 0.0 means exact match. We have done this for 2 reasons. Firstly, to make it the same (roughly)
as the other exit conditions, the closer to zero, the better the match. 2nd reason is, Avisynth Floats have only 6 significant
digits of decimal precision, so eg 0.999999 is about a good as it gets (closest to 1.0 without actually being 1.0) if we were
setting a threshold for normal range LumaCorrelation (-1.0 -> 1.0), but when we 'swizzle' the numbers to make 0.0 an exact
match, we can supply a more precise threshold which will be used at double precision within the function,
eg 0.0000000000000000000000000000001 is still a valid float (and in this case it is only 1 significant digit of precision).
RT_LumaCorrelation is slower than RT_LumaDifference but it may be more appropriate if there is a big difference in
luma beteen DBClip and FindClip, it is less perturbed by difference in brightness or contrast.

FD, FD is the Threshold for RT_FrameDifference, if the match is less or equal to this then the Exit Condition is satisfied.
The RT_FrameDifference Exit Condition uses the ChromaWeight arg to weight chroma for YUV colorspaces.
For RGB, RT_FrameDifference is equivalent to RGBDifference. For Y8 it is equivalent to LumaDifference.
For non Y8 YUV, returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
EXPERIMENTAL: Also currently accepts ChromaWeight -ve (eg -1.0) where will return
Lumadif + (1.0 - Lumadif/255.0) * ((Udif + Vdif)/2.0)
This may well be removed at some future date (although would do no harm to leave in-situ).

PC, PC is the Threshold for RT_LumaPixelsDifferent, if the match is less or equal to this then the Exit Condition is satisfied.
PC is a pixel count, ranging 0 -> (Width * Height). Allows you to set maximum number of pixels that are different.
Also takes PCThresh as an argument, difference between individual pixels has to be greater than this to be counted.
(PCThresh default=0, count all differences).
(RGB converted to Luma-Y).

Usage:-
Using PAL SD Jurassic Park with 182050 frames (UT_Video Compressed YV12) it took 15:06 to create DB and PrevDB FingerPrint
and Locator DB on a 2.13Ghz Core Duo, using this script.

AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
return RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)

And test script searching approx 175,000 frames per FindFrame:
AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
#RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)
GScript("""
LUMATOL=0
End=FrameCount-1
For(i=3000,13000,1000) {
S=RT_TimerHP()
j = RT_QwikScanGetNear(Last,DB,PNDB,End,i,Last,maxdistance=End-i,lumatol=LUMATOL,flags=$04,fd=0.0,xp=2)
T=RT_TimerHP-S
RT_Debugf("%d] Got %d in %f secs",i,j,T)
}
""")
return Last

Results:- LumaTol = 0
00000005 398.69018555 RT_DebugF: 3000] Got 3000 in 0.019132 secs
00000006 398.70736694 RT_DebugF: 4000] Got 4000 in 0.017023 secs
00000007 398.73089600 RT_DebugF: 5000] Got 5000 in 0.023335 secs
00000008 398.75195313 RT_DebugF: 6000] Got 6000 in 0.020897 secs
00000009 398.76660156 RT_DebugF: 7000] Got 7000 in 0.014457 secs
00000010 398.78686523 RT_DebugF: 8000] Got 8000 in 0.020073 secs
00000011 398.80563354 RT_DebugF: 9000] Got 9000 in 0.018632 secs
00000012 398.82150269 RT_DebugF: 10000] Got 10000 in 0.015690 secs
00000013 398.85241699 RT_DebugF: 11000] Got 11000 in 0.030706 secs
00000014 398.86682129 RT_DebugF: 12000] Got 12000 in 0.014216 secs
00000015 398.88012695 RT_DebugF: 13000] Got 13000 in 0.013135 secs

Results:- LumaTol = 4 (many more candidate frames to test)
00000016 444.04776001 RT_DebugF: 3000] Got 3000 in 0.059216 secs
00000017 444.26177979 RT_DebugF: 4000] Got 4000 in 0.213837 secs
00000018 446.82482910 RT_DebugF: 5000] Got 5000 in 2.562858 secs
00000019 454.65585327 RT_DebugF: 6000] Got 6000 in 7.830853 secs
00000020 456.44543457 RT_DebugF: 7000] Got 7000 in 1.789409 secs
00000021 465.26766968 RT_DebugF: 8000] Got 8000 in 8.822010 secs
00000022 465.99948120 RT_DebugF: 9000] Got 9000 in 0.731642 secs
00000023 466.64910889 RT_DebugF: 10000] Got 10000 in 0.649458 secs
00000024 471.09573364 RT_DebugF: 11000] Got 11000 in 4.446451 secs
00000025 471.81393433 RT_DebugF: 12000] Got 12000 in 0.718018 secs
00000026 475.26538086 RT_DebugF: 13000] Got 13000 in 3.451269 secs


Get it here:- http://www.mediafire.com/download/jn3l36pb9zo63cl/RT_Stats_25%2626_dll_v1.40Beta02_20140529.zip

[jmac698]

Hi,
Your RT_Stats has really grown! I'm quite impressed, there's a lot of useful stuff in there. I made good use of it in http://forum.doom9.org/showthread.php?t=170995, but I noticed a few things. Your idea to convert a float to a colour was useful to me, but you didn't complete the idea - you need a way to store/plot the colour into a frame. I came up with a way to plot with layer but it's slow and limited. I've always wanted to operate on images as arrays.

What I would like is something like RT_PutImageAsArrayByte(clip, frame, value, index), RT_GetImageAsArrayByte(clip,frame,value,index), PutImageAsArrayInt(clip, frame, value, index) etc.

These are basically plot functions. Index scans left-to-right and top-to-bottom. This is determined from the clip size. So an 8x8 clip hold 64 values. You could also write a script function to turn it into a (luma) plot. Int would naturally be stored as YU then YV, which is a bit awkward for (writing a function to make a colour) plot, so maybe you could add a real plot as well :)


Another thing I noticed is that the timer doesn't work for the "compile" phase of the script, which is where all my loops are run, so I can't time them. I suppose avstimer is the solution.

[StainlessS]

Already answered (+ve-ley) in your thread.

[StainlessS]

Another thing I noticed is that the timer doesn't work for the "compile" phase of the script, which is where all my loops are run, so I can't time them. I suppose avstimer is the solution.

No idea what you mean, I use it at compile time all of the time.


START=RT_TimerHP() # or RT_Timer(), matched pairs ONLY.
...
END=RT_TimerHP()
TIME_TAKEN = END - START # In seconds


EDIT:

RT_Timer()
Returns time in seconds since start of process.
Usage:
s=RT_Timer()
Test()
e=RT_Timer()
Str="Test Start="+String(s,"%.2f")+" End="+String(e,"%.2f")+" Time="+String(e-s,"%.2f")+"secs"
SubTitle(str)
RT_Debug(str)


RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elapsed in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only thing that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str="Test Start="+String(s,"%.3f")+" End="+String(e,"%.3f")+" Time="+String(e-s,"%.6")+"secs"
SubTitle(str)

[Seedmanc]

Is there any way to measure rendering speed of a script inside of it, using RT? To disable heavy parts of processing dynamically, for example.
I came up with something quick and dirty like this without gscript
pre=0.0
post=0.0
frames=0.0
fps=0.0
scriptclip("
\ post=RT_Timer()
\ time=post-pre
\ pre=(time>=1)?post:pre
\ fps=(time>=1)?current_frame-frames:fps
\ frames=(time>=1)?current_frame:frames
\RT_Subtitle(string(fps))")
It works, but I'm sure there should be a way to make it look less ugly, maybe even an inbuilt function.

Also, another question, how heavy are RT functions? I read somewhere that they are generally slower than similar (if you can even compare) non-runtime functions given their execution for every frame.
The reason I'm asking both questions is that I'm looking for ways of optimizing a somewhat exotic way of using Avisynth - real-time processing of video, captured from a videogame for output on a larger screen. Thus the speed and lag introduced by a script matters. Here's my thread about it, if you wonder: http://forum.doom9.org/showthread.php?t=171105

[StainlessS]

How about this

Avisource("Test.avi")

Start = RT_Timer() # Timing relative to start ie average timing for entire length of clip so far.
CondS = """
T=RT_Timer
RT_Subtitle("%.3f",current_frame/Max(T-Start,0.000001))
Return Last
"""
ScriptClip(CondS)


To disable heavy parts of processing dynamically

What do you mean by that ?

You cannot generally compare runtime functions with filters, they do a different job.
eg, AverageLuma() returns a float, not a clip.
Some filter functions may eg return the same value as AverageLuma would but as a local variable rather than a function return (ShowChannels can do that).
Any standard filter used in eg Scriptclip, has it's constructor called at every frame, and also destroyed at every frame, and so filters that have 'heavy'
start up costs, have them at every frame.

[StainlessS]

New version RT_Stats v1.40.
Post #1 of 3

Functions added.

RT_WriteFile(String FileName, string format,dat1,...,datn,bool "Append"=False)
Writes a formatted string to FileName file. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the
text string that is written, uses C/CPP printf() style formatting.
Format: compulsory string controlling format and describing the datn type args that are expected.
datn: Variable number of data args of any type (excluding clip).
Append, Default False, optional and MUST be supplied with name (eg Append=True) as we do not know where optional data args end,
Appends to file if already exists, else opens new FileName.

printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned (o=octal, x=Hex).
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

Formatting Insertion point is marked with '%' character in the format string (as in Avisynth String function), if you wish to use
a percent character within the output string, it should be inserted twice in the format string ie '%%' will produce a single '%'.
The data datn arg strings do not require a double '%' character.

A Backslash character '\' introduces escape sequences, to insert a backslash character itself, you must supply a double
backslash sequence ie '\\'.
Converts embedded escape character sequences (Case Significant):-
'\\' Converted to '\' Single Backslash
'\a' Converted to Chr(7)
'\b' Converted to Chr(8)
'\t' Converted to Chr(9)
'\n' Converted to Chr(10) NewLine
'\r' [and Chr(13)] Converted to Chr(10) NewLine
'\v' Converted to Chr(11)
'\f' Converted to Chr(12)
'\x', where x is ANY OTHER CHARACTER not included above, will be copied verbatim, ie '\x'.

Escape sequences are replaced only when occurring in the format string, if you need escapes in data string then use RT_String.

Example:
RT_WriteFile(FileName,"Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013,Append=False)

***
***
***

RT_ChanAve(clip c,int "n"=current_Frame,string "Prefix"=RCA_")
Sets channel averages for clip c frame n, as local variables, 0.0 -> 255.0. RGB24/32, YUY2, Planar.
Returns the number of channels in clip ie 3 unless Y8 where returns 1.
Default Prefix is "RCA_" so for YV12, would return 3 (3 channels) and set
YUV
RCA_Ave_0 = Luma Ave
RCA_Ave_1 = U Ave
RCA_Ave_2 = V Ave.
RGB,
RCA_Ave_0 = Red Ave
RCA_Ave_1 = Grn Ave
RCA_Ave_2 = Blue Ave.
Y8
RCA_Ave_2 = Luma Ave
RCA_Ave_2 = 128
RCA_Ave_3 = 128,
NOTE, No support for alpha channel of RGB32, although could be added if anybody thought it useful.(returns num of channels = 3)

***
***
***

RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0)
Returns difference between clip c frame n and clip c2 frame n2 (0.0 -> 255.0).
ChromaWeight range 0.0 -> 1.0.
For RGB, returns same as RGBDifference.
For Y8 returns same as LumaDifference.
For non Y8 Planar returns Lumadif + Chromadif with chroma weighted by ChromaWeight arg.
Returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).

***
***
***

RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=current_frame,int "x2"=0,int"y2"=0,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns amount of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Thresh Default 0, returns number of pixels that are not exactly the same.
Return value is in range 0.0 (meaning none) to 255.0 meaning 100% of pixels.

***
***
***

RT_LumaPixelsDifferentCount(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=current_frame,int "x2"=0,int"y2"=0,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns number of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Thresh Default 0, returns number of pixels that are not exactly the same.
Note, returns the number of pixels whose difference is greater than Thresh, so ranges 0 -> (Width*Height).

***
***
***

RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elapsed in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only thing that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str="Test Start="+String(s,"%.3f")+" End="+String(e,"%.3f")+" Time="+String(e-s,"%.6")+"secs"
SubTitle(str)

***
***
***

RT_SignalDAR2(int darx,int dary)
As for RT_SignalDAR() except it allows setting of numerator and denominator individually.
Signal DAR to MEGUI. Sets Global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

***
***
***

RT_VarType(var)
Returns type of var arg as an int.
Returns,
0 = Type BOOL
1 = Type INT.
2 = Type FLOAT
3 = Type STRING
4 = Type CLIP
-1 = Type UNDEFINED
Avoids multiple tests in script to establish a variable type.
eg, RT_VarType(66.6) returns 1, ie FLOAT.
NOTE, If called without an arg, it will return type of implicit Last (Undefined if no clip implicitly/explicitly assigned to Last).

***
***
***

RT_VarIsSame(var Var1, ...,var Varn,Bool "Sig")
Accepts two or more variables of any type. NOTE, if only 1 arg supplied, will use implicit Last as first arg.
Returns Bool, False if args are not all of same type AND/OR value.
Return True, if all args same type and value.
Sig, Bool Default True. Only used where variables are either String or Clip.
Type String,
Compares strings same length and contents.
Sig==True, case significant comparison ('a' != 'A').
Sig==False, case insignificant comparison ('a' = 'A').
Type Clip,
Compares clip video attributes, frame Width, Height, ColorSpace.
If first clip has no video (HasVideo==False) then all other clips must also have no video (other video attributes not tested).
If Sig==False, Audio Not considered in comparison.
If Sig==True, audio is considered.
If first clip has no audio (HasAudio==False) then all other clips must also have no audio (other audio attributes not tested).
If first clip does have audio, then all other clips must have audio and same Number of Audio Channels, Sample Rate,
and Sample Type (int16, Float etc).
No other attributes are taken into account, (ie FrameRate, FrameCount etc).
*** NOTE ***, as we cannot tell where variable number of Varn args ends, you must use optional 'sig' arg with the name ie 'sig=True'.

***
***
***

RT_DBaseGetStrAttrib(String Filename,Int ix)
There are 10 (currently) available String attributes in the DataBase file header, that can be used for whatever purpose you like, they are
1024 (maximum) characters in length excluding nul terminating character (nul not stored in file).
Filename, DBASE Filename, no default.
Ix, String Attribute array element index range 0 -> 9.
Returns the value of the user set string attribute.
NOTE, if supplied ix is -1, then will return the number (Int) of available string attributes (currently 10).
NOTE, if supplied ix is -2, then will return the maximum string length allowed (Int) of string attributes (currently 1024).

***
***
***

RT_DBaseSetStrAttrib(String Filename,Int ix, String s)
There are 10 (currently) available string attributes in the DataBase file header, that can be used for whatever purpose you like,
RT_DBaseAlloc() initializes all String Attributes to "", ie zero length.
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 9.
s, The string that String_Attrib(ix) will be set to.
Returns 0. Error if type not of type String.

***
***
***

RT_DBaseTypeName(Int Type)
Type, Int (0 -> 5), as value returned by RT_DBaseFieldType(Filename,Field), returns the type as a string.
Returns 0="Bool", 1="Int", 2 = "Float", 3 = "String", 4 = "Bin", 5 = "Double".
NOTE, type 5 "Double" is an RT_Stats Private type, ie used internally by RT_Stats to store floating point values as double,
if used from within Avisynth script, will get and set values as Type Float (but stored in DB as double, 8 bytes instead of 4).
There is little point to using type double in avisynth script, will just make files bigger and take a little longer to access.

***
***
***

RT_ArrayGetStrAttrib(String Filename,Int ix)
There are 10 (currently) available String attributes in the Array file header, that can be used for whatever purpose you like, they are
1024 (maximum) characters in length excluding nul terminating character (nul not stored in file).
Filename, Array Filename, no default.
Ix, String Attribute array element index range 0 -> 9.
Returns the value of the user set string attribute.
NOTE, if supplied ix is -1, then will return the number (Int) of available string attributes (currently 10).
NOTE, if supplied ix is -2, then will return the maximum string length allowed (Int) of string attributes (currently 1024).

***
***
***

RT_ArraySetStrAttrib(String Filename,Int ix, String s)
There are 10 (currently) available string attributes in the Array file header, that can be used for whatever purpose you like,
RT_ArrayAlloc() initializes all String Attributes to "", ie zero length.
Filename, ARRAY Filename, no default.
Ix, Attribute array element index range 0 -> 9.
s, The string that String_Attrib(ix) will be set to.
Returns 0. Error if type not of type String.

***
***
***

RT_ArrayTypeName(Int Type)
Type, Int (0 -> 5), as value returned by RT_DBaseFieldType(Filename,Field), returns the type as a string.
Returns 0="Bool", 1="Int", 2 = "Float", 3 = "String", 4 = "Bin", 5 = "Double".
NOTE, type 5 "Double" is an RT_Stats Private type, ie used internally by RT_Stats to store floating point values as double,
if used from within Avisynth script, will get and set values as Type Float (but stored in DB as double, 8 bytes instead of 4).
There is little point to using type double in avisynth script, will just make files bigger and take a little longer to access.

***
***
***

RT_FileRename(String OldFilename,String NewFileName)
Renames a file from OldFilename to NewFilename.
This function renames the file or directory specified by OldFilename to the name given by NewFilename.
The OldFilename must be the path of an existing file or directory.
The NewFilename must not be the name of an existing file or directory.
You can use the function to move a file from one directory or device to another by giving a different path in the NewFileName argument.
However, you cannot use the function to move a directory. Directories can be renamed, but not moved.

Return 0 on success.
On Error, returns
-1, Access denied or read only, or NewFileName already exists or could not be created (invalid path),
or oldname is a directory and newname specifies a different path.
-2, OldFilename File or Path not found.
-3, NewFileName contains invalid characters.
-4, Unknown error.

***
***
***

RT_QwikScanCreate(clip c,string DB,string "PrevDB"="",string "NextDB"="",int "matrix"=(Width>1100||Height>600?3:2),bool "Debug"=False)
Creates a DataBase set used to quickly scan a clip for frame matching, DataBases used by RT_QwikScan.
c, clip used to create database frame fingerprint info.
PrevDB, DBase filename, used to find frames prior to a 'start frame'.
NextDB, DBase filename, used to find frames after a 'start frame'.
Either PrevDB or NextDB (or both) must be given.
Matrix, used to convert RGB to Luma_Y for Fingerprint creation.
Debug, True sends a little info to DebugView window (google).
Maximum PrevDB/NextDB size is restricted to 524283 records(frames).

***
***
***

[StainlessS]

New version RT_Stats v1.40.
Post #2 of 3

RT_QwikScan(clip SearchClip,Int SeachStart,clip FindClip,Int FindFrame,string DB,string PNDB,
\ Int "LumaTol"=1,int "Flags"=1,
\ Float "LC"=1.0,Float "LD"=1.0,Float "FD"=1.0,Float "PD"=1.0,int "PC"=(SearchClip.Width*SearchClip.Height+128)/255, int "XP"=1,
\ Float "ChromaWeight"=1.0/3.0,Int "PDThresh"=0
\ int "Maxdistance"=SearchClip.FrameCount-1,bool "Inclusive"=True,string "Prefix"="QWKS_")

Developer function only, not for the casual scriptor.
Function that scans a clip using pair of database files to find a sequence of candidate frames that could match a target FindFrame.
Returns frame number of possible matching frame, or -1 meaning NOT FOUND.

ARGS:-

SearchClip, the clip to search, and used as arg to RT_QwikScanCreate to create DB, PrevDB and NextDB database files that hold
"FingerPrint" and frame location data to assist in rapid location of candidate frames.

SearchStart, frame number in SearchClip where searching will start, see also Inclusive.

FindClip, clip holding FindFrame to match.

FindFrame, Frame number of the FindClip frame that is to be matched.

DB, Filename of the 'FingerPrint' Dbase. Created via RT_QwikScanCreate.

PNDB, Filename of the frame locator Dbase. Created via RT_QwikScanCreate as either PrevDB or NextDB.
Use the PrevDB if searching frames prior to SearchStart in SearchClip, or the NextDB if search frames after SearchStart.
See also 'Inclusive'.

OPTIONAL ARGS:

LumaTol (default 1), +- Luma Tolerance (0 -> 16). Allow for mismatch in luma between SearchClip and FindClip. If eg the FindClip
pixels can vary by as much as +- 4 pixel levels, then set LumaTol to 4, the bigger this number the slower the function, but it
should be still quite fast with most clips even up to a LumaTol of maximum 16, the bigger this number, the more frames will fit
the fingerprint and so there will be more candidate frames to check.

Flags (Int, default=1 ie LumaCorrelation Exit Condition), Bit flags, sets the additional Exit Conditions of the function.

Bit 0 set eg $01, Sets the LumaCorrelation Exit Condition.
Bit 1 set eg $02, Sets the LumaDifference Exit Condition.
Bit 2 set eg $04, Sets the FrameDifference Exit Condition.
Bit 3 set eg $08, Sets the PixelsDifferent Exit Condition.
Bit 4 set eg $10, Sets the PixelsDifferentCount Exit Condition.

To eg set Exit conditions LumaCorrelation + FrameDifference + PixelsDifferent, then add 1 + 4 + 8 (1+4+8=13=$0D).

If exit conditions set, then there may be info generated in the Exit and Best Match returned local variables.

LC, LumaCorrelation Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 0 of Flags is set.
LC is the Threshold for RT_LumaCorrelation, if the match is less or equal to this then the Exit Condition is satisfied.
We have 'swizzled' the values returned by RT_LumaCorrelation() here to something akin to half range "Pearson's Distance".
See Pearson’s Distance here: http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
where Pearson's Distance is equivalent to:

PD(x,y) = 1.0 - Correlation(x,y)

and our Swizzled half range version using equivalent to:-

SwizzledLC(x,y) = Min( (1.0 - RT_LumaCorrelation(x,y)) * 255.0, 255.0 )

Correlation Pearsons_Distance Swizzled_LC
1.0 0.0 0.0 Total +ve Correlation
0.0 1.0 255.0 UnCorrelated
-1.0 2.0 255.0 Total -ve Correlation

so instead of the RT_LumaCorrelation normal range -1.0 -> 1.0, we use 0.0 -> 255.0 where 0.0 means exact match and 255.0 means
unmatched, Swizzled RT_LumaCorrelation results that were -ve are clipped at 255.0 so we consider Total -ve Correlation to be no
more distant than UnCorrelated, ie photo invert of find frame cannot be more distant than 'totally unmatched'.
We have done this for 2 reasons. Firstly, to make it the same (roughly) as the other exit conditions, the closer to zero, the
better the match. 2nd reason is, Avisynth Floats have only 6 significant digits of decimal precision. So eg 0.999999 is about as
good as it gets (closest to 1.0 without actually being 1.0). When we 'swizzle' the numbers to make 0.0 an exact match, we can
supply a more precise threshold which will be used at double precision within the function.
Eg 0.0000000000000000000000000000001 is still a valid float (and in this case it is only 1 significant digit of precision).
Despite the swizzling, we will keep the name LC LumaCorrelation (PD already taken).
LumaCorrelation is slower than LumaDifference but it may be more appropriate if there is a big difference in
luma beteen SearchClip and FindClip, it is less perturbed by difference in brightness or contrast.
See RT_LumaCorrelation.

LD, LumaDifference Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 1 of Flags is set.
LD is the Threshold for RT_LumaDifference, if the match is less or equal to this then the Exit Condition is satisfied.
See RT_LumaDifference.

FD, FrameDifference Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 2 of Flags is set.
FD is the Threshold for RT_FrameDifference, if the match is less or equal to this then the Exit Condition is satisfied.
The RT_FrameDifference Exit Condition uses the ChromaWeight arg to weight chroma for YUV colorspaces.
For RGB, RT_FrameDifference is equivalent to RGBDifference. For Y8 it is equivalent to LumaDifference.
For non Y8 YUV, returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
See RT_FrameDifference.

PD, PixelsDifferent Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 3 of Flags is set.
PD is the Threshold for RT_LumaPixelsDifferent, if the match is less or equal to this then the Exit Condition is satisfied.
PD is amount of pixels, that are different with 0.0 meaning none and 255.0 meaning 100%.
Also takes PDThresh as an argument, difference between individual pixels has to be greater than this to be counted.
PixelsDifferent is similar to below PixelsDifferentCount, but using a frame size relative way of specifiying the exit condition.
(PDThresh default=0, count all differences). (RGB converted to Luma-Y).
See RT_LumaPixelsDifferent and also PDThresh.

PC, PixelsDifferentCount Exit Condition Threshold (int, default=(SearchClip.Width*SearchClip.Height+127)/255,
Range 0 -> SearchClip.Width*SearchClip.Height).
Selected when bit 4 of Flags is set.
PC is the Threshold for RT_LumaPixelsDifferentCount, if the match is less or equal to this then the Exit Condition is satisfied.
PC is a pixel count, ranging 0 -> (Width * Height). Allows you to set maximum number of pixels that are different.
Also takes PDThresh as an argument, difference between individual pixels has to be greater than this to be counted.
(PDThresh default=0, count all differences). (RGB converted to Luma-Y).
See RT_LumaPixelsDifferentCount and also PDThresh.

XP, (Default 1), Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=Continue searching while next frame is a better Exit Condition match ('next frame' meaning dependant upon search direction).
2=Continue searching while next frame is a better or equal Exit Condition match ('next frame' meaning dependant upon search direction).
When an exit condition occurs eg LumaDifference, then QWKS_BM_LD_XP will be set to the count of additional frames traversed due
to finding a match that is better or equal (depending upon whether XP==1 or XP==2) to the initial frame that caused the exit condition,
In such cases, both QWKS_BM_LD and QWKS_BM_LD_FRM will also be set to (in this case) LumaDifference and the final frame (with XP offset)
that was found to be a better or equal match.

ChromaWeight (Float, default 1.0/3.0, Range 0.0 -> 1.0), Argument to the FD FrameDifference Exit Condition (YUV only).

PDThresh, (Int, Default 0, Range 0 -> 255). Argument to the PD PixelsDifferent and PC PixelsDifferentCount Exit Conditions.
Difference between individual pixels has to be greater than this to be counted. (default=0, count all differences).

MaxDistance (Int, default SearchClip.FrameCount-1). Maximum number of frames to search, defaults SearchClip.FrameCount-1, but silently
limited to the maximum possible distance depending upon where SearchStart is and whether searching forwards or backwards.
A MaxDistance of 1 will search only the SearchStart (if Inclusive) and the frame next to it.

Inclusive, (Bool Default True). If true (default) then also tests SearchStart.

Prefix, (String Default "QWKS_"). Prefix for names of local variables returned to give status of exit conditions and Best Match.

-------------------------------------------------------------------------

RETURNS:-
Returns frame number of CANDIDATE (possible matching frame), or -1 meaning NOT FOUND.

LOCAL VARIABLES returned where using default Prefix of "QWKS_":

QWKS_EXIT_FLAGS,Int, bit flags, (always exists). If Exit condition occurred then there will be one of more bits set for the conditions
that caused exit, bit numbers same as described for Flags arg. QWKS_EXIT_FLAGS will always be 0 when flags arg == 0 (no exit
conditions required) OR, function returned -1, ie frame NOT FOUND.
If exit condition occurred and XP arg is 0, then the function return frame and QWKS_BM_xx_FRM (described below) will be the same,
if XP arg is non zero, then XP processing is tried on all successful exit conditions and the QWKS_BM_xx_FRM frame numbers
(and concerned QWKS_BM_xx differences) may all be different from the function return frame. So, when XP processing, the
function return frame is only the initial frame that caused exit, XP results are in QWKS_BM_xx_FRM and QWKS_BM_xx.

QWKS_EXIT_COUNT, Int (always exists), Count of set bits in QWKS_EXIT_FLAGS (ie number of exit conditions that succeeded).

QWKS_BM_FLAGS,Int, bit flags, (always exists). Returns flag bits of valid BEST MATCHES found for requested exit conditions, bit numbers
are as described for Flags arg. If Flags arg == 0, then QWKS_BM_FLAGS will always be 0, ie no exit conditions were tried.
If one or more exit conditions caused exit then QWKS_BM_FLAGS will have the same bits set as QWKS_EXIT_FLAGS, and possibly other bits
set giving info on best matches found for other exit conditions that did not successfully cause exit.

QWKS_BM_COUNT, Int (always exists), Count of set bits in QWKS_BM_FLAGS. (ie number of Best Match vars set)

Below Only exist/updated if associated QWKS_BM_FLAGS bit is set.
NOTE, the below frame numbers are the frames after XP processing if XP arg non zero. The associated difference is also for the
frame after XP processing is done. If XP arg == 0, then all frame numbers (and associated differences) are for the function result
frame. (The function result frame is the first frame that was encountered that had a successful exit condition and does not have
XP processing applied, the valid frame numbers QWKS_BM_xx_FRM can all be different from the function return frame if better or equal
frames were found via XP processing).

QWKS_BM_LC,(Float) LumaCorrelation of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 0 is set.
QWKS_BM_LD,(Float) LumaDifference of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 1 is set.
QWKS_BM_FD,(Float) FrameDifference of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 2 is set.
QWKS_BM_PD,(Float) PixelsDifferent of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 3 is set.
QWKS_BM_PC,(Int) PixelsDifferentCount of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 4 is set.

QWKS_BM_LC_FRM,(Int) LumaCorrelation BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 0 is set.
QWKS_BM_LD_FRM,(Int) LumaDifference BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 1 is set.
QWKS_BM_FD_FRM,(Int) FrameDifference BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 2 is set.
QWKS_BM_PD_FRM,(Int) PixelsDifferent BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 3 is set.
QWKS_BM_PC_FRM,(Int) PixelsDifferentCount BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 4 is set.

QWKS_BM_LC_XP,Int. Number of additional frames traversed because a better or equal match was found for LumaCorrelation exit condition.
QWKS_BM_LD_XP,Int. Number of additional frames traversed because a better or equal match was found for LumaDifference exit condition.
QWKS_BM_FD_XP,Int. Number of additional frames traversed because a better or equal match was found for FrameDifference exit condition.
QWKS_BM_PD_XP,Int. Number of additional frames traversed because a better or equal match was found for PixelsDifferent exit condition.
QWKS_BM_PC_XP,Int. Number of additional frames traversed because a better or equal match was found for PixelsDifferentCount exit condition.
Above only exist/updated for conditions that have a bit set in QWKS_BM_FLAGS otherwise may not exist. They will exist
whether or not the associated exit condition succeded or whether XP arg is zero or non zero. In cases where XP arg was 0 or where
associated exit condition failed, or where XP processing failed to find a better or equal match, then the QWKS_BM_xx_XP
variables will contain 0 (will be non zero only where associated exit condition succeded and found a better/equal XP match).

[StainlessS]

New version RT_Stats v1.40 (see 1st post).
Post #3 of 3

RT_QwikScan continued:


If the function returns a valid frame number (greater or equal to 0) and exit conditions were set in Flags arg, then QWKS_EXIT_FLAGS
will be non zero and have 1 bit set for each condition that caused exit, and the best match status variables
will exist/be updated, the other status variables not flagged in QWKS_EXIT_FLAGS but which are flagged in QWKS_BM_FLAGS will also
exist and be valid, but are BEST MATCHES ONLY.

If the function returns -1 NOT FOUND, but an EXIT condition was set in Flags and QWKS_BM_FLAGS is non zero,
then all associated BEST MATCH status variables will exist and be set (associated to the QWKS_BM_FLAGS set bits).

-------------------------------------------------------------------------


FINGERPRINT & LUMATOL:
The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
Each FingerPint component is derived from a pixel count for a portion of the possible 0 -> 255 luma pixel value range.
Lets say that one of the fingerprint components is derived from the 128->191 luma pixel value range, the fingerpint component
is computed for every frame in the SearchClip eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height},
this result is in range 0.0 -> 255.0, with 255.0 being equal to 100%. [There would be another three fingerprint components computed,
in different ranges. We do not actually use range 128->191, and are reluctant to give any real used ranges as they may change, if you
want to see the real current ranges used, then see the current source code]. The 4 fingerprint components are stored in the
FingerPrint DB and are EXACT fingerprint component values. For the FindFrame, we use a FingerPrint component Range with LumaTol
tolerance. For FindClip(FindFrame), we might compute a fingerprint component range for [128->191]:
FingerPrintMin = {luma PixelCount in range [128+LumaTol] to [191-LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
FingerPrintMax = {luma PixelCount in range [128-LumaTol] to [191+LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
So using computed FingerPrintMin and FingerPrintMax for four different ranges as upper and lower limits, if any frame in the
SeachClip (DBase) has all four EXACT FingerPrint components within all four fingerprint component limits, then it is a candidate
frame and eligible for exit condition testing if exit conditions were set, or returned to the client as a result CANDIDATE frame if
exit conditions were not set.

The PNDB Frame Locator Dbase holds records with 1024 fields per record (frame). The first 256 of these fields give frame numbers
of the next nearest frame (previous[PrevDB] or next[NextDB]) that roughly complies with the first component FingerPrint value.
Eg, field 42 holds the frame number of the nearest frame whose first component FingerPrint value is greater or equal to 42.0
and less than 43.0. Fields 256 to 511 hold frame info for the 2nd FingerPrint component, etc. A value of -1 means that there is
no frame whose fingerprint component complies.

Without Exit Conditions:
This is roughly how the function works without Exit Conditions (Flags=0):-
It first establishes a range of component FingerPrint values for FindFrame, based on LumaTol.
If Inclusive, then checks SearchStart for a FingerPrint match and returns SearchStart if good match.
Next it Checks the Frame Locator Dbase (PNDB) looking for the next nearest frame that could comply with the FingerPrint
range. It then checks that candidate frame in the FingerPrint DBase to see if it more exactly matches the FingerPrint, if it
does match, the frame is returned to the client and if not then it moves the 'Cursor' in the Frame Locator DBase up to that
frame and repeats the searching until found or not.
So, without Exit conditions, the function returns a sequence of candidate frame numbers that may or may not be the exact frame
you are looking for, it will likely skip vast swathes of frames and then deliver a stream of frames from the same scene as
your FindFrame, where you would likely want to establish a more exact match using some other function.
Without Exit conditions, QWKS_EXIT_FLAGS and QWKS_BM_FLAGS will both be 0 and no associated exit condition nor Best Match
local variables will be set (may not even exist).

With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a candidate frame
to the client. If Exit condition are set, then at least one of those exit conditions also has to be satisfied, if not satisfied, then
the candidate frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and an Exit
condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, if any (or all) of the exit conditions succeed then there will be a +ve frame number
returned by the function, QWKS_EXIT_FLAGS and QWKS_BM_FLAGS bits will be set, the returned frame number will be the first frame
that satisfied an exit condition, and does NOT include any XP arg processing.
If any exit condition is satisfied, and the XP arg is set, the function will continue scanning adjacent frames to the frame
where an exit condition was satisfied and step onward until a better match for the exit condition is NOT found.
(XP=1 is better than, and XP=2 is better or equal). XP Processing occurs on satisfied exit conditions and the XP processed frame
numbers and differences are returned via QWKS_BM_xx and QWKS_BM_xx_FRM local vars.
When using exit conditions, use the return frame number only as an indicator of success, use the returned local vars irrespective of
whether or not XP processing is done (although if XP==0, then returned frame will be same as QWKS_BM_xx_FRM).

Tuning:
Where there is significant difference between SearchClip and FindClip, it could be problematic to find a suitable value for LumaTol.
If LumaTol is set too high, then each and every search could take much longer to complete, if set too low then will skip over possible
matches as it will see them as not fitting the LumaTol fingerprint range (and so not even included in Best Matches).
Where both matching clip frames are identical (NOT processed/re-encoded) then can use LumaTol=0 (default is 1).
If you eg apply a ColorYUV(Off_y = 4) on exact copy of frame to find in SearchClip, then you would need at least a LumaTol of 4 for
the FingerPrint locator to include the frame as a CANDIDATE frame, (in this case, every single pixel luma value would have had 4 added
to it), ColorYUV(Off_y = -4) would also be detected by LumaTol = 4, and in both cases where using LumaDifference exit condition, you
would need at least a LD of 4.0 to produce a satisfied exit condition, an LD of eg 1.0 would not produce an exit condition but would
give a BEST match for the correct frame, if using an LC exit condition on same clip then result diff would be about 0.0.
We did a test search on SD PAL Jurassic Park frame 173,000, a scene at end of movie where helicopter is flying away into the distance
across the ocean with caption "Directed by Steven Spielberg".
Where find frame had Blur(1.58,1.58) applied (ie maximum Blur) it was found that a LumaTol of 5 was required to enable the Fingerprint
locator to work at all. Even though the original and blurred frame looked reasonably similar, there were obvious differences eg the the
blurred one looked more dull with indistinct text. Tests showed that there were 2 pixels that were different by 75 luma pixel levels
and there were no doubt many more that were different to a lesser degree. So, where processing akin to a maximum blur is suspected, a
rough guess might be about LumaTol=5. Results below where Flags=1, LC=0.0 and StartFrame = 0 (search from frame 0)
(The below FindFrm->FoundFrm lumadiff is carried out on the found frame, not returned from RT_QwikScan).

FindFrm->FoundFrm
LumaTol SECONDS EXIT_FLAGS BM_FLAGS RESULT_FRAME BM_LC RT_LumaDifference
0 0.011211 0 0 -1 --- ---
1 0.593617 0 1 173087 46.869492 12.616102
2 0.411454 0 1 173005 19.927631 7.802255
3 2.059347 0 1 173004 17.460129 7.176220
4 3.024366 0 1 173004 17.460129 7.176220
5 5.267460 0 1 173000 2.454145 2.577775 <---
6 9.544320 0 1 173000 2.454145 2.577775
8 18.132370 0 1 173000 2.454145 2.577775
12 55.960625 0 1 173000 2.454145 2.577775
16 129.069290 0 1 173000 2.454145 2.577775

It may be possible to write a script function to tune for LumaTol, if a search fails (and you know that the frame is present),
it could search on LumaCorrelation LC=0.0 (LC less perturbed by brightness/contrast changes) and step through LumaTol from 0 upwards
until BM_LC no longer changes and QWKS_BM_LC is below some threshold (maybe eg 2.55 ie 1.0%). Also note above that BM_LC is smaller
than RT_LumaDiff(FindFrm->FoundFrm) when LumaTol is at lowest active setting (5).
When acceptable LumaTol is found then might test using eg RT_LumaDifference between FindFrame and FoundFrame and after adding
a little leeway get an approximate value for doing an LD LumaDifference search (if that were what was required exit condition).
NOTE above, even when LumaTol at maximum 16 with timimg about 2 Mins, it is still considerably quicker than approx 18 Mins to
to find Frame 173000 (1hr:55:20) doing a standard LumaDifference search (using MatchFrames.avs), although would also need to
create the DBase set too (only once if multiple frames).
There is much room for research into how best to use this function.

Usage:-

Eg, send to debug window if BEST MATCH for LumaDifference was established (bit 1).
(RT_BitTst(QWKS_BM_FLAGS,1)) ? RT_DebugF("Best Match for LumaDifference = %f @ frame Number %d",QWKS_BM_FD,QWKS_BM_FD_FRM) : NOP
Eg, send to debug window if LumaDifference caused exit(bit 1).
(RT_BitTst(QWKS_EXIT_FLAGS,1)) ? RT_DebugF("LumaDifference Caused EXIT: Diff = %f @ frame Number %d",QWKS_BM_FD,QWKS_BM_FD_FRM) : NOP

Using PAL SD Jurassic Park with 182050 frames (UT_Video Compressed YV12) it took 15:06 to create DB FingerPrint DB and PrevDB
Locator DB on a 2.13Ghz Core Duo, using this script. (NOTE, The time taken to create all three DataBases, is not much different
to the time it would take to scan the clip to find a single frame using usual methods.
EDIT: Test for single frame match using MatchFrames.avs took ~18:20 to locate a single frame, scanning nearly entire clip using
RT_FrameDifference as locator, so if searching for more than one frame it is a small price to pay, and perhaps {as in this case}
even quicker than a standard single frame search (it is quicker to create a DB FingerPrint for one frame than to compare two frames).

AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="JPARK_PREV.DB"
return RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)

# And test script searching approx 175,000 frames per FindFrame:
AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="JPARK_PREV.DB"
#RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)
GScript("""
LUMATOL=0
SearchStart=FrameCount-1 # Search direction from end to beginning, using PrevDB.
For(FindFrame=3000,13000,1000) {
S=RT_TimerHP()
j = RT_QwikScan(Last,SearchStart,Last,FindFrame,DB,PNDB,lumatol=LUMATOL,Flags=1,LD=0.0,maxdistance=SearchStart-FindFrame)
T=RT_TimerHP-S
RT_Debugf("%d] Got %d in %f secs",i,j,T)
}
""")
return Last

Results:- LumaTol = 0
00000005 398.69018555 RT_DebugF: 3000] Got 3000 in 0.019132 secs
00000006 398.70736694 RT_DebugF: 4000] Got 4000 in 0.017023 secs
00000007 398.73089600 RT_DebugF: 5000] Got 5000 in 0.023335 secs
00000008 398.75195313 RT_DebugF: 6000] Got 6000 in 0.020897 secs
00000009 398.76660156 RT_DebugF: 7000] Got 7000 in 0.014457 secs
00000010 398.78686523 RT_DebugF: 8000] Got 8000 in 0.020073 secs
00000011 398.80563354 RT_DebugF: 9000] Got 9000 in 0.018632 secs
00000012 398.82150269 RT_DebugF: 10000] Got 10000 in 0.015690 secs
00000013 398.85241699 RT_DebugF: 11000] Got 11000 in 0.030706 secs
00000014 398.86682129 RT_DebugF: 12000] Got 12000 in 0.014216 secs
00000015 398.88012695 RT_DebugF: 13000] Got 13000 in 0.013135 secs

Results:- LumaTol = 4 (many more candidate frames to test)
00000016 444.04776001 RT_DebugF: 3000] Got 3000 in 0.059216 secs
00000017 444.26177979 RT_DebugF: 4000] Got 4000 in 0.213837 secs
00000018 446.82482910 RT_DebugF: 5000] Got 5000 in 2.562858 secs
00000019 454.65585327 RT_DebugF: 6000] Got 6000 in 7.830853 secs
00000020 456.44543457 RT_DebugF: 7000] Got 7000 in 1.789409 secs
00000021 465.26766968 RT_DebugF: 8000] Got 8000 in 8.822010 secs
00000022 465.99948120 RT_DebugF: 9000] Got 9000 in 0.731642 secs
00000023 466.64910889 RT_DebugF: 10000] Got 10000 in 0.649458 secs
00000024 471.09573364 RT_DebugF: 11000] Got 11000 in 4.446451 secs
00000025 471.81393433 RT_DebugF: 12000] Got 12000 in 0.718018 secs
00000026 475.26538086 RT_DebugF: 13000] Got 13000 in 3.451269 secs

[jmac698]

Amazing work, from what I can tell, it's a database to quickly match frames in a large range, even approximately. That could help find the different edits in laser/vhs/dvd copies of a movie, or on a smaller scale, find the dropped/duped frames that always show in multiple passes of captures.

Just an idea for autotuning things in general I had though, I used strong matches to get statistics to tune other matches. For example in an experiment with matching by luma correlation on extremely noisy frames, I found a strong indication of where one frame was different from others in the 5 copies, but somewhat similar to itself in the 5 copies, and between the two thresholds I could find other frames. This then helped where there was less motion and less of a strong indicator that the frame was unique compared to surrounding, but then relied more on the indicator that the same frames correlated strongly.

I mean,
p1a p1b p1c
p2a p2b p2c
p3a p3b p3c
I find that p1b<<>>p1a,p1c and p2b<<>>p2a,p2c and p3b<<>>p3a,p3c,
then p1b=p2b=p3b
In frames d,e,f,
p1e?p1d,p1f but p1e==p2e==p3e
<<>> means strongly not equivalent, = means weakly equivalent, == means strongly equivalent, ? means indeterminate
Maybe this could help somehow.

[StainlessS]

Well have a go Jmac, you can make both PrevDB and NextDB and do bi-directional searches (one at a time), function intended to be quite flexible.
In the no-exit-condition case, it returns a stream of candidate frames where you can do your own comparisons/matching, the
exit conditions are there to save a little time when it can be done automatically, I intend using mixtures of both.
In no-exit-condition mode, you might set up a for/next loop with eg 'j' as loop counter, give j as StartFrame and assign result back to j,
thereby skipping all intervening non-candidate frames.
Could make possible/plausible lots of differing tasks, edits detection, cut and spice where two scenes of separate clips overlap, + lots more
probs I'm sure.

I have not done the histogram thing yet (well I have a version that I hacked up for myself) but have not forgotten, it will come eventually.
I had to get this out of the way so I could finally try to resolve a problem here:- http://forum.doom9.org/showthread.php?t=170332

Its a pig of a prob, and thread that prompted the function.

gotta go, peace bro.

[Seedmanc]

@StainlessS:
It looks like your fps measuring script calculates the running average fps, so the "true" fps is only achieved at the end of the video, while I'm going for "current" fps, measured over a short timespan (1 second, for example).
I formatted my script a bit better to demonstrate what I mean:
timespan=1
pre=0.0
frames=0.0
fps=0.0
FrameEvaluate("""
post = RT_Timer()
tick = (post - pre) >= timespan
pre = tick?
\ post:
\ pre
fps = tick?
\ current_frame - frames:
\ fps
frames = tick?
\ current_frame:
\ frames
""")
scriptclip("rt_subtitle( string( fps / timespan))")

Unfortunately I can't make this script into a function that would return a numerical value of fps calculated over a timespan supplied as a parameter. I think it's because I can't get rid of having to declare those variables before frameevaluate, but without declaring them, I can't use conditional assignment later. Maybe there's any way to rewrite the assignments differently? Basically I only need it happen when the "tick" is true, which means the needed timespan of seconds has passed already. After that I calculate the amount of frames processed over that time and thus get the fps.

As for what I mean by dynamically disabling heavy filters, it's something like this:
last=(MeasureFPS(timespan)<30)?last:SlowFilter()
As I said, I'm working on realtime live processing of video so I need to take speed into consideration to avoid framedrops. Of course something like this wouldn't make any sense in the usual "offline" encoding of video.
Dynamical switching already works for me, it's just I can't make the fps measurement nicely into a function and it makes fps rather irregular in its current implementation.

[StainlessS]

How about this (I'm not sure that its correct)


GSCript("""
Function MyTimer(clip clp,Int HalfFrameRes) {
clp
CurrTime = RT_TimerHP
if(current_frame == 0 || !RT_VarExist("Prev") || Prev!=current_frame-1) {
Global StartFrame = current_frame Global StartFrame2 = current_frame
Global StartTime = CurrTime Global StartTime2 = CurrTime
Global Prev=current_frame
}
FrameCnt = current_frame-StartFrame
TimeExpired = Max(CurrTime-StartTime,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
if(ToggleOnce) {
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
} Else if(Toggle2) {
Global StartFrame = StartFrame2
Global StartTime = StartTime2
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
}
Global Prev=current_frame
Return FrameCnt/TimeExpired
}
""")

Avisource("Test.avi")

ScriptClip("""RT_Subtitle("%.3f",MyTimer(25))""")


And multi Instance version putting sub on frame

Function MyTimer(clip clp,Int "HalfFrameRes") {
myName="MyTimer: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
HalfFrameRes=Default(HalfFrameRes,25)
Assert(HalfFrameRes>0,myName+"HalfFrameRes Must be greater than 0")
FuncS="""
Function MyTimer@@@(clip clp,Int HalfFrameRes) {
clp
CurrTime = RT_TimerHP
if(current_frame==0 || !RT_VarExist("Prev@@@") || Prev@@@!=current_frame-1) {
# Start/Reset timer within script rather than in Multi-instance Global vars init
Global StartFrame@@@ = current_frame Global StartFrame2@@@ = current_frame
Global StartTime@@@ = CurrTime Global StartTime2@@@ = StartTime@@@
Global Prev@@@=current_frame
}
FrameCnt = current_frame-StartFrame@@@
TimeExpired = Max(CurrTime-StartTime@@@,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
# Comment out below Lines
RT_Subtitle("%.3f",FrameCnt/TimeExpired)
RT_Subtitle("%d] %.3f ToggleOnce=%s Toggle2=%s\nStartFrame =%d StartTime =%f\nStartFrame2=%d StartTime2=%f\nFrameCnt=%d TimeExpired=%f",
\ current_frame,FrameCnt/TimeExpired,ToggleOnce,Toggle2,
\ StartFrame@@@,StartTime@@@,StartFrame2@@@,StartTime2@@@,FrameCnt,TimeExpired,align=1)
if(ToggleOnce) {
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
} Else if(Toggle2) {
Global StartFrame@@@ = StartFrame2@@@
Global StartTime@@@ = StartTime2@@@
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
}
Global Prev@@@=current_frame
# EDIT: Uncomment below line if commenting out RT_Subtitles to give Global MeasuredFPS
# First Multi-Instance Variable name will be MeasuredFPS_MTIM_1
# Global MeasuredFPS@@@ = FrameCnt/TimeExpired
Return Last
}
ARGS = "HalfFrameRes"
clp.GScriptClip("MyTimer@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
RT_IncrGlobal("MTIM_InstanceNumber")
InstS = RT_StrReplace(FuncS,"@@@","_MTIM_"+String(MTIM_InstanceNumber))
# RT_TxtWriteFile(InstS,"DEBUG_MTIM_"+String(MTIM_InstanceNumber)+".TXT")
Return GScript(InstS)
}

Avisource("Test.avi")
MyTimer()



EDITED: RT_TimerHP requires RT_Stats v1.40, Can use RT_Timer instead.

[StainlessS]

Dont know if anyone has done a similar function, this seems to yield ApparentFPS {after dupe removal}.
(gives static scene as 0.0 FPS)
Perhaps saves counting duplicates (always a fun pastime).


GSCript("""
Function ApparentFPS(clip clp,Int "HalfFrameRes",Float "DupeThresh",Float "FrameRate",Bool "MaxFPS",Float "ChromaWeight") {
myName="ApparentFPS: "
clp
HalfFrameRes=Default(HalfFrameRes,25)
DupeThresh=Default(DupeThresh,1.0)
FR=Float(Default(FrameRate,clp.FrameRate))
MaxFPS=Default(MaxFPS,False)
ChromaWeight = Float(Default(ChromaWeight,1.0/3.0))
Assert(HalfFrameRes>0,myName+"HalfFrameRes Must be greater than 0")
Assert(DupeThresh>0.0,myName+"DupeThresh Must be greater than 0.0")
Assert(ChromaWeight>=0.0 && ChromaWeight<=1.0,myName+"ChromaWeight 0.0 -> 1.0")
CurrTime = current_frame / FR
if(current_frame==0 || !RT_VarExist("Prev") || Prev!=current_frame-1) {
Global StartFrame = current_frame Global StartFrame2 = current_frame
Global StartTime = CurrTime Global StartTime2 = CurrTime
Global DupeCnt = 0 Global DupeCnt2 = 0
Global MaxApparentFPS = (RT_VarExist("MaxApparentFPS")) ? MaxApparentFPS : 0.0
Global Prev = current_frame
}
Diff = RT_FrameDifference(clp,clp,n2=current_frame-1,ChromaWeight=ChromaWeight)
if(Diff < DupeThresh && current_frame > 0) {
Global DupeCnt = DupeCnt + 1
Global DupeCnt2 = DupeCnt2 + 1
}
FrameCnt = current_frame-StartFrame
UniqueFrameCnt = FrameCnt - DupeCnt
TimeExpired = Max(CurrTime-StartTime,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
ApparentFPS = UniqueFrameCnt/TimeExpired
Global MaxApparentFPS = (ApparentFPS>MaxApparentFPS) ? ApparentFPS : MaxApparentFPS

if(ToggleOnce) {
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
Global DupeCnt2 = 0
} Else if(Toggle2) {
Global StartFrame = StartFrame2
Global StartTime = StartTime2
Global DupeCnt = DupeCnt2
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
Global DupeCnt2 = 0
}
Global Prev = current_frame
Return (MaxFPS) ? MaxApparentFPS : ApparentFPS
}
""")

#Avisource("Test.avi")
Avisource("CAP1.avi")
MAXFPS=False
ScriptClip("""RT_Subtitle("%.3f",ApparentFPS(maxfps=MaxFPS))""")


Requires RT_Stats v1.40

EDIT: And Multi-Instance version


Function ApparentFPS(clip clp,Int "HalfFrameRes",Float "DupeThresh",Float "FrameRate",Float "ChromaWeight") {
myName="ApparentFPS: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
HalfFrameRes=Default(HalfFrameRes,25)
DupeThresh=Default(DupeThresh,1.0)
FR=Float(Default(FrameRate,clp.FrameRate))
ChromaWeight=Float(Default(ChromaWeight,1.0/3.0))
Assert(HalfFrameRes>0,myName+"HalfFrameRes Must be greater than 0")
Assert(DupeThresh>0.0,myName+"DupeThresh Must be greater than 0.0")
Assert(ChromaWeight>=0.0 && ChromaWeight<=1.0,myName+"ChromaWeight 0.0 -> 1.0")
FuncS="""
Function ApparentFPS@@@(clip clp,Int HalfFrameRes,DupeThresh,Float FR,Float ChromaWeight) {
clp
CurrTime = current_frame / FR
if(current_frame==0 || !RT_VarExist("Prev@@@") || Prev@@@!=current_frame-1) {
# Start/Reset timer within script rather than in Multi-instance Global vars init
Global StartFrame@@@ = current_frame Global StartFrame2@@@ = current_frame
Global StartTime@@@ = CurrTime Global StartTime2@@@ = StartTime@@@
Global Dropped@@@ = 0
Global DupeCnt@@@ = 0 Global DupeCnt2@@@ = 0
Global MaxApparentFPS@@@ = (RT_VarExist("MaxApparentFPS@@@")) ? MaxApparentFPS@@@ : 0.0
Global Prev@@@ = current_frame
}
Diff = RT_FrameDifference(clp,clp,n2=current_frame-1,ChromaWeight=ChromaWeight)
if(Diff < DupeThresh && current_frame > 0) {
Global Dropped@@@ = Dropped@@@ + 1
Global DupeCnt@@@ = DupeCnt@@@ + 1
Global DupeCnt2@@@ = DupeCnt2@@@ + 1
}
FrameCnt = current_frame-StartFrame@@@
UniqueFrameCnt = FrameCnt - DupeCnt@@@
TimeExpired = Max(CurrTime-StartTime@@@,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
# EDIT: Below lines give Global return values
# First Multi-Instance ApparentFPS@@@ Variable name will be ApparentFPS_AFR_1
# First Multi-Instance MaxApparentFPS@@@ Variable name will be MaxApparentFPS_AFR_1
Global ApparentFPS@@@ = UniqueFrameCnt/TimeExpired
Global MaxApparentFPS@@@ = (ApparentFPS@@@>MaxApparentFPS@@@) ? ApparentFPS@@@ : MaxApparentFPS@@@
# Comment out below Lines
RT_Subtitle("%.3f (Max=%.3f)",UniqueFrameCnt/TimeExpired,MaxApparentFPS@@@)
RT_Subtitle("%d] FrameRate=%.3f ApparentFrameRate=%.3f (Max=%.3f)" +
\ "\nToggleOnce=%s Toggle2=%s\nStartFrame =%d StartTime =%f\nStartFrame2=%d StartTime2=%f" +
\ "\nFrameCnt=%d TimeExpired=%f\nDropped=%d DupeCnt=%d DupeCnt2=%d UniqueFrameCnt=%d",
\ current_frame,FrameCnt/TimeExpired,UniqueFrameCnt/TimeExpired,MaxApparentFPS@@@,ToggleOnce,Toggle2,
\ StartFrame@@@,StartTime@@@,StartFrame2@@@,StartTime2@@@,FrameCnt,TimeExpired,
\ Dropped@@@,DupeCnt@@@,DupeCnt2@@@,UniqueFrameCnt,align=1)
if(ToggleOnce) {
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
Global DupeCnt2@@@ = 0
} Else if(Toggle2) {
Global StartFrame@@@ = StartFrame2@@@
Global StartTime@@@ = StartTime2@@@
Global DupeCnt@@@ = DupeCnt2@@@
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
Global DupeCnt2@@@ = 0
}
Global Prev@@@ = current_frame
Return Last
}

ARGS = "HalfFrameRes,Dupethresh,FR,ChromaWeight"
clp.GScriptClip("ApparentFPS@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
RT_IncrGlobal("AFR_InstanceNumber")
InstS = RT_StrReplace(FuncS,"@@@","_AFR_"+String(AFR_InstanceNumber))
RT_TxtWriteFile(InstS,"DEBUG_AFR_"+String(AFR_InstanceNumber)+".TXT")
Return GScript(InstS)
}

#Avisource("TEST.avi")
Avisource("CAP1.avi")
ApparentFPS()


EDITED:

[StainlessS]

Ooops, posted RT_Stats v1.40 in wrong folder on MediaFire (was not available, was available via SendSpace), Fixed.

[StainlessS]

New Version RT_Stats v1.41, See 1st post.


v1.41, 15 Sept 2014, Fixed borked v1.40, (released wrong source + binary), RT_QwikScan PC, PD related stuff.
Changed RT_QwikScan LumaTol arg to float.



RT_QwikScan(clip SearchClip,Int SeachStart,clip FindClip,Int FindFrame,string DB,string PNDB,
\ Float "LumaTol"=1.0,int "Flags"=1,
\ Float "LC"=1.0,Float "LD"=1.0,Float "FD"=1.0,Float "PD"=1.0,int "PC"=(SearchClip.Width*SearchClip.Height+127)/255, int "XP"=1,
\ Float "ChromaWeight"=1.0/3.0,Int "PDThresh"=0
\ int "Maxdistance"=SearchClip.FrameCount-1,bool "Inclusive"=True,string "Prefix"="QWKS_")



FINGERPRINT & LUMATOL:
The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
Each FingerPint component is derived from a pixel count for a portion of the possible 0 -> 255 luma pixel value range.
Lets say that one of the fingerprint components is derived from the 128->191 luma pixel value range, the fingerpint component
is computed for every frame in the SearchClip eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height},
this result is in range 0.0 -> 255.0, with 255.0 being equal to 100%. [There would be another three fingerprint components computed,
in different ranges. We do not actually use range 128->191, and are reluctant to give any real used ranges as they may change, if you
want to see the real current ranges used, then see the current source code]. The 4 fingerprint components are stored in the
FingerPrint DB and are EXACT fingerprint component values. For the FindFrame, we use a FingerPrint component Range with LumaTol
tolerance. For FindClip(FindFrame), we might compute a fingerprint component range for [128->191]:
FingerPrintMin = {luma PixelCount in range [128+LumaTol] to [191-LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
FingerPrintMax = {luma PixelCount in range [128-LumaTol] to [191+LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
So using computed FingerPrintMin and FingerPrintMax for four different ranges as upper and lower limits, if any frame in the
SeachClip (DBase) has all four EXACT FingerPrint components within all four fingerprint component limits, then it is a candidate
frame and eligible for exit condition testing if exit conditions were set, or returned to the client as a result CANDIDATE frame if
exit conditions were not set.

v1.41, Have changed LumaTol arg from type Int to type Float, default 1.0. (Will still accept type Int so will not break script).
Whole number values of LumaTol will behave as previously described.
Where a non integer value of Lumatol is given, then LumaTol is split into integer and fractional parts, the integer part is
processed normally where the FingerPrint value is derived from a pixel count for the particular range. After being converted
into a float via eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height}, then the fractional
part of LumaTol is added to the result and limited to range 0.0->255.0. So, the fractional part is a way of giving an additional
margin of error to LumaTol in a frame size relative way.


EDIT:
Change to FingerPrint source

void __cdecl RT_Fingerprint_Lo(const AVSValue &std,const AVSValue &xtra,FINGER &finger,unsigned int *hist,IScriptEnvironment* env,double LTol) {
// NOTE, LTol optional default 0.0
char *myName="RT_Fingerprint_Lo: ";
LTol = max(min(LTol,16.0),0.0);
MYLO mylo;
RT_MYstats_Lo(RTHIST_F,std,xtra,mylo,myName,env,hist);
const int rng[4][2]={{0 ,85},{86,170},{171,255},{64,191}};
const int Pixels = mylo.pixels;
double frac,whole;
frac=modf(LTol,&whole);
int Tol = int(whole);
int i,j;
if(frac == 0.0) {
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0];
const int e = rng[i][1];
int s2 = s+Tol;
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.lolim[i] = sm * 255.0 / Pixels;
if(Tol==0) {
finger.hilim[i]=finger.lolim[i];
} else {
sm=0;
s2 = max(s-Tol,0);
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.hilim[i] = sm * 255.0 / Pixels;
}
}
} else {
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0];
const int e = rng[i][1];
int s2 = s+Tol;
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.lolim[i] = max( (sm * 255.0 / Pixels - frac) ,0.0);
sm=0;
s2 = max(s-Tol,0);
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.hilim[i] = min( (sm * 255.0 / Pixels + frac) ,255.0);
}
}
}

[StainlessS]

@ Forensic,

Are lurking about out there, not online since April.

You only gave two lines of documentation for AvgLumaDif and I think that you never said finally whether the function was as required,
so I've been waiting to add to RT_Stats, anyway, this is what I've knocked up as docs, is it to your satisfaction, will integrate in next version (v1.42).



RT_AvgLumaDif(clip clp, int "n"=current_frame,Bool "Slices"=False,Int "Matrix"=(Width>1100||Height>600?3:2) )

The function, returns the average of pixel pair differentials for a given frame n, ignoring frame perimeter (1 pixel
perimeter around frame edge), range 0.0 -> 255.0.

Args:
n, Int, Default current_frame.
Slices, Bool Default False.
False, Analyze every adjacent pixel pair in the frame with the exception of frame perimeter.
True, Analyze adjacent pixel pairs in the frame, all 4x4 pixel block boundaries are treated as perimeter pixels
and omitted from processing along with the frame perimeter pixels.
Matrix, Int, Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709,

The function finds the average luminosity differential by analyzing every adjacent pixel pair within the video frame,
(both horizontal and vertical, not diagonal) with an option to only process every other 2x2 horizontal and vertical block
to ensure any slice (e.g. h.264, h.265) boundaries are excluded. By calling this function before and again after a given
filter, this function quantifies a filter's affect on luminosity contrast.
NOTE: The perimeter pixels are intentionally ignored as they often incur higher quantization errors.

'x'= Process, '.'=Don't Process

Slices=false Slices=true
........................ ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
........................ ........................

[StainlessS]

Warning of change to RT_Stats v1.42 RT_QwikScan, XP arg (time to give views if any).

Currently implemented:

XP, (Default 1), Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=Continue searching while next frame is a better Exit Condition match ('next frame' meaning dependant upon search direction).
2=Continue searching while next frame is a better or equal Exit Condition match ('next frame' meaning dependant upon search direction).
When an exit condition occurs eg LumaDifference, then QWKS_BM_LD_XP will be set to the count of additional frames traversed due
to finding a match that is better or equal (depending upon whether XP==1 or XP==2) to the initial frame that caused the exit condition,
In such cases, both QWKS_BM_LD and QWKS_BM_LD_FRM will also be set to (in this case) LumaDifference and the final frame (with XP offset)
that was found to be a better or equal match.


Intend to change XP = 1 mode. ('Better Than' mode)

Instead of terminating scan when next frame is not better (even if equal) will do instead:


0 ) Will search for continually better matching frames, until
Worse match or MaxDistance reached,
then returns best so far.

OR

Equal Match, will drop anchor (remember where it got to) and continue scanning until
1) Worse match or MaxDistance reached,
it will return the remembered frame where it dropped anchor.
2) Better match,
then exits the equal match scanning and updates best so far scan position and continues as for step 0).

The last frame returned will be better than all previous matches, but some adjacent previous matches may have been equal to each other.

I could add another XP Mode 3=Strictly Better Match, that did not scan equal match paths if anyone thought it had functionality.

EDIT: I have just encountered a float match condition that did indeed provide better result with above amendment (Hence the change).

EDIT: Will add XP mode 3 = Strictly Better Match (have almost done it).

EDIT: You can treat RT_QwikScan as permanently EXPERIMENTAL, LumaTol Float fractional part stuff may change, as may the
way the fingerprint is computed. The function works just fine at it is but it would be nice if LumaTol could be made purely ornamental.
Currently, the fractional LumaTol thing is really intended for use where integer part is 0.

EDIT: Have now changed slightly the fractional part functionality of LumaTol a bit, works much better and more
targeted at specific frames, speeds it up a bit.

[StainlessS]

RT_Stats v1.42, new version see 1st post.


v1.40, 9 Sept 2014, Fixed, RT_DBaseGetAttrib returned float instead of Int (for int attrib).
Added args Start and End to RT_QueryBorderCrop and RT_QueryLumaMinMax. Added RT_WriteFile + other funcs.
Now 1024 Array & DBase Attributes. Array and DBase now supports type double as private type for RT_Stats
internal routines, set or get as type Float from Avisynth script.
DBase max number of fields increased to 1024. Added DBase/Array String Attributes.
Added RT_QwikScan + other stuff.
v1.41, 15 Sept 2014, Fixed borked v1.40, (released wrong source + binary), RT_QwikScan PC, PD related stuff.
Changed RT_QwikScan LumaTol arg to float.
v1.42, 19 Sept 2014. Modified Float LumaTol imlementation. Added additional XP mode to RT_QwikScan, and affects
BEST MATCH results. Added RT_AvgLumaDif.


RT_QwikScan Doc mods.

XP, (Default 1), Sets eXtra Paranoia for EXIT Conditions.
0 = Off
1 = LOOSELY BETTER THAN (v1.42 changed STRICTLY BETTER THAN to mode 3, and using this one instead).
Continue searching while next frame is a loosely better Exit Condition match ('next frame' meaning dependant upon search direction).
Differs from STRICTLY BETTER THAN in that it scans along equal matches and if a better match found along that equal matches
path, then will return that match, if no better match occurs along an equal match path, then returns the previous frame number
that was better than previous matches. So, the last frame returned will be better than all previous matches, but some adjacent
previous matches may have been equal to each other.
2 = BETTER THAN OR EQUAL
Continue searching while next frame is a better or equal Exit Condition match ('next frame' meaning dependant upon search direction).
3 = STRICTLY BETTER THAN
Continue searching while next frame is a better Exit Condition match ('next frame' meaning dependant upon search direction).
Does not scan along equal match paths.

When an exit condition occurs eg LumaDifference, then QWKS_BM_LD_XP will be set to the count of additional frames traversed due
to finding a match that is better or equal (depending upon whether XP==1 or XP==2, or XP==3) to the initial frame that caused the exit
condition. In such cases, both QWKS_BM_LD and QWKS_BM_LD_FRM will also be set to (in this case) LumaDifference and the final frame
(with XP offset) that was found to be a better or equal match.
v1.42, BEST MATCHES also now comply with XP = 2 (BETTER OR EQUAL) setting, Where XP != 2, then returns the first encountered
frame position where matches are equal, XP == 2 will returns the last frame position encountered where matches are equal.



FINGERPRINT & LUMATOL:
The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
Each FingerPint component is derived from a pixel count for a portion of the possible 0 -> 255 luma pixel value range.
Lets say that one of the fingerprint components is derived from the 128->191 luma pixel value range, the fingerprint component
is computed for every frame in the SearchClip eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height},
this result is in range 0.0 -> 255.0, with 255.0 being equal to 100%. [There would be another three fingerprint components computed,
in different ranges. We do not actually use range 128->191, and are reluctant to give any real used ranges as they may change, if you
want to see the real current ranges used, then see the current source code]. The 4 fingerprint components are stored in the
FingerPrint DB and are EXACT fingerprint component values. For the FindFrame, we use a FingerPrint component Range with LumaTol
tolerance. For FindClip(FindFrame), we might compute a fingerprint component range for [128->191]:

FingerPrintMin = {luma PixelCount in range [128+LumaTol] to [191-LumaTol]} * 255.0 / {SearchClip.Width*SearchClip.Height}
FingerPrintMax = {luma PixelCount in range [128-LumaTol] to [191+LumaTol]} * 255.0 / {SearchClip.Width*SearchClip.Height}

So using computed FingerPrintMin and FingerPrintMax for four different ranges as upper and lower limits, if any frame in the
SeachClip (DBase) has all four EXACT FingerPrint components within all four fingerprint component limits, then it is a candidate
frame and eligible for exit condition testing if exit conditions were set, or returned to the client as a result CANDIDATE frame if
exit conditions were not set.

v1.42, Have changed LumaTol arg from type Int to type Float, default 1.0. (Will still accept type Int so will not break script).
Whole number values of LumaTol will behave as previously described.
Where a non integer value of Lumatol is given, then LumaTol is split into integer and fractional parts, and processed as described
below.

PixelCountMin_INT = PixCount[128+LumaTol.Int] to PixCount[191-LumaTol.Int] # Sum of
PixelCountMin_FRAC = Int((PixCount[128+LumaTol.Int] + PixCount[191-LumaTol.Int]) * LumaTol.Frac) # Fractional count of extremities
FingerPrintMin = (PixCountMin_INT - PixCountMin_FRAC) * 255.0 / {SearchClip.Width*SearchClip.Height}

PixelCountMax_INT = PixelCount[128-LumaTol.Int] to PixelCount[191+LumaTol.Int] # Sum of
PixelCountMax_FRAC = Int((PixelCount[128-LumaTol.Int-1] + PixelCount[191+LumaTol.Int+1]) * LumaTol.Frac) # Frac count expanded
FingerPrintMax = (PixelCountMin_INT + PixelCountMin_FRAC) * 255.0 / {SearchClip.Width*SearchClip.Height}

Fractional LumaTol as implemented in v1.42 is a big improvement on integer only Lumatol, and intended primarily for extending
LumaTol=0 to have a little extra tolerance. As the fractional part of LumaTol is applied to the actual existing pixel count of a frame,
so it is to some degree self tuning, if eg a count for PixelCount[128-LumaTol.Int-1] is zero, then there will be no additional speed
overhead for that particular FindFrame FingerPrint component match, if the count is high, then more candidate frames will be searched,
this is contrary to v1.41 Float LumaTol where we just added the fractional part where it would incur a speed penalty on all frames.



With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a candidate frame
to the client. If Exit condition are set, then at least one of those exit conditions also has to be satisfied, if not satisfied, then
the candidate frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and an Exit
condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, if any (or all) of the exit conditions succeed then there will be a +ve frame number
returned by the function, QWKS_EXIT_FLAGS and QWKS_BM_FLAGS bits will be set, the returned frame number will be the first frame
that satisfied an exit condition, and does NOT include any XP arg processing.
If any exit condition is satisfied, and the XP arg is set, the function will continue scanning adjacent frames to the frame
where an exit condition was satisfied and step onward until a better match for the exit condition is NOT found.
(XP=1 is LOOSELY BETTER THAN, and XP=2 is BETTER OR EQUAL, XP=2 is STRICTLY BETTER THAN, see XP). XP Processing occurs on satisfied
exit conditions and the XP processed frame numbers and differences are returned via QWKS_BM_xx and QWKS_BM_xx_FRM local vars.
When using exit conditions, use the return frame number only as an indicator of success, use the returned local vars irrespective of
whether or not XP processing is done (although if XP==0, then returned frame will be same as QWKS_BM_xx_FRM).
v1.42, BEST MATCHES also now comply with XP = 2 (BETTER OR EQUAL) setting, Where XP != 2, then returns the first encountered
frame position where matches are equal, XP == 2 will returns the last frame position encountered where matches are equal.


New Function:


RT_AvgLumaDif(clip clp, int "n"=current_frame,Bool "Slices"=False,Int "Matrix"=(Width>1100||Height>600?3:2) )

The function, returns the average of pixel pair differentials for a given frame n, ignoring frame perimeter (1 pixel
perimeter around frame edge), range 0.0 -> 255.0.

Args:
n, Int, Default current_frame.
Slices, Bool Default False.
False, Analyze every adjacent pixel pair in the frame with the exception of frame perimeter.
True, Analyze adjacent pixel pairs in the frame, all 4x4 pixel block boundaries are treated as perimeter pixels
and omitted from processing along with the frame perimeter pixels.
Matrix, Int, Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709,

The function finds the average luminosity differential by analyzing every adjacent pixel pair within the video frame,
(both horizontal and vertical, not diagonal) with an option to only process every other 2x2 horizontal and vertical block
to ensure any slice (e.g. h.264, h.265) boundaries are excluded. By calling this function before and again after a given
filter, this function quantifies a filter's affect on luminosity contrast.
NOTE: The perimeter pixels are intentionally ignored as they often incur higher quantization errors.

'x'= Process, '.'=Don't Process

Slices=false Slices=true
........................ ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
........................ ........................


EDIT: Modified Float LumaTol now implemented as so:

void __cdecl RT_Fingerprint_Lo(const AVSValue &std,const AVSValue &xtra,FINGER &finger,unsigned int *hist,IScriptEnvironment* env,double LTol) {
// NOTE, LTol optional default 0.0
char *myName="RT_Fingerprint_Lo: ";
LTol = max(min(LTol,16.0),0.0);
MYLO mylo;
RT_MYstats_Lo(RTHIST_F,std,xtra,mylo,myName,env,hist); // Get luma histogram
const int rng[4][2]={{0 ,85},{86,170},{171,255},{64,191}}; // FingerPrint Component Histogram Ranges
const int Pixels = mylo.pixels;
double frac,whole;
frac=modf(LTol,&whole);
int Tol = int(whole);
int i,j;
if(frac == 0.0) { // Integer LumaTol, no fractional part
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0]; // Start of range
const int e = rng[i][1]; // End of range
int s2 = s+Tol; // contract lolim range by LumaTol Int
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
const double smd = sm * 255.0 / Pixels; // cannot be bigger than 255.0
finger.lolim[i] = smd;
if(Tol==0) {
finger.hilim[i]=smd; // EXACT FINGERPRINT components, search identical frames OR DB FINGER
} else {
sm=0;
s2 = max(s-Tol,0); // expand hilim range by LumaTol Int
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.hilim[i] = sm * 255.0 / Pixels; // cannot be bigger than 255.0
}
}
} else {
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0];
const int e = rng[i][1];
int s2 = s+Tol; // contract lolim range by LumaTol Int
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
sm -= int((hist[s2] + hist[e2]) * frac + 0.5); // Fractional reduction of extremities
finger.lolim[i] = sm * 255.0 / Pixels;
sm=0;
s2 = max(s-Tol,0); // expand hilim range by LumaTol Int
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
if(s2 > 0) sm += int(hist[s2-1] * frac + 0.5); // Add frac parts if possible
if(e2 < 255) sm += int(hist[e2+1] * frac + 0.5);
finger.hilim[i] = sm * 255.0 / Pixels;
}
}
}

[StainlessS]

Ah, hem. I dont know what to say here other than I have a recommendation, if you dont want it, dont get it, I am in no way affiliated
with said device other than that I bought one.

Not the best place to post this, but I just got one and am so goddam impressed (my PC is core Duo 2.4, so I'm easlily impressed),
anyway, it does have my sincere recommendation from a company that I've been using (well a few years ago{EDIT: couple} bought out by another internet company)
since about 1984 (originally on Tottenham Court Rd and New Oxford Street), always a good experience. Brilliant tablet PC, if you dont like it, then tell
somebody else, (well you can call me names if you like but I'm used to that). Dont know why I'm posting here other that I think it is magic, If moderator
wants to spank me, then at least put in some other forum, reccomendation is with good intent. Well good buy.
http://www.morgancomputers.co.uk/product_detail/14895/Advent-Vega-Tegra-Note-7-Quad-Core-1-8GHz-1GB-16GB-Android-4-3-Tablet/#

PS, Comes with Kit-Kat. (As Avisynth now seems all but dead, me guesses that this will soon become my next fav devolopment thingy).

@IanB, if you are reading this, would be nice if you could say one way or another if you have deserted the project,
I'm sure that you would understand that some would be interested in a such statement, either way.

[StainlessS]

RT_Stats v1.43, new version see 1st post.

New DBase and Array funcs:


RT_DBaseGetID(String Filename,Int ix)
There are 16 ID attributes in the DBase file header that may be used internally by DBase utility functions rather than using user
attributes. This function allows script functions to use the same ID's (int or float) for whatever purpose a scriptor requires.
Built in utilty functions will use these to eg match a DBase to some clip, store Width or whatever.
Filename, DBase Filename, no default.
Ix, ID array element index range 0 -> 15.
Returns the value of the set ID, Int or Float (or type Int 0 if not yet set).


RT_DBaseSetID(String Filename,Int ix, Value)
There are 16 ID attributes in the DBase file header, that may be used internally by DBase utility functions rather than using user
attributes. This function allows script functions to use the same ID's (int or float) for whatever purpose a scriptor requires.
RT_DBaseAlloc() initializes all ID's to type Int, 0.
Filename, DBase Filename, no default.
Ix, ID array element index range 0 -> 15.
Value, Int or Float, The value that ID(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.


RT_ArrayGetID(String Filename,Int ix)
There are 16 ID attributes in the Array file header that may be used internally by array utility functions rather than using user
attributes. This function allows script functions to use the same ID's (int or float) for whatever purpose a scriptor requires.
Built in utilty functions will use these to eg match an array to some clip, store Width or whatever.
Filename, ARRAY Filename, no default.
Ix, ID array element index range 0 -> 15.
Returns the value of the set ID, Int or Float (or type Int 0 if not yet set).


RT_ArraySetID(String Filename,Int ix, Value)
There are 16 ID attributes in the Array file header, that may be used internally by array utility functions rather than using user
attributes. This function allows script functions to use the same ID's (int or float) for whatever purpose a scriptor requires.
RT_ArrayAlloc() initializes all ID's to type Int, 0.
Filename, Array Filename, no default.
Ix, ID array element index range 0 -> 15.
Value, Int or Float, The value that ID(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.


New RT_QwikScan related func:

RT_QwikScanEstimateLumaTol(clip SearchClip,clip FindClip,int "n"=current_frame,int "n2"=current_Frame,int "Matrix"=(Width>1100||Height>600?3:2))
Returns an estimate of minimum required LumaTol for use by RT_QwikScan().
SearchClip, clip using DBase in RT_QuickScan, ie with EXACT fingerprints in DBase.
FindClip, clip to estimate LumaTol for.
n = SearchClip frame number, default current_frame in runtime environment.
n2= FindClip frame number, default current_frame in runtime environment.
Matrix, RGB Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used. Conversion of RGB to YUV.
The clip/frame order IS SIGNIFICANT, make sure you have them the correct way around, FindClip(n2) is the frame for which
LumaTol is estimated, SearchClip(n) is the frame be be matched to.
Implemented for script that had many almost identical frames where purpose was to find UNDO's in a paint program capture clip
and remove the UNDO's from the clip. Was used to get a LumaTol estimate on frame prior to suspected UNDO frame where LumaTol
would be sufficiently great to also detect the start of an edit sequence as frame sort after, would be closer to the UNDO frame
than the frame for which the estimate was made. In this case, SearchClip(n) would be the suspect UNDO frame and FindClip(n2) would
be the frame we get an estimate for where goal is to find a frame that matches SearchClip(n).
The function returns 16.0 (maximum LumaTol) if no valid value of LumaTol is possible.
Thread that this function was implemented for (probably post after linked one):
http://forum.doom9.org/showthread.php?p=1694208#post1694208

[wiseant]

Hi StainlessS

First of all, Re post #269 in this thread:
I have been away from this forum since December 2013
I am pleased to see that you are still working on RT_Stats
Also happy to see that you are using SendSpace as well as MediaFire - I probably did access MediaFire previously with IE - hence the pop-up problem - I normally use FireFox with AdBlock Plus

I just downloaded your latest RT_Stats and read through RT_Stats.txt - but I can't figure out how to accomplish what I want to do

Re this script for resizing a folder of images:

RT_Debug("Processing BMP File",FN)
k = ImageSource(FN,end=end).ConvertToRGB32()

iHeight = k.height
iWidth = k.width


if (iHeight >= iWidth) {
iHeight1 = canvasheight
iWidth1 = (canvasheight * iWidth) / iHeight
x = canvaswidth - iWidth1
x1 = x/2
#
if (iwidth1 % 2 > 0) {
iwidth1 =iwidth1+1
}
k = k.bilinearresize(iWidth1,canvasheight).addborders(x1,0,x1,0,$101010)

}
#
if (iWidth > iHeight) {
iWidth1 = canvaswidth
iHeight1 = (canvaswidth*iHeight)/iWidth
y = canvasheight-iHeight1
#
if (iheight1 % 2 > 0) {
iheight1 = iheight1+1
}
y1 = y/2
k = k.bilinearresize(iWidth1,iHeight1).addborders(0,y1,0,y1,$101010)
}
#
if (iwidth1 > canvaswidth) {
iwidth1 = canvaswidth
iheight1 = (canvaswidth*iheight)/iwidth
y = canvasheight-iheight1
#
if (iheight1 % 2 > 0) {
iheight1 =iheight1+1
}
y1 = y/2
k = k.bilinearresize(iWidth1,iHeight1).addborders(0,y1,0,y1,$101010)
}
#
if (iHeight1 > canvasheight) {
iHeight1 = canvasheight
iWidth1 = (canvasheight*iWidth)/iHeight
x = canvaswidth-iWidth1
#
if (iwidth1 % 2 > 0) {
iwidth1 = iwidth1+1
}
x1 = x/2
k = k.bilinearresize(iWidth1,iHeight1).addborders(x1,0,x1,0,$101010)

}
RT_Debug("NewWidth= ",String(k.Width),"NewHeight =",String(k.height))
picclip = (IsClip(picclip)) ? picclip ++ k : k
RT_Debug("")
}
""")
PicClip
assumefps(fps)


What I want to do is to have a subtitle for each image. I could have a text file that has each Caption (subtitle) on a line by line basis - corresponding to each image, i.e. if I have 500 images, I would have a text file that has 500 lines of text.

So after each image get processed [resized] - I want to add the corresponding caption [subtitle]

Any suggestions to get me started?

TIA
wiseant

[StainlessS]

Hi again Wiseant, nice to see you back.

Rather than reinventing the wheel again, how about just using the ImageSplicer.avs script in RT_Stats AVS folder, perhaps it will serve.

As a test, I had a group of 100 BMP files named "CropTest_001.BMP" -> "CropTest_100.BMP", and executed this script to create a
Captions.Txt file.


GScript("""
S=""
for(i=1,100) {
S=RT_TxtAddStr(S,RT_String("%3d ) My Name is CropTest_%03d.BMP",i,i))
}
RT_Writefile("Captions.Txt","%s",S)
""")
Return MessageClip("All Done")


Then used this client script to call ImageSplicer()


Import("ImageSplicer.avs")

FN=".\CROPTEST\*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF"
SzLimit= -768
FAR=1.333
AutoCrop=True
AutoLevelStrength=1.0
ShowBorder=True
ShowAutoLevel=True
Subs=True
CropThresh = -1.0
ScanPerc=49.0
Baffle=4
Debug=False
Debug_QBC=False
Resizer="Spline64resize(_W_,_H_)"
Matrix=2

ImageSplicer(FN,SzLimit,FAR=FAR,AutoCrop=AutoCrop,AutoLevelStrength=AutoLevelStrength,ShowBorder=ShowBorder,ShowAutoLevel=ShowAutoLevel, \
Subs=Subs, CropThresh=CropThresh,Baffle=Baffle,ScanPerc=ScanPerc,Debug=Debug,Debug_QBC=Debug_QBC,Resizer=Resizer,matrix=matrix)

CaptionsFN = "Captions.Txt"
Captions = RT_ReadTxtFromFile(CaptionsFN)

ScriptClip("""
S=RT_TxtGetLine(Captions,current_frame)
Subtitle(S,Align=5,Size=24)
Return Last
""")

Return Last


You need to edit the config settings of course.

EDIT: Just for those curious about the config stuff:

Function ImageSplicer(String "FileName",int "SzLimit",Float "FAR",Bool "AutoCrop",Float "AutoLevelStrength",bool "ShowAutoLevel",bool "ShowBorder", \
bool "Subs", Float "CropThresh",Float "FPS",int "WMod",int "HMod",int "Baffle",Float "ScanPerc", \
bool "Debug", bool "Debug_QBC",String "Resizer",int "Matrix",float "Ignore") {
##########################################
VERS="ImageSplicer v1.10 - 17 Mar 2013" # Requires GScript, RT_Stats v1.10
#
# ImageSplicer, joins together images of different sizes into a sort of slideshow, auto cropping borders, auto levelling, resizing or original size,
# border padding to chosen aspect ratio.
# Gets max width/height of all images (after optional auto cropping) and creates a background 'canvas' based on SzLimit & FAR.
# SzLimit==0: Images kept at original dimensions (post cropping), canvas made to encompass all cropped images.
# SzLimit>0: The shorter canvas dimension (FAR) limited to AT MOST SzLimit, where images will be resized to fit canvas.
# Avoids upsizing to a larger size than biggest cropped source image, if biggest less than SzLimit then output is biggest.
# SzLimit<0: The shorter canvas dimension (FAR) set at abs(SzLimit) and images resized to fit canvas.
#
# Args:-
# Filename: Default "*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF". Image Path and wildcard with mulitple pipe separated extensions.
# SzLimit: Default=0, 0=No resize. Greater than 0 is limit on shorter dimension (see FAR), -ve resize to abs(SzLimit) shorter dimension.
# FAR: Default 4.0/3.0. Output Frame Aspect Ratio, if FAR>=1.0 then SzLimit affects height, else affects width.
# AutoCrop: Default=True. Crops borders off images.
# AutoLevelStrength: Default 0.0, Auto luma leveling, range 0.0 -> 1.0, 0.0 is OFF.
# ShowAutoLevel: Default False. If true then shows Auto leveling on right hande side of image ONLY.
# ShowBorder: Default False. When padding to maintain Frame Aspect Ratio, borders are shown in red.
# Subs: Default False. Shows Subtitled info on frames.
# CropThresh: Default -0.5
# CropThresh <=0.0 is PC levels AUTO thresh where cropthresh is set to minimum luma pixel + abs(CropThresh).
# Default is very sensitive threshold to avoid possibility of overcropping (-0.5 -> -1.0, usually safe, more -ve less safe).
# You may raise (make more -ve) auto CropThresh if not cropping enough but risk overcropping on dark images if you do.
# CropThresh >0.0 is explicit threshold @ PC Levels, probably best avoided.
# RGB clips are converted to YUV-Y using Matrix arg @ PCLevels and CropThresh acting on Average Luma of scanline (h/v).
# FPS: Default 1.0. Frames Per Second of output clip.
# WMod: Default 2. Output clip width is a multiple of WMod.
# HMod: Default 2. Output clip height is a multiple of HMod.
# Baffle: Default 4 (1->32), affects cropping.
# Number of scanlines that must break a CropThresh to be accepted as Image. Avoids noise. Not usually altered.
# Baffle also affects speed, higher can be faster for big borders but also means that more scanlines must break threshold to be accepted as image.
# ScanPerc: Default 49.0, cropping. Percentage of image to scan per side. Not usually altered.
# Debug: Default False. Show debug info via DebugView. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
# Debug_QBC: Default False. Show debug messages from RT_QueryBorderCrop.
# Resizer: Default "Spline36Resize(_W_,_H_)", the resizer to use. "_W_" and "_H_" are replaced by the resize width & height.
# Matrix: 2->3, Default If pic width <=720 Then 2(PC601) Else 3(PC709). Used in converting RGB to YUV-Y.
# Ignore: Default 0.2, Cropping. Percentage, Threshold of pixels to ignore when getting Luma minimum, avoids a few noisy pixels.
# As for YPlaneMin Threshold arg.

[wiseant]

Hi StainlessS

It is good to be back - there is a lot of expertise in this forum
Thanks for the info - I will try your scripts later today.

I am able to add captions this way:


GScript("""
c0="Hello World"
c1="Hello and GoodBye"
c2="Love is all you need"
c3="avisynth rocks"
c4="Gscript is awesome"
...
caption = Eval("c"+String(i))
...
RT_Debug("NewWidth= ",String(k.Width),"NewHeight =",String(k.height))
k=k.subtitle((Caption), 8, 460)


I could create an avsi file with the captions

Now, the next thing I want to do is to add transitions using vcmohan's TransAll plugin

e.g.

TransRipple(k0,k1,60,32)

where I would need to create an array of k values - I'll try to figure out the syntax I need today

[StainlessS]

Captions would be easiest handled in a text file, and add them relative to frame number.
I cant say I've ever used Dissolve for anything, nor anything like it, you are on your own there.
Best I can suggest is to get aquainted with something like ImageView (or I played with Slide Show Movie Maker about 10 years ago).
They will have stuff for dissolve etc and can be done with a few clicks rather than days of hand editing stuff together.
If you are talking about 500+ pics, that is a lot of hand editing, adding Trim/Splace or Dissolve like stuff.
You could prepare a clip of single frames with captions, and then save as BitMaps, then perhaps either use ImageView or
create a script to pull single frames in one at a times and concatenate a load of duplicates, and dissolve with what youve got so far.
Or, you might want to delay adding the captions until dissolve's etc done and all joined together, then render Subtitle captions relative to frame
number (eg current_frame/25 is caption for frame if you have 25 frames per original picture).
Anyway, if you choose to use Avisynth for this, do it in stages, with separate script for each stage.
Forget the "c0="Hello World" stuff, too labour intensive.

EDIT: I think the movie maker thing I mentioned allowed you to save as executable where you could select to play in order,
or random order as a slide show or use it as a Screen Saver. (EDIT: And display original filenames or adjust frame display duration
etc).

EDIT: What exactly are you trying to do, will the clips be of differing lengths and perhaps have a music audio track ?
Also, what dissolves are you considering, ie just a single one TransRipple, or more than one ?

[StainlessS]

EDIT: What exactly are you trying to do, will the clips be of differing lengths and perhaps have a music audio track ?
Also, what dissolves are you considering, ie just a single one TransRipple, or more than one ?

Have anticipated reply, something for you to play with.
STEP_1: ImageSplicer_Client.avs, Create AVI of Slides without captions, Edit to suit.

# ImageSplicer_Client.avs
Import("ImageSplicer.avs")

FN=".\IMAGE\*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF"
SzLimit= -768
FAR=1.333
AutoCrop=True
AutoLevelStrength=1.0
ShowBorder=False
ShowAutoLevel=False
Subs=False
CropThresh = -0.5
ScanPerc=49.0
Baffle=4
Debug=False
Debug_QBC=False
Resizer="Spline64resize(_W_,_H_)"
Matrix=2

ImageSplicer(FN,SzLimit,FAR=FAR,AutoCrop=AutoCrop,AutoLevelStrength=AutoLevelStrength,ShowBorder=ShowBorder,ShowAutoLevel=ShowAutoLevel, \
Subs=Subs, CropThresh=CropThresh,Baffle=Baffle,ScanPerc=ScanPerc,Debug=Debug,Debug_QBC=Debug_QBC,Resizer=Resizer,matrix=matrix)


Return Last # Save file as RGB32 AVI, Project_1.AVI


STEP_2: MakeDB.avs, Make a DB containing caption + other stuff.

# MakeDB.avs

PROJECT = "PROJECT_1"

FPS = 25.0 # O/P Project rate, (calc frames required)
USE_AUDIO = True # If False, dont read audio length (allow script use without existing wav files)
# Also allows setup of Int FrameCount

TEXT = PROJECT + ".TXT" # Source text containing data inserted into DBase
DB = PROJECT + ".DB"
#
DB_TYPESTRING = "ss1024ii" # String Caption[256], String AudioFile[1024], Int Transition, Int FrameCount
#
FIELD_CAPTION = 0
FIELD_AUDIO = 1
FIELD_TRANSITION = 2
FIELD_FRAMECOUNT = 3
#
RECORDS = 6 # Number of images in project
#
TYPE_BOOL = 0
TYPE_INT = 1
TYPE_FLOAT = 2
TYPE_STRING = 3
TYPE_BIN = 4
#

HASH = RT_Ord("#") # ASCII code for hash

RT_DBaseAlloc(DB,RECORDS,DB_TYPESTRING) # Allocate DBase, RECORDS records, and FIELDS fields as per TYPESTRING

TXT = RT_ReadTxtFromFile(TEXT)
NLINES = RT_TxtQueryLines(TXT)
RT_DBaseSetAttrib(DB,0,FPS) # Pass FPS into Caption&Dissolve script

GSCript("""
c = 0 cb = ColorBars().Trim(0,-1).KillAudio
Record = 0 Field = 0
For(Line=0,NLINES-1) {
S=RT_TxtGetLine(TXT,Line)
RT_DebugF("Line=%d Record=%d Field=%d S=%s",Line,Record,Field,S,name="MakeDB: ")
if(RT_Ord(S) != HASH) { # Ignore single hash line when hash at START OF LINE ONLY
Type = RT_DBaseFieldType(DB,Field)
if(Type==TYPE_STRING) {
RT_DBaseSetField(DB,record,field,S) # Set String
} Else If(Type==TYPE_INT){
RT_DBaseSetField(DB,record,field,RT_NumberValue(S)) # Set Int
}
if(Field < 2) { # We Only set first 3 Fields in this script (0 -> 2)
Field = Field + 1
} Else {
newc = cb.RT_Subtitle("%3d )\nCaption = %s\nAudioFile = %s\nTransition= %d",
\ record+1,RT_DBaseGetField(DB,record,0),RT_DBaseGetField(DB,record,1),RT_DBaseGetField(DB,record,2))
if(USE_AUDIO) {
AFN = RT_DBaseGetField(DB,record,1) # Audio Filename
AUD = WavSource(AFN)
Duration = AudioDuration(AUD) # Avisynth v2.6 ONLY
Frames = Int(Duration * FPS + 1.0)
RT_DBaseSetField(DB,record,FIELD_FRAMECOUNT,Frames) # Length of this clip in video frames
newc = newc.RT_Subtitle("Duration = %.2f\nFrames=%d",Duration,Frames,align=5)
}
c = (c.IsClip) ? c ++ newc : newc
Record = Record + 1 # next time new record
Field = 0 # start new at field 0
if(Record == RECORDS) {
Line = NLINES # Break, no more, ignore empty lines
}
}
}
}
Return c
""")

Return Last


STEP_3: Caption&Dissolve.avs, Final step create movie including audio + captions (can add-in more transitions)

#Caption&Dissolve.avs

PROJECT = "PROJECT_1"
AVI_FN = PROJECT + ".AVI"
DB = PROJECT + ".DB"
#
FIELD_CAPTION = 0
FIELD_AUDIO = 1
FIELD_TRANSITION = 2
FIELD_FRAMECOUNT = 3
#
TYPE_BOOL = 0
TYPE_INT = 1
TYPE_FLOAT = 2
TYPE_STRING = 3
TYPE_BIN = 4
#
RECORDS = RT_DBaseRecords(DB)
FIELDS = RT_DBaseFields(DB)
#
OVERLAP = 50

FPS = RT_DbaseGetAttrib(DB,0) # Get as set by MakeDB.AVS
AVI = AviSource(AVI_FN).AssumeFPS(FPS)

TRANSITION_NONE = 0
TRANSITION_DISSOLVE = 1

GSCript("""
c = 0
For(record=0,RECORDS - 1) {
Frames = RT_DBaseGetField(DB,record,FIELD_FRAMECOUNT)
Transition = RT_DBaseGetField(DB,record,FIELD_TRANSITION) # For whatever you want it
k = AVI.Trim(record,-1)
Caption= RT_DBaseGetField(DB,record,FIELD_CAPTION)
Caption= RT_String("%d/%d ) %s",record+1,RECORDS,Caption) # Add TrackNo indicator
RT_DebugF("%s",Caption)
k = k.SubTitle(Caption,Align=1,Size=24)
k = k.Loop(Frames,0,0) # Make Clip same length as audio
AUD = WavSource(RT_DBaseGetField(DB,record,FIELD_AUDIO))
k = AudioDub(k,AUD)
k = k.Trim(0,0) # Exactly to length
if(!c.IsClip) {
c = k # Non to join to
} Else {
if(Transition == TRANSITION_DISSOLVE) {
c = Dissolve(c,k,OVERLAP)

### INSERT MORE TRANSITION CODE ELSE's HERE

} Else { # CATCH ALL - TRANSITION_NONE
c = c ++ k
}
}
}
Return c
""")

Return Last



DUMMY DATA for STEP_2 MakeDB.avs, ie "PROJECT_1.TXT"

#1) Hash is a comment and record separator, Caption, AudioFile, Transition code
All you need is love - The Beatles
.\MUSIC\AllYouNeedIsLove.wav
1
#2)
If I Fell - The Beatles
.\MUSIC\IfIFell.wav
1
#3)
A hard days night - The Beatles
.\MUSIC\AHardDaysNight.wav
1
#4)
I am The walrus - The Beatles
.\MUSIC\IAmTheWalrus.wav
1
#5)
Strawberry Fields Forever - The Beatles
.\MUSIC\StrawberryFieldsForever.wav
1
#6)
Lucy in the sky with diamonds - The Beatles
.\MUSIC\LucyInTheSkyWithDiamonds.wav
1

I've only implemented Transition 1

Good Luck.

EDIT: AudioDuration() in STEP_2 requires v2.6, Also, MUST NOT have any blank or missing lines in Project_1.txt
Hashes are not needed, just visual separators.

[wiseant]

Hi StainlessS

Wow, that is a lot of information - thanks once again for your time.


Captions would be easiest handled in a text file, and add them relative to frame number.


Yes, I agree with that - the only problem I see is correctly matching each caption to each image. Probably best to process the stream of images first - then create the text file of Captions. The order depends on the naming convention - e.g. if I have image names of 1.bmp ... 500.bmp then the sequence goes 1.bmp, 10.bmp, 100.bmp, 101.bmp ... 199.bmp, 2.bmp, 20.bmp, etc. And if I have descriptive names without numbers - it would be difficult to presume the order that the images are processed in. But if I add a one or two letter prefix -> ts0001.png ... ts0100.png - the order is preserved


EDIT: What exactly are you trying to do, will the clips be of differing lengths and perhaps have a music audio track ?
Also, what dissolves are you considering, i.e. just a single one TransRipple, or more than one ?


Right now, I am focusing on doing transitions with the TransAll plugin - I got it to work whereby there are different transitions between each image. Now I need to experiment with the fastest way of doing captions - I will try your script re reading captions from a text file.

The next thing I want to look at is adding audio - but I don't think I want an audio clip for each image - they will only display for 3 to 5 secs - so once I know the duration of the entire image sequence I can add audio. For example, if I have 500 images x 3 sec = 1500 sec = 25 minutes - I could create a 25 minute audio file or loop a 3 minute audio 5 times [though I need to find out how to do that].

Here is my script for transitions - I am using the old template simply because I am familiar with it - haven't had time to study "ImageSplicer.avs"

I created a new variable k1 - so the transitions are between k and k1. And the transition used is based on the modulus values.

You are a better coder than me, so I am sure there is a more elegant way to do this . . .



fps = 24
end = 48
GScript("""
picclip = 0
DEBUG = True
result = RT_WriteFileList("D:\Temp\*.png","sept.List")
Assert((result!=0),"No BMP files found")
BMPLIST = RT_ReadTxtFromFile("sept.List")
FILES = RT_TxtQueryLines(BMPLIST)
RT_Debug("Found BMP files = " + String(FILES))
RT_Debug(" ")
canvaswidth = 960
canvasheight = 400
For(i=0,FILES-2) {
FN = RT_TxtGetLine(BMPLIST,i)
FN1 = RT_TxtGetLine(BMPLIST,i+1)
RT_Debug("Processing BMP File",FN)
RT_Debug("Processing BMP File",FN1)
k = ImageSource(FN,end=end).ConvertToRGB32()
k1 = ImageSource(FN1,end=end).ConvertToRGB32()
iHeight = k.height
iWidth = k.width
iHeightN = k1.height
iWidthN = k1.width
#
if (iHeight >= iWidth) {
iHeight1 = canvasheight
iWidth1 = (canvasheight * iWidth) / iHeight
x = canvaswidth - iWidth1
x1 = x/2
#
if (iwidth1 % 2 > 0) {
iwidth1 = iwidth1+1
}
k = k.bilinearresize(iWidth1,canvasheight).addborders(x1,0,x1,0,$101010)
}
#
if (iHeightN >= iWidthN) {
iHeight1N = canvasheight
iWidth1N = (canvasheight * iWidthN) / iHeightN
x = canvaswidth - iWidth1N
x1 = x/2
#
if (iwidth1N % 2 > 0) {
iwidth1N = iwidth1N+1
}
k1 = k1.bilinearresize(iWidth1N,canvasheight).addborders(x1,0,x1,0,$101010)
}
#
if (iWidth > iHeight) {
iWidth1 = canvaswidth
iHeight1 = (canvaswidth*iHeight)/iWidth
y = canvasheight-iHeight1
#
if (iheight1 % 2 > 0) {
iheight1 = iheight1+1
}
y1 = y/2
k = k.bilinearresize(iWidth1,iHeight1).addborders(0,y1,0,y1,$101010)
}
#
if (iWidthN > iHeightN) {
iWidth1N = canvaswidth
iHeight1N = (canvaswidth*iHeightN)/iWidthN
y = canvasheight-iHeight1N
#
if (iheight1N % 2 > 0) {
iheight1N = iheight1N+1
}
y1 = y/2
k1 = k1.bilinearresize(iWidth1N,iHeight1N).addborders(0,y1,0,y1,$101010)
}
#
if (iwidth1 > canvaswidth) {
iwidth1 = canvaswidth
iheight1 = (canvaswidth*iheight)/iwidth
y = canvasheight-iheight1
#
if (iheight1 % 2 > 0) {
iheight1 = iheight1+1
}
y1 = y/2
k = k.bilinearresize(iWidth1,iHeight1).addborders(0,y1,0,y1,$101010)
}
#
if (iwidth1N > canvaswidth) {
iwidth1N = canvaswidth
iheight1N = (canvaswidth*iheightN)/iwidthN
y = canvasheight-iheight1N
#
if (iheight1 % 2 > 0) {
iheight1N =iheight1N+1
}
y1 = y/2
k1 = k1.bilinearresize(iWidth1N,iHeight1N).addborders(0,y1,0,y1,$101010)
}
#
if (iHeight1 > canvasheight) {
iHeight1 = canvasheight
iWidth1 = (canvasheight*iWidth)/iHeight
x = canvaswidth-iWidth1
#
if (iwidth1 % 2 > 0) {
iwidth1 = iwidth1+1
}
x1 = x/2
k = k.bilinearresize(iWidth1,iHeight1).addborders(x1,0,x1,0,$101010)
}
#
if (iHeight1N > canvasheight) {
iHeight1N = canvasheight
iWidth1N = (canvasheight*iWidthN)/iHeightN
x = canvaswidth-iWidth1N
#
if (iwidth1N % 2 > 0) {
iwidth1N = iwidth1N+1
}
x1 = x/2
k1 = k1.bilinearresize(iWidth1N,iHeight1N).addborders(x1,0,x1,0,$101010)
}
#
RT_Debug("NewWidth= ",String(k.Width),"NewHeight =",String(k.height))
#
mod1=Fmod(i, 1)
mod2=Fmod(i, 2)
mod3=Fmod(i, 3)
mod4=Fmod(i, 5)
mod5=Fmod(i, 7)
mod6=Fmod(i, 11)
mod7=Fmod(i, 13)
mod8=Fmod(i, 17)
mod9=Fmod(i, 19)
#
if (mod1==0) {
m=TransPush(k,k1,20)
}
#
if (mod2==0) {
m=TransSlantRollIn(k,k1,20)
}
#
if (mod3==0) {
m=TransWipe(k,k1,20,"up")
}
#
if (mod4==0) {
m=TransSlideOut(k,k1,20,"west")
}
#
if (mod5==0) {
m=TransSprite(k,k1,20,"left")
}
#
if (mod6==0) {
m=TransVenetianBlinds(k,k1,20,type="vert")
}
#
if (mod7==0) {
m=TransVenetianBlinds(k,k1,20,type="hor")
}
#
if (mod8==0) {
m=TransWeave(k,k1,20,"weave")
}
#
if (mod9==0) {
m=TransWipe(k,k1,20,"down")
}
#
picclip = (IsClip(picclip)) ? picclip ++ m : m
RT_Debug("")
}
""")
PicClip
assumefps(fps)



And here is a link for a sequence with transitions:

https://www.mediafire.com/?kvf57gfkcycqv36

I just extracted 100 images from "Tears of Steel" open source project - there are a few frames that have blocks - ? decode/encode error?

wiseant

[StainlessS]

Wiseant,
I currently have less time to spend here, my interests have dispersed.
Take whatever you can from what was offerred, and if of use then please take.
Good luck.

[StainlessS]

@Wiseant
Sorry I was a little curt in my last post, I am not able to assist at present, a little busy getting into android
stuff, have spent a little time trying to (successfully) un-screw my botched flashing of my new tab (which I absolutely love,
way better than previous 3 or 4 that I've had).
Anyways, hope you sort your stuff out, and may your god smile upon you. :)

[wiseant]

@StainlessS

No problem. I was able to figure everything out - I used your suggestion about creating a "Captions" file then reading it and using Subtitle() to place it. I also figured out how to do transitions - pleased with the simple routine that I created . . .

Android stuff eh - have you looked into the Intel XDK?

"Intel® XDK HTML5 Cross-platform Development Tool provides a simplified workflow to enable developers to easily design, debug, build, and deploy HTML5 web and hybrid apps across multiple app stores, and form factor devices. "

[StainlessS]

Thanks wiseant, I'll take a look. I'm not really interested in eg Windows phone or iPhone but I guess that could change.
I wanna get into all sides of Android, ie base Linux and diffs to android, customising ROM's, and apps.

Glad you got your stuff sorted, bye.

[fvisagie]

Is it a concept principle of AviSynth, that variables by reference are impossible?

I am unsure how useful it may be to you, but referencing variables can be approximated with Eval():
strvar = "Hello, World!"
strvarptr = "strvar"

MessageClip(Eval(strvarptr))

[StainlessS]

Yes, that can be done and better in a plugin.

Some parts of RT_ already do similar to set some variables although usually only taking a variable name Prefix and
setting some variable to a particular value.
(Plugin can set both Global and Local vars to a particular value, whether vars already existing or not)

Some parts of RT_ also return a string so as to set multiple variables all at once within an Eval.
eg, Return_String="Var_A = 27 Var_B= 42 Var_C = 66", where "Var_" might be the Prefix supplied to function.

EDIT: And I guess that 'by reference' is not actually being used here anyway, the item that is actually being set is (in your example) the
un-named argument to MessageClip() which cannot be referenced anyway or anywhere.

[StainlessS]

Update to previously posted DGIndex script template:

/*
ReDAR_DGIT.avs v0.1 - DGIndex AVS Template - By StainlessS.
Requires GSCript, RT_Stats v1.30+, RoboCrop, TIVTC, FFT3DFilter, NicAudio plugins.
Requires MediaInfo CLI version.

Progressive, or NTSC FILM with soft pulldown.
Purpose:- Crop letterboxing, with additional crop/resize to LESS wide screen DAR, CANNOT make more wide (DAR = Display Aspect Ratio).
Useful to crop to eg 16:9 on 16:9 hand held device.
Can also resize to SAR 1:1 [Sample (pixel) Aspect Ratio] for devices/players that cannot deal with non 1:1 clips.
Can Also call TFM and TDecimate to remove soft pulldown from NTSC FILM when Honour Pulldown Flags was used in DGIndex.
Signals resultant DAR for MeGUI in Global vars MeGUI_DarX and MeGUI_DarY.
*/

SHOW=False # Show Info on frame.
FRAME_DAR =RT_Undefined # Float, Input DAR of frame before RoboCrop(including letterboxing). RT_Undefined auto interrogates d2v file.
OUT_DAR =2.21/1.0 # Float, 0.0 = Use RoboCrop'ed DAR : Other = Additional crop to output DAR, eg FRAME 16:9, Robocrop'ed 2.4:1, OUT 16:9.
OUT_DAR_IS_LIMIT=True # If False, then OUT_DAR is required output DAR. (Error if OUT_DAR > RoboCrop'ed DAR, Cannot widen aspect ratio)
# If True and OUT_DAR != 0.0(0.0=RoboCrop'ed DAR), then reduce DAR only if RoboCrop'ed DAR is greater than OUT_DAR.
# Allows setting of maximum allowable output DAR(feel less like a peeping Tom for 2.4:1, eg limit to 16:9 on 4:3 device).

OUT_HEIGHT=0 # Int, 0 = Use RoboCrop'ed Height, Otherwise vertical resize to OUT_HEIGHT
OUT_WIDTH =0 # Int, 0 = Use OUT_SAR : Otherwise ignore OUT_SAR and use explicit OUT_WIDTH
OUT_SAR =0.0 # Float, Used if OUT_WIDTH=0 : 0.0 = Original SAR(crop only if OUT_HEIGHT=0) : 1.0 = 1:1 SAR
#
STRENGTH =0.75 # Float, Luma Auto Level strength (0.0 -> 1.0, 0.0=off)
UPCONV=True # Bool, False = no upsample(Progressive Encoded), True=upsample to YUY2(Progressive encoded as Interlaced), Can leave True.
###### Less often Changed
SAMPLES=32 # Int, Frames to sample, used by both RoboCrop and Auto Luma Levelling.
THRESH=-32 # int, -32 = RoboCrop AUTO Thresh
CROPMODE=2 # int, RoboCrop CropMode. 1=CropLess, 2= CropMore(Default)
TFM=True # If True, and FrameRate > 29.0 and MPEG-2 and DGIndex 'Honour Pulldown Flags' then Call TFM and TDecimate.
TFM_ANIME=False # True if Anime, For TDecimate
XMOD=0 # Int, 0=AUTO For crop/resize.
WMOD=4 # Int, 0=AUTO For crop/resize. (4 avoids problems on eg mp4 encodes displayed in MPC-HC)
MEDIAINFO="C:\BIN\MEDIAINFO.EXE" # To Check Aspect Ratio in IFO and compare with VOB : If conflict use IFO_MODE
IFO_MODE= 0 # 0 Error Alert on VOB/IFO DAR mismatch, 1 = Use IFO Frame Aspect Ratio
# List of programs that DONT like WMOD=2 for eg YUV.
# WMOD=0(AUTO) For YV12 will use eg 2 but switch to 4 if Process Name in WMOD4_PROGS list below (can append other names as in Task Manager).
WMOD4_PROGS="
VirtualDubMod.exe # VirtualDub Current OK
AVSEdit.exe # Built in Player
mplayer2.exe
wmplayer.exe
mpc-hc.exe
"
ROBOCROP_DEBUG=False
AUTOLEVEL_DEBUG=False
######################
### Filled in by DGIndex when using as DGIndex Template
VideoFileName ="__vid__"
AudioFileName =LCase("__aud__")
AudioDelay =Value("__del__")
######################
MPEG2Source(VideoFileName,UpConv=(UPCONV)?1:0) # Upconv to convert to YUY2, unmangle chroma if encoded as interlaced.

AudioExt=RT_GetFileExtension(AudioFileName)

Audio= (AudioExt==".ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
\ : (AudioExt==".mpa"||AudioExt=="mp1"||AudioExt==".mp2"||AudioExt==".mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
\ : (AudioExt==".wav") ? RaWavSource(AudioFileName)
\ : (AudioExt==".dts") ? NicDTSSource(AudioFileName)
\ : 0

Assert(!isInt(Audio),"NO AUDIO")

(!isInt(Audio)) ? AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0) : NOP # Trim, chop/add audio to length

(!isInt(Audio) && AudioRate() <> 44100) ? ResampleAudio(44100) : NOP
#(!isInt(Audio) && AudioRate() <> 48000) ? ResampleAudio(48000) : NOP
#(!IsInt(Audio) && (AudioRate() <> 44100 && AudioRate() <> 48000))?ResampleAudio(44100) : NOP
#(!IsInt(Audio))? ConvertToMonO().ResampleAudio(44100) : NOP

######
GSCript("""
myName="ReDAR_DGIT: "
D2VVOBS_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=1)
D2V_START=RT_NumberValue(D2VVOBS_S) + 3
D2V_LINES = 12
SS0=""
if(TFM && FrameRate>29.0) {
MPEG_LINE = RT_FileFindStr(VideoFileName,"MPEG_Type",Sig=False,start=D2V_START,lines=D2V_LINES)
MPEG_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=MPEG_LINE)
MPEG2 = (RT_FindStr(MPEG_S,"=2",Pos=9) >= 0)
if(MPEG2) {
HONOUR_LINE = RT_FileFindStr(VideoFileName,"Field_Operation",Sig=False,start=D2V_START,lines=D2V_LINES)
HONOUR_S=RT_ReadTxtFromFile(VideoFileName,lines=1,Start=HONOUR_LINE)
HONOUR_PULLDOWN_FLAGS=(RT_FindStr(HONOUR_S,"=0",Sig=False,Pos=15) > 0)
if(HONOUR_PULLDOWN_FLAGS) {
FIN_LINE = RT_FileFindStr(VideoFileName,"FINISHED",Sig=False)
Assert(FIN_LINE>=0,myName+"'FINISHED' NOT FOUND in d2v file")
FIN_S = RT_ReadTxtFromFile(VideoFileName,lines=1,Start=FIN_LINE)
IsFILM = (RT_FindStr(FIN_S,"FILM",Sig=False,Pos=9) > 0)
FILM_PERC = RT_NumberValue(FIN_S,Pos=9)
FILM_PERC = (IsFILM) ? FILM_PERC : 100-FILM_PERC
if(FILM_PERC >= 95) {
SS0=RT_String("Calling TFM and TDecimate")
RT_DebugF(SS0,name=myName)
SS0=SS0+Chr(10)
TFM(d2v=VideoFileName)
TDecimate(mode=(TFM_ANIME)?1:0)
}
}
}
}
# Cannot eg Trim/Add frames before here, will interfere with TFM and TDecimate
if(!Defined(FRAME_DAR)) { # Interrogate d2v for FRAME Display Aspect Ratio (incl letterboxing)
FDAR_LINE=RT_FileFindStr(VideoFileName,"Aspect_Ratio=",Sig=False,start=D2V_START,lines=D2V_LINES)
Assert(FDAR_LINE>=0,myName+"'Aspect Ratio' NOT FOUND in d2v file")
FDAR_S=MidStr(RT_ReadTxtFromFile(VideoFileName,lines=1,start=FDAR_LINE),14)
FNDS=RT_String("16:9,625\n16:9,525\n4:3,625\n4:3,525") # DGIndex version of MPEG1 PAR (y/x)
REPS=RT_String("0.7031\n0.8437\n0.9375\n1.1250") # Actual numbers replacing (MPEG1 PAR=y/x)
FDAR_S=RT_StrReplaceMulti(FDAR_S,FNDS,REPS) # Convert MPEG1 625(PAL),525(NTSC) Ratio to number
FDAR_N=Value(FDAR_S) # May be Float
COLON=RT_FindStr(FDAR_S,":") # Ratio ?
FDAR_D=(COLON==0) ? 1 : RT_NumberValue(FDAR_S,Pos=COLON+1)
Assert(FDAR_N>0 && FDAR_D>0,myName+"Error Reading d2v 'Aspect Ratio' (N="+String(FDAR_N)+",D="+String(FDAR_D)+")")
# Get Frame DAR from:- 1:1 SAR, or MPEG2 DAR, or MPEG1 PAR(y/x)
FRAME_DAR=(FDAR_N==1 && FDAR_D==1) ? Float(Width)/Height : (COLON!=0) ? Float(FDAR_N)/FDAR_D : (Float(Width)/Height)/FDAR_N
FRAME_DAR = Round(FRAME_DAR * 1000.0) / 1000.0
If(MEDIAINFO != "" && Exist(MEDIAINFO)) {
MPEG_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=2)
MPEG_S= RT_TxtGetLine(MPEG_S) # remove newline
M_EXT = RT_GetFileExtension(MPEG_S)
if(M_EXT == ".VOB") {
IFO = RT_FilenameSplit(MPEG_S,7)
IFO = LeftStr(IFO,Strlen(IFO)-1)
IFO = IFO + "0.IFO"
if(Exist(IFO)) {
RT_DebugF("File Is VOB and IFO exists",name=myName)
TMP=RT_GetFullPathName("~"+RT_LocalTimeString + ".txt")
RT_DebugF("Calling MediaInfo on IFO '%s'",IFO,name=myName)
# write Aspect Ratio in form "1.333" to file TMP
result=RT_Call(
\ RT_QuoteStr(MEDIAINFO) +
\ " --LogFile=" +
\ RT_QuoteStr(TMP) +
\ " --Inform=Video;%DisplayAspectRatio% " +
\ RT_Quotestr(IFO)
\ , hide=True,Debug=True
\ )
if(result==0 && Exist(TMP)) {
MI_TXT = RT_TxtGetLine(RT_ReadTxtFromFile(TMP))
RT_FileDelete(TMP)
Val = Value(MI_TXT)
RT_DebugF("MediaInfo Text = %s",MI_TXT,name=myName)
if(Val != FRAME_DAR) {
RT_DebugF("IFO(%f) and d2v(%f) Aspect Ratios DISAGREE, Using IFO AR = %f",Val,FRAME_DAR,Val,name=myName)
Assert(IFO_MODE==1,RT_String("IFO(%f) and d2v(%f) Aspect Ratios DISAGREE\nSet FRAME_DAR manually"
\ ,Val,FRAME_DAR))
FRAME_DAR=Val
} Else {
RT_DebugF("IFO and d2v Aspect Ratios AGREE on %f",FRAME_DAR,name=myName)
}
} Else {
RT_DebugF("MediaInfo Fails, result = %d",result,name=myName)
}
}
}
}
}

PROCESSNAME=RT_GetProcessName()
XMOD = (XMOD!=0) ? XMOD : RT_ColorSpaceXMod()
WMOD = (WMOD!=0) ? WMOD : (IsYUV() && RT_TxtFindStr(WMOD4_PROGS,PROCESSNAME,Sig=False)>=0) ? 4 : RT_ColorSpaceXMod
HMOD = Max(RT_ColorSpaceYMod(Laced=false),2)
RT_DebugF("ProcessName = %s WMOD=%d HMOD=%d",PROCESSNAME,WMOD,HMOD,name=myName)
OUT_DAR =Float(OUT_DAR)
OUT_SAR =Float(OUT_SAR)
SAR=RT_GetSAR(FRAME_DAR) # Sample (pixel) aspect ratio of input frame
SS1=RT_String("INPUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",Width,Height,FRAME_DAR,SAR,Float(Width)/Height)
RT_DebugF(SS1,name=myName)
SS1=SS0+SS1
RoboCrop(samples=SAMPLES,thresh=THRESH,cropmode=CROPMODE,debug=ROBOCROP_DEBUG,hmod=HMOD,Laced=false)
DAR=RT_GetDAR(SAR) # Image DAR after RoboCrop'ed letterboxing
SS2=RT_String("RoboCrop: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",Width,Height,DAR,SAR,Float(Width)/Height)
RT_DebugF(SS2,name=myName)
if(OUT_DAR_IS_LIMIT && OUT_DAR > DAR) {
OUT_DAR = DAR # Do not report Error, we are just limiting DAR
RT_DebugF("Ignoring OUT_DAR, (RoboCrop'ed DAR is Less))",name=myName)
}
Assert(OUT_DAR <= DAR,"OUT_DAR("+String(OUT_DAR,"%.3f") + ") Must be <= DAR("+String(DAR,"%.3f")+")")
OUT_SAR = (OUT_SAR == 0.0) ? SAR : OUT_SAR
OUT_DAR = (OUT_DAR == 0.0) ? DAR : OUT_DAR
OUT_HEIGHT=OUT_HEIGHT/HMOD*HMOD
OUT_WIDTH =OUT_WIDTH /WMOD*WMOD
H = (OUT_HEIGHT == 0) ? Height : OUT_HEIGHT # RoboCrop'ed OR user supplied resize height
# If OUT_WIDTH==0 Then use H(Height or user OUT_HEIGHT) & OUT_SAR(SAR or user OUT_SAR) to calc output width ELSE user OUT_WIDTH.
W = (OUT_WIDTH==0) ? Int(H * OUT_DAR / Float(OUT_SAR) / WMOD) * WMOD : OUT_WIDTH
if(OUT_WIDTH==0 && H==Height && OUT_SAR==SAR) { # Output RoboCrop'ed Width,Height, No resize, non-fractional cropping
CROP_WIDTH = W # Crop Only, maintain SAR, with approx OUT_DAR (occurs when original OUT_WIDTH==0 and OUT_HEIGHT==0 and OUT_SAR==0.0)
CROP_LEFT = Int((Width - CROP_WIDTH) * 0.5 / XMOD + 0.5) * XMOD # Crop centering, CROP_WIDTH and CROP_LEFT are Int
} Else { # Crop/Resize with fractional cropping to OUT_DAR
CROP_WIDTH = Width * (Float(OUT_DAR) / DAR)
CROP_LEFT = (Width - CROP_WIDTH) * 0.5 # Crop centering, CROP_WIDTH and CROP_LEFT are Float
}
# Now decide none, crop only, or crop/resize. CROP_WIDTH and CROP_LEFT may be Int or Float.
if(CROP_WIDTH==Width && CROP_LEFT==0 && W==Width && H==Height) {
SS3=RT_String("NO CROP/RESIZE, %dx%d DAR=%.3f",Width,Height,DAR)
} Else if(Int(CROP_LEFT/XMOD)*XMOD == CROP_LEFT && CROP_WIDTH==W && H==Height) { # Crop Only
CROP_LEFT=Int(CROP_LEFT)
CROP_WIDTH=Int(CROP_WIDTH)
Crop(CROP_LEFT,0,CROP_WIDTH,-0) # Cropping does not alter SAR
DAR=RT_GetDAR(SAR) # Calc resultant DAR after crop
SS3=RT_String("Crop(%.d, %d, %d, %d)",CROP_LEFT,0,CROP_WIDTH,-0)
} Else {
DAR=RT_GetCropDAR(DAR,CROP_LEFT,0,CROP_WIDTH,-0) # Pre-calc resultant DAR of Last clip, after possible fractional cropping
Spline36Resize(W,H, CROP_LEFT,0,CROP_WIDTH,-0)
SAR=RT_GetSAR(DAR) # Calc new SAR due to Resizing
SS3=RT_String("Resize(%d,%d, %.1f, %d, %.1f, %d)",W,H, CROP_LEFT,0,CROP_WIDTH,-0)
}
RT_DebugF(SS3,name=myName)
SS9=""
if(STRENGTH>0.0) { # Auto Luma Levelling
Eval(RT_QueryLumaMinMax(samples=SAMPLES,debug=AUTOLEVEL_DEBUG)) # Get luma min/max over SAMPLES frames
if(IsRGB()) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
SS9=RT_String("Levels(%d,1.0,%d,%d,%d,Coring=False)",ALMin,AlMax,CSMin,CSMax)
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring
}
}
RT_DebugF ("%s\nOUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f ",SS9,Width,Height,DAR,SAR,Float(Width)/Height,name=myName)
(SHOW) ? RT_Subtitle("%s\n%s\n%s\n%s\nOUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",SS1,SS2,SS3,SS9,Width,Height,DAR,SAR,Float(Width)/Height) : NOP
RT_SignalDar(OUT_DAR) # Tell MeGUI the Exact required Display Aspect Ratio rather than approximate.
RT_DebugF("Signalled MeGUI DAR=%f MeGUI_DarX=%d MeGUI_DarY=%d",OUT_DAR,MeGUI_DarX,MeGUI_DarY,name=myName)
ConvertToYV12
# FFT3DFilter(Sigma=1.6,plane=0,Sharpen=0.03)
""")
Return Last

[Forensic]

@StainlessS
If RT_Stats supports it for a planar video, what is the command to change every pixel with Y=250 pixel to now be Y=180? (my current method is extremely slow). Thank you in advance.

[StainlessS]

If RT_Stats supports it for a planar video, what is the command to change every pixel with Y=250 pixel to now be Y=180? (my current method is extremely slow).

Sorry Forensic, nothing to do that (that would require a standard filter rather than a runtime environment function).
How about MaskTools Lut stuff ?

EDIT:
Maybe something like this:

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}

Avisource("Test_hotspot.avi").ConvertToYV12()
ORG=Last
#TARGET= 250
TARGET= 200
SWAP = 180
#S=RT_string("(x==%d) ? %d : x",TARGET,SWAP)
S=RT_string("(x>=%d) ? %d : x",TARGET,SWAP)
q=MT_Polish(S)
RT_DebugF("%s\n%s",S,Q)
MT_lut(MT_Polish(S),U=2,V=2)

D1=ClipDelta(ORG,Last)
D2=ClipDelta(ORG,Last,Amp=True)

T= StackHorizontal(ORG,Last)
B= StackHorizontal(D1,D2)
StackVertical(T,B)


There dont seem to be any Luma of 250 so I modded a bit to do something (uncomment relevant lines)

[Forensic]

Worked like a charm. Thank you!!!!!

[martin53]

Sorry my question is off topic here - but I didn't dare to post in the AviSynth release thread since I believe this question must be too stupid. And since I used RT_Stats for my test below, it's at least a little related to this thread ;)

Problem: With the most recent AviSynth 2.60 RC1, the AviSynth.h from this distribution, empty NegInvert/Simplesample and VS Express 2013, I just couldn't successfully use a "LoadPlugin()" command in AvsPmod (V2.51).
There is always a crash window like
Traceback (most recent call last):
File "avsp.pyo", line 9295, in OnMenuVideoRefresh
File "avsp.pyo", line 14083, in ShowVideoFrame
File "avsp.pyo", line 14598, in UpdateScriptAVI
File "pyavs.pyo", line 138, in __init__
File "pyavs.pyo", line 249, in CreateErrorClip
UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 24: ordinal not in range(128)
I fiddled the whole day because I thought I made some mistake regarding Unicode/multibyte characters, created new projects, installed VC2010 :confused:
Finally I decided to test the same thing with a proved plugin like RT_Stats (your honor, StainlessS).
And
LoadPlugin("C:\RT_Stats26.dll")
Version()

fails the same. Vdub fails, too (less copy/paste friendly). Same with AviSynth26a5.

But I can use RT_Stats26.dll and my VC2013-compiled test plugin from the autoload plugin directory! (I just did not want to include my work in progress there too soon, but instead only load it into a designated test script)

Question: I read the AviSynth manual (the LoadPlugin section), looks to me as if LoadPlugin is still the right command. Can someone please retest, and any idea what I am doing wrong?

[Groucho2004]

What does VDub report?

[martin53]

Thanks to Au3Info_x64:
Avisynth open failure:
LoadPlugin: ƒvƒ‰ƒOƒCƒ“�‰Šú‰»ƒGƒ‰�[
(......\__TestScript.avs, line 1)
And this is exactly the same error message I get when I provide a non-existent filename, an invalid path, or an empty string to the function.

[Groucho2004]

Post the script (the actual .avs file, don't just paste the content here).

[StainlessS]

I defer to the right honourable Groucho2004 on this.
Martin53, that was a real long safari you were on, me thought you got eaten by Lions and Tigers and Bears! Oh my!

[StainlessS]

Same problem was reported here, although there does not seem to be any indication of the cause.
http://forum.doom9.org/showthread.php?p=1697206#post1697206

[martin53]

Post the script (the actual .avs file, don't just paste the content here).

With all respect - are you asking me to add a 14 characters text file as an attachment, instead of copying it into the post, or do I get you wrong? I wrote several of these scripts and all do the same. Open AvsPmod, File|New, type LoadPlugin("") and press F5. That's all.

@StainlessS: I wasn't away all the time, but busy ... Nice to feel you're glad I post again.

[StainlessS]

Yep, I saw you about 2 or 3 weeks ago.
I think Grouchy may have wanted to view in HEX/ASCII editor, have you viewed in HEX ?
Also, if you have some free time, could you take a gander at this:- http://forum.doom9.org/showthread.php?p=1708424#post1708424
Any comments appreciated.

[StainlessS]

Here same 0xC1 error as reported in link to similar problem as in post #323.
http://forum.doom9.org/showthread.php?p=1696498#post1696498

Perhaps you just copied runtimes into system32/SysWOW64 instead of using installer. (also might require more current version).

EDIT: Your attached file just looks like plain ASCII. Above should fix your problem I think (reinstall latest runtimes, not copy).

[martin53]

1st of all, thanks for the support!

I remember two things: when I ran the installer, I got a windows compatibility problem assistant window - or how that may be called in english. After the installer had finished, it proposed to run it again with the proposed compatibility settings; and since I did not see an objection, I confirmed that, so I effectivly ran the installer twice.
Today, when I first opended AVSInfoTool127, it showed the old 26a5 version first, but after some seconds, without any intervention from my side, the GUI kinda reopened and showed RC1. When I ran AVSInfoTool again, I think it re-opened once more after some 5 or 10 seconds, but before and afterwards it always showed RC1.
'Version()' also shows RC1.
The dll in C:\Windows\system32 is 393 kB, with date 01-13-2015, as is the one in SysWOW64.

I just removed the compatibility settings for the installer and re-installed, same. Then uninstalled via control panel / reinstalled with installer: Same.

EDIT I don't remember copying/moving dlls manually on this machine. Of the MS VS redistributalbles, these are installed (read from control panel):
2005 8.0.56336 426KB and 8.0.50727.42
2005 (x64) 8.0.61000
2008 x64 9.0.30729.6161
2008 x86 9.0.21011, 9.0.30729, 9.0.30729.17, 9.0.30729.4974 and 9.0.30729.6161
2010 x64 10.0.40219
2010 x86 10.0.40219
Visual Studio 2012 x64 Redistributables 14.0.0.1 (*not* 'Microsoft Visual C++ 20xx Redistributable...')
Visual Studio 2012 x86 Redistributables 14.0.0.1
2013 x64 12.0.30501
2013 x86 12.0.30501

[Groucho2004]

Today, when I first opended AVSInfoTool127, it showed the old 26a5 version first, but after some seconds, without any intervention from my side, the GUI kinda reopened and showed RC1. When I ran AVSInfoTool again, I think it re-opened once more after some 5 or 10 seconds, but before and afterwards it always showed RC1.
I can't even begin to imagine how this could happen. Something really fishy going on on your system. Sorry, at this point I have no advice.

[Wilbert]

Do you have multiple avisynth.dll's on your system?

[StainlessS]

Hi people. Just warning of removal of two functions from RT_Stats.

RT_ArrayGetNels(string FileName)
Filename, Array Filename, no default.
Returns int, total number of elements in array as set by RT_ArrayAlloc, or current number of elements if RT_ArrayAppend used to add
elements to end of single dimension array, or RT_ArrayExtend used to increase Dim1 dimension.
For a 3 dimensional array where Dim1=10, Dim2=20,Dim3=30 would return 10*20*30 ie 6000.

***
***
***

RT_ArrayGetNelMax(string FileName)
Filename, Array Filename, no default.
Returns int, maximum number of elements that could theoretically be stored in current array, taking into account the
current number of dimensions and the size of dimensions 2 and 3.


Dont know if anybody (but me) actually uses the ARRAY functions, dont think these two funcs are too important.
Reason is that I will be implementing file size greater than 2GB and those fn's will in some cases not be able to return the total number
of elements in an ARRAY.

I have so far implemented for big DBase file, described a bit here:

RT_DBaseRecordsMax(String Filename,Bool "Current"=True)
Filename, DBASE Filename, no default.
Return maximum number of records that can be stored in the DBase file.
When Current==True (default [new argument v1.45]) takes into account the file system and space on drive available to
the user (ie takes into account user disk space quotas, and eg FAT32 file system).
When Current==False, (as pre v1.45) returns maximum number of records that could be stored in file based on
Min((256GB - 1MB - DBase header size) / Record Size, $7FFFFFFE) [where record size is sum of all field sizes].
So, new to v1.45, shows as default the current maximum which will vary if copied to eg FAT32 filesystem, or when disk
space is reduced. Maximum possible file size has been limited to about 256GB, for no particular reason, maximum number of
records is $7FFFFFFE, ie maximum +ve integer - 1 (a lil bit more than 2 billion). Where available disk space is 'unlimited',
max size will still be limited to about 256GB or $7FFFFFFE records.


Hope the changes do not cause major problems for anyone, say here if they do, and make alternative suggestions if you have any.

Think I might also take this opportunity to remove some args from these two function, below

RT_LumaSceneChange(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2), \
Float "Bias"=0.0,Float "Gain"=1.0,Float "Cont"=1.0,Float "Rpow"=1.0,Float "Spow"=1.0,Float "SPMid"=0.5,Bool "Pord"=false)

RT_LumaMovement(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(Width>1100||Height>600?3:2),
int "Mode"=0,int "BlkW"=64,int "BlkH"=64 \
Float "Bias"=0.0,Float "Gain"=1.0,Float "Cont"=1.0,Float "Rpow"=1.0,Float "Spow"=1.0,Float "SPMid",Bool "Pord"=false)


The above args seemed like a good idea at the time, and were only included as part of a bug fix update, left in before I could verify if there were of much use.
They seem not to be much use at all, except with occasional clip [unfortunately, the only clip I had tested with was such a clip].

EDIT: This function will still be available (although not implemented quite like described).

RT_ArrayGetDim1Max(string FileName)
Filename, Array Filename, no default.
Returns int, maximum theoretical Dim1 dimension for the current array with current fixed size Dim2 and Dim3 dimensions.
For 1 dimension Array: (maximum +ve integer - header size) / ElSize.
For 2 dimension Array: (maximum +ve integer - header size) / (ElSize * Dim2).
For 3 dimension Array: (maximum +ve integer - header size) / (ElSize * Dim2 * Dim3).


Also this

RT_ArrayGetDim(string FileName,int "Dim"=0)
Returns number of dimensions in array or the size of any 1 of the dimensions.
Filename, Array Filename, no default.
Dim, default 0. (Range 0 -> 3).
0, returns the number of dimensions in the array.
1->3, returns size of that dimension.
Invalid dimensions will return 0, eg RT_ArrayGetDim(FileName,Dim=2) on single dimension array returns 0.
Error if Dim less than than 0 or greater than 3.

[martin53]

Do you have multiple avisynth.dll's on your system?

Thanks for your attention also, Wilbert! No, only one AviSynth install.

But I got the guilty.:devil:
It is LoadPluginEx.dll from 11-03-2003, size 60KB, part of the warpsharppackage_25_dll_20031103, this package is in my autoload plugins folder. I'll see if I'll have problems when I move the dll to a quarantine folder.

Again - that was really quick how you all responded and gave good hints!:thanks:

[martin53]

Dont know if anybody (but me) actually uses the ARRAY functions

My 2013 AWB script heavily uses arrays, but not RT_ArrayGetNels(). It also uses RT_LumaSceneChange(), but not with the red arguments. :)

[StainlessS]

@M53, I guess this has probably been suggested already, but Groucho2004 Avisynth Info Tool Avisynth Info Tool
is pretty good, if it points out any C plugins, would be best to move to separate directories (other than autoload plug), and separate for
C v2.0 and later C plugins as they both have differing load function requirements.

Ah good, somebody does use those ARRAY funcs, I shall probably do ARRAY utility funcs to read in frame to ARRAY and write to frame from ARRAY,
and probably histogram reading funcs.
Removal of the Bias etc args from those other funcs would allow me to not consider them dead functions, and I could now perhaps improve.
Thanx for you reply.

[martin53]

StainlessS,
I recently mulled over StrReplace.
Today, there are two algorithms. One locates the find substring in the input, takes the left part until the found position from the input string plus concatenated replacement string as preliminary output, and continues search after the position plus length of the find string.
The alternative, known as StrReplaceDeep, does not put the replacement string to the preliminary output without ever searching it, too, but puts it left to the next recursion's input string.
Both suffer from specific drawbacks, which can make them unexpectedly unsuitable. #1 does not recursively replace("......", "..", ".") to "." since find matches that start inside the replacement string are not found. #2 becomes a black hole if the replacement string contains the full find string.

The best method to cover both drawbacks is, to my opinion, to use 'as much as possible' of the replacement string for the next find recursion. 'As much as possible' can easily be identified as 'at least one character of the original input string after the find string must be included', i.e. only the right part of the replacement string which is 1 char shorter than the find string, plus the remaining input string give the next input string. The left part of the replacement string which needed to be cut off is directly concatenated to the preliminary output string.
Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Original:- http://forum.doom9.org/showthread.php?t=147846 By Vampiredom, Gavino, IanB
{k=StrLen(replace)-min(StrLen(replace),StrLen(find)-1) i=s.FindStr(find) return(i==0?s:s.LeftStr(i-1)+replace.LeftStr(k)+(replace.MidStr(k+1)+s.MidStr(Strlen(find)+i)).StrReplace(find,replace))}
My suggestion is that this implementation replaces both RT_StrReplace and RT_StrReplaceDeep.

[martin53]

The embedded script implements part of AvsInfoTool with the help of the RT_InternalFunctions(), RT_PluginFunctions() and RT_FSelSaveAs() functions.

It also calls each function to retrieve the type of the return value.
Some functions make the system crash or hang - an indication that they are not tolerant against unreasonable input values. A list of such functions, and a prepared return value instead of the call helps against that.

The intention is to show the functionality of the RT_Stats plugin and to obtain a list of all functions you have on your system. Maybe you didn't know one or the other.
Try {RT_Version()} catch (msg) {assert(false,"This script requires the RT_Stats plugin.")}
Try {GEval("")} catch (msg) {assert(false,"This script requires the GScript plugin.")}

#============================================================================
Filename=RT_FSelSaveAs(title="Save Filter list",dir="",filt="Text files|*.txt",fn="_AllFunctions.txt",debug=true)
Assert(FileName.IsString(), "Please provide a file name to save the filter list")

#============================================================================
RT_WriteFile(Filename, VersionString(), Append=false)
WriteInternalFunctionsSyntax(RT_InternalFunctions(), Filename)
WritePluginFunctionsSyntax(RT_PluginFunctions(), Filename)
BlankClip().subtitle("Function info is complete.",align=5)

#============================================================================
function WriteInternalFunctionsSyntax(string fu_list, string Filename) {
GScript("""
done=""
while(fu_list != "") {
i = FindStr(fu_list+" ", " ")
fu_name = LeftStr(fu_list, i-1)
fu_list = MidStr(fu_list, i+1)
params = RT_PluginParam(fu_name)
if (FindStr(done, fu_name+params)==0) {
done = done + " "+fu_name+params
RT_WriteFile(Filename, "internal %s ( %s ) -> %s", fu_name, params, GetReturnType(fu_name, params), Append=true)
}
}
""")
}
#============================================================================
function WritePluginFunctionsSyntax(string fu_list, string Filename) {
GScript("""
done=""
while(fu_list != "") {
i = FindStr(fu_list+" ", " ")
fu_name = LeftStr(fu_list, i-1)
fu_list = MidStr(fu_list, i+1)
i = FindStr(fu_name, "_")
if (i>0) {
params = RT_PluginParam(fu_name)
if (FindStr(fu_list, "_"+fu_name)==0 && FindStr(done, "_"+fu_name+params)==0) {
done = done + " _"+fu_name+params
RT_WriteFile(Filename, "plugin %s ( %s ) -> %s", fu_name, params, GetReturnType(fu_name, params), Append=true)
}
}
}
""")
}
#============================================================================
#function GetReturnType(string fu, string params) { return "?" }
#============================================================================
function GetReturnType(string fu, string params) {
GScript("""
RT_DebugF("Fu=%s %s", fu, params, name="GetReturnType")
if (MidStr(fu,3,1)=="+") {fu = LeftStr(fu,2)+MidStr(fu,2)}
if (MidStr(fu,2,1)=="+") {fu = LeftStr(fu,1)+fu}
else if (fu=="Assert") {return "conditional halt"}
else if (FindStr(fu, "avss_26_DSS2")>0) {return "clip"}
else if (FindStr(fu, "ImageSeq_")>0) {return "clip"}
else if (FindStr(fu, "LoadCPlugin")>0) {return "void"}
else if (FindStr(fu, "LoadVirtualdubPlugin")>0) {return "void"}
else if (FindStr(fu, "MDepan")>0) {return "clip"}
else if (FindStr(fu, "MSCDetection")>0) {return "clip"}
else if (FindStr(fu, "ResampleAudio")>0) {return "clip"}
else if (FindStr(fu, "Resize")>0) {return "clip"}
else if (FindStr(fu, "RT_FSel")>0) {return ". "}
else if (FindStr(fu, "Source")>0) {return "clip"}
else {
bc = BlankClip(width=8,height=8,length=1,pixel_type="YV12")
fp = LeftStr(params,2)=="cc" ? "(bc,bc)"
\ : LeftStr(params,2)=="cf" ? "(bc,0.1)"
\ : LeftStr(params,3)=="cci" ? "(bc,bc,0)"
\ : LeftStr(params,5)=="ciiii" ? "(bc,0,0,0,0)"
\ : LeftStr(params,4)=="ciii" ? "(bc,0,0,0)"
\ : LeftStr(params,3)=="cii" ? "(bc,0,0)"
\ : LeftStr(params,2)=="ci" ? "(bc,0)"
\ : LeftStr(params,1)=="c" ? "(bc)"
\ : LeftStr(params,2)=="ff" ? "(0.1,0.1)"
\ : LeftStr(params,1)=="f" ? "(0.1)"
\ : LeftStr(params,2)=="ii" ? "(0,0)"
\ : LeftStr(params,1)=="i" ? "(0)"
\ : LeftStr(params,2)=="ss" ? "("+chr(34)+"1"+chr(34)+","+chr(34)+"1"+chr(34)+")"
\ : LeftStr(params,2)=="si" ? "("+chr(34)+"1"+chr(34)+",1)"
\ : LeftStr(params,1)=="s" ? "("+chr(34)+"1"+chr(34)+")"
\ : "()"
Try {
r = Eval("current_frame=0 "+fu+fp)
r = r.IsBool ? "bool" : r.IsClip ? "clip" : r.IsInt ? "int" : r.IsFloat ? "float" : r.IsString ? "string" : "?"}
catch (msg) { Try {r = r.IsBool ? "bool" : r.IsClip ? "clip" : r.IsInt ? "int" : r.IsFloat ? "float" : r.IsString ? "string" : "?" } catch(msg) {r = "?"}
}
if (r=="?") {
bc = BlankClip(width=8,height=8,length=1,pixel_type="RGB32")
fp = LeftStr(params,2)=="cc" ? "(bc,bc)"
\ : LeftStr(params,2)=="cf" ? "(bc,0.1)"
\ : LeftStr(params,3)=="cci" ? "(bc,bc,0)"
\ : LeftStr(params,5)=="ciiii" ? "(bc,0,0,-0,-0)"
\ : LeftStr(params,4)=="ciii" ? "(bc,0,0,0)"
\ : LeftStr(params,3)=="cii" ? "(bc,0,0)"
\ : LeftStr(params,2)=="ci" ? "(bc,0)"
\ : "(bc)"
Try {
r = Eval("current_frame=0 "+fu+fp)
r = r.IsBool ? "bool" : r.IsClip ? "clip" : r.IsInt ? "int " : r.IsFloat ? "float" : r.IsString ? "string" : "?"
} catch (msg) {
r = "?"
}
}
if (r=="?" && LeftStr(params,1)=="c") {r="clip?"}
return r
}
""")
}

[Groucho2004]

The attached script implements part of AvsInfoTool with the help of the RT_InternalFunctions(), RT_PluginFunctions() and RT_FSelSaveAs() functions.
Don't attach stuff here, it could take days to be approved. Post it to pastebin or similar.

[martin53]

Don't attach stuff here, it could take days to be approved. Post it to pastebin or similar.

Hmm, my opinion: better days to be approved than deleted in a few months. I embedded it directly now.

[creaothceann]

You can specify on pastebin that it should never be deleted.

[StainlessS]

M53,
There is requirement for both, deep replacement is rarely needed and would yield unexpected results on occasion.
There is no 'black hole' as you suggest.
(NOTE, the link given in your version [removed here] as original source of StrReplace is actually a link to GScript thread.)

I did not implement RT_StrReplaceDeep, exactly as in script version, it does if you like, 'super deep' replacement, ie rescans even prior to
replacements, perhaps it should not, I will reconsider there.

Note, snip below (excluding M53 version) implements bug fixes for the v2.58 string concatenation bug.


# test 1
SOUR = "......"
FND = ".."
REP = "."
# result
# '......' # SOUR
# '...' # ORG_StrReplace
# '...' # RT_StrReplace
# '.' # M53_StrReplace
# '.' # StrReplaceDeep
# '.' # RT_StrReplaceDeep

# test 2
#SOUR = ".... .. ...."
#FND = ".. "
#REP = " ."
# result
# '.... .. ....' # SOUR
# '.. . .....' # ORG_StrReplace
# '.. . .....' # RT_StrReplace
# '.. . .....' # M53_StrReplace
# '.. . .....' # StrReplaceDeep
# ' ......' # RT_StrReplaceDeep, As implemented, rescans whole string (even prior to current replaced)

# test 3, 'Black hole does not exist !
SOUR = ".... .. ...."
FND = ".."
REP = "...."
# result
# '.... .. ....' # SOUR
# '........ .... ........' # ORG_StrReplace
# '........ .... ........' # RT_StrReplace
# '.......... .... ..........' # M53_StrReplace : whats with this ???
# '........ .... ........' # StrReplaceDeep
# '........ .... ........' # RT_StrReplaceDeep (only deep if Replace string shorter than Find string).

# test 4, Original requirement test (compact SPACE, ie ALL ' ' to ' ')
SOUR = "[10] 9 -0.13 0.16 0.004 0.99927"
FND = " "
REP = " "
# result
# '[10] 9 -0.13 0.16 0.004 0.99927' # SOUR
# '[10] 9 -0.13 0.16 0.004 0.99927' # ORG_StrReplace
# '[10] 9 -0.13 0.16 0.004 0.99927' # RT_StrReplace
# '[10] 9 -0.13 0.16 0.004 0.99927' # M53_StrReplace
# '[10] 9 -0.13 0.16 0.004 0.99927' # StrReplaceDeep
# '[10] 9 -0.13 0.16 0.004 0.99927' # RT_StrReplaceDeep (only deep if Replace string shorter than Find string).


S = ORG_StrReplace (SOUR,FND,REP)
SS = RT_StrReplace (SOUR,FND,REP)
SSS = M53_StrReplace (SOUR,FND,REP)
SSSS = StrReplaceDeep (SOUR,FND,REP)
SSSSS = RT_StrReplaceDeep(SOUR,FND,REP)

blankclip

return RT_Subtitle("'%s'\n'%s'\n'%s'\n'%s'\n'%s'\n'%s'",SOUR,S,SS,SSS,SSSS,SSSSS)


# Note, recursively calls ORG_StrReplace not StrReplace
function ORG_StrReplace(string s, string find, string replace) {
# # Original:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 : By Vampiredom, Gavino, IanB
# i = s.FindStr(find)
# return (i == 0 ? s : s.LeftStr(i-1)+replace+s.MidStr(Strlen(find)+i).StrReplace(find, replace))
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
i=s.FindStr(find)
return(i==0?s:RT_StrAddStr(s.LeftStr(i-1),replace,s.MidStr(Strlen(find)+i).ORG_StrReplace(find,replace)))
}

# Note, recursively calls M53_StrReplace not StrReplace
Function M53_StrReplace(string s,string find,string replace) {
k=StrLen(replace)-min(StrLen(replace),StrLen(find)-1)
i=s.FindStr(find)
return(i==0?s:s.LeftStr(i-1)+replace.LeftStr(k)+(replace.MidStr(k+1)+s.MidStr(Strlen(find)+i)).M53_StrReplace(find,replace))
}

# Note, recursively calls ORG_StrReplace not StrReplace
Function StrReplaceDeep(string s,string find,string replace) # DEEP Repeated, string replacements
# When replacing eg pairs of SPACE characters by single SPACE, StrReplace can leave some pairs non replaced.
# Assumes that you want to rescan where replaced string could constitute part of a new find substring,
# Length of replace must be shorter than length of find, else will call StrReplace instead.
# Based On:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
# {i=s.FindStr(find)return i==0?s:s.LeftStr(i-1)+( strlen(replace)<strlen(find)? \
# StrReplaceDeep(replace+s.MidStr(Strlen(find)+i),find,replace) : replace+StrReplace(s.MidStr(Strlen(find)+i),find,replace))}
# Converted to use RT_Stats RT_StrAddStr() to avoid 2.58/2.6a3 string concatenation bug:-
{
i=s.FindStr(find)
return i==0?s:RT_StrAddStr(s.LeftStr(i-1),( strlen(replace)<strlen(find)
\ ? StrReplaceDeep(RT_StrAddStr(replace,s.MidStr(Strlen(find)+i)),find,replace)
\ : RT_StrAddStr(replace,ORG_StrReplace(s.MidStr(Strlen(find)+i),find,replace))))
}




EDIT: Cocked up the 'black hole test', EDITED.

EDIT: Note, some functions (including some of mine) return variable result types, eg -ve int to denote error, or float result.
EDIT: Or eg DBase funcs return any type.

EDIT: Original post of StrReplaceDeep:- http://forum.doom9.org/showthread.php?p=1596053#post1596053

[martin53]

(NOTE, the link given in your version [removed] as original source of StrReplace is actually a link to GScript thread.)
Umm... maybe once did an overshoot replace in my scripts collection, will immediately correct that.
# test 3, 'Black hole does not exist !
SOUR = ".... .. ...."
FND = ".."
REP = "...."
# result
# '........ .... ........' # ORG_StrReplace
# '........ .... ........' # RT_StrReplace
# '.......... .... ..........' # M53_StrReplace : whats with this ???
# '........ .... ........' # StrReplaceDeep
# '........ .... ........' # RT_StrReplaceDeep (only deep if Replace string shorter than Find string).
OK, my apologies that I didn't experiment with (RT_-)StrReplaceDeep, but just figured out the algorithm theoretically, and did not take into account the black hole preventer switch that checks if replace is shorter than find. My version does not have that yet, maybe it was introduced after I copied it to my script base.
I thought of that switch, too, but I consider it less reasonable than the new proposal, e.g. with find=".." and replace="blahblah." Also I judge the above M53_StrReplace result OK, since 'you get what you asked for' with computers, not 'what you had in mind to ask for' or whatever. But that is biased for sure. It's a suggestion, I don't insist of course. In my scripts, I remove e.g. repeated empty lines when I open a playlist, thus I needed StrReplaceDeep; but I hate to distinguish between two functions and so use 'M53_StrReplace', as you call it, all over my script base. I'll tell when I'll ever be dissatisfied with it :)

EDIT: Black hole occurs if find string is in replace string? I.e.
Function StrReplaceDeep(string s,string find,string replace)
...
{
i=s.FindStr(find)
return i==0?s:RT_StrAddStr(s.LeftStr(i-1),( strlen(replace)<strlen(find) replace.FindStr(find) == 0
\ ? StrReplaceDeep(RT_StrAddStr(s.LeftStr(i-1),replace,s.MidStr(Strlen(find)+i)),find,replace)
\ : RT_StrAddStr(replace,ORG_StrReplace(s.MidStr(Strlen(find)+i),find,replace)))) # or M53_StrReplace() here
}
green part replaces red part, blue part added because StrReplaceDeep(">>>>><", "><", "<") = ???

[StainlessS]

Post #339, added test 4, and added SOUR to result comments.

As M53_StrReplace complies with original intent of StrReplaceDeep (SPACE compacting), will modify to use your algo (for RT_StrReplaceDeep).

[martin53]

Just to satisfy possible interest in solution variants, and as a reference to understand the concepts, my personal - maybe stable - current algo
#=====================================================================================================================
Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Based On:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
{
# use RT_Stats RT_StrAddStr() if present to avoid 2.58/2.6a3 string concatenation bug
function StrAddStr(a,b,c){Try{return RT_StrAddStr(a,b,c)}catch(msg){return a+b+c}}
i=s.FindStr(find) f=StrLen(find) r=StrLen(replace) k=r-min(r,f-1)
return i==0?s:replace.FindStr(find)==0
\ ? StrReplace(StrAddStr(s.LeftStr(i-1),replace,s.MidStr(i+f)),find,replace)
\ : StrAddStr(s.LeftStr(i-1),replace.LeftStr(k),StrReplace(replace.MidStr(k+1)+s.MidStr(i+f),find,replace))
}
#=====================================================================================================================
Function StrReplaceDeep(string s,string find,string replace) # Repeated, string replacements
{return StrReplace(s,find,replace)}
#=====================================================================================================================
with
StrReplace(">>>>>><",">><","><<") == "><<<<<<" (recursive call with later replacements more left in string than previous replacements)
StrReplace(">......",">." ,">.>") == ">.>.>.>.>.>.>" (recursive call with find string in replace string, replace string longer than find string, no black hole)
StrReplace("[10]______9___-0.13____0.16___0.004_0.99927","__","_") == "[10]_9_-0.13_0.16_0.004_0.99927" (original intent)

[StainlessS]

Thank you M53, bit busy at moment, shall revisit later.

It also uses RT_LumaSceneChange(), but not with the red arguments.

Have now deleted both RT_LumaSceneChange() and RT_LumaMovement() from plug, RT_LumaSceneChange without red args is identical to RT_LumaDifference
and RT_LumaMovement replacing with RT_FrameMovement. Args something like this, (not finalized).


RT_LumaDifference, "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i" # No difference

RT_FrameDifference,"cc[n]i[n2]i[ChromaWeight]f[x]i[y]i[w]i[h]i[x2]i[y2]i[altscan]b" # Additional args after Chromaweight

RT_FrameMovement, "cc[n]i[n2]i[x]i[y]i[w]i[h]i[x2]i[y2]i[altscan]b[matrix]i[Chromaweight]f[blkw]i[blkh]i[Mode]i"
# Mode selects LumaDiff or FrameDiff, perhaps later additional modes.



RT_FrameDifference will implement 'Altscan' (elsewhere uses 'Interlaced' [bad choice of arg name], to mean 'every other line').
May at some point implement eg 'ChromaI' bool used where YV12 interlaced chroma (gets more complicated when mixed with Altscan
and two comparison frames). Also, will 'auto' detect if both gray frames (perhaps 'DetectGray' average pixel difference [EDIT: from gray] float arg, currently 1.0) and switch to ChromaWeight=0.0 if both frames grayscale. (EDIT: For color clips with grayscale scene, eg "A Matter of Life and Death", 'Sin City' might be tricky, best treat as Grayscale all through ie ChromaWeight=0.0)

DBase limit extended to lesser of available disk space, or 1TB-1MB max size or $7FFFFFFE records.
Array to 1TB-1MB, or $7FFFFFFE max dim 1 for 1 dim array. For 2 dim array, total size is 256MB for dim 2, or for 3 dim array is 256MB for elements in
dimensions 2 and 3.

Any of the supplied AVS scipts in zip will be updated to use the new plug versions.
If I have to remove a few funcs, then I might as well do all script breaking changes in one go.

EDIT:
Some additional plugz or modz:

env->AddFunction("RT_ForceProcess", "c[Startframe]i[EndFrame]i", RT_ForceProcess, 0);
env->AddFunction("RT_DBaseGetTypeString", "s",RT_DBaseGetTypeString, 0);
env->AddFunction("RT_DBaseSetID", "si.+",RT_DBaseSetID , 0);
env->AddFunction("RT_DBaseCheckID", "si.+",RT_DBaseCheckID, 0);
env->AddFunction("RT_ArraySetID", "si.+",RT_ArraySetID , 0);
env->AddFunction("RT_ArrayCheckID", "si.+",RT_ArrayCheckID, 0);


RT_ForceProcess scans clip (possibly extended to audio too at some future point, If I can get TWriteAVI to support audio out).
RT_DBaseGetTypeString, used to duplicate a DBase with identical fields.
RT_DBaseSet/CheckID (and array ver$), can set or check multiple DBase/Array ID's in one call.
Number of DB/ARR ID's raised from 16 to 128.

EDIT: On 2nd thoughts, think I limited DBase/Arr to lesser of Available_DiskSpace-1MB or 1TB, and not 1TB-1MB.

EDIT: "A Matter of Life and Death" (http://en.wikipedia.org/wiki/A_Matter_of_Life_and_Death_%28film%29)
originally released in USA as "Stairway to Heaven".

[StainlessS]

Yo dudes.
Anybody ever heard of a plugin to determine if a clip/range is Grayscale ? (did search on IsGray/IsGrayScale and variant spellings).
I'm gonna implement (well basically have already), with x,y,w,h frame coords, and alternate raster line scanning.
What I've done works on Planar YUY2 and RGB, and should be very fast (or slower if you demand full scan).
Could set Local var with eg ISGrayScale bool, and/or metric on Grayness.
Should be very fast on colour ranges where Only IsGrayScale bool result is required, would exit scan when threshold number of pixels exceeded,
could test after every scanline and so avoid full frame scan when determined to be color.
For YUV, % threshold number of pixels varying from $xx8080.
For RGB, % threshold number of pixels varying from Min(R,G,B) == Max(R,G,B).
This will be implemented as runtime Function in RT, any interest in a separate standard plugin ? (does one already exist ?)

EDIT: Actually IsColor() would maybe more appropriate as above % threshold of pixels varying from Gray could return True (in RT runtime or as local var in filter plug) or set metric if required where % of pixels not gray could be rendered on frame and/or set as local var (slower if render or set local scan full metric). IsColor() does not show in search either.

EDIT: Got me a yodelling fox outside right now, 03:22 AM :)
EDIT: A couple of days ago a fox casually sauntered past me about 5 feet away, did not even bother looking at me,
they are getting really cheeky nowadays in London.
EDIT: And the escaped green parakeets are all over the place for the past couple of years. (group escape about 30 years ago
in Richmond Park).

EDIT: Modded IsColor() type func to count differences from Gray more like so.
Let Cdf = allowed maximum difference from Gray that will NOT be counted (default 0),
Let CTh = % Threshold to calculate number of differences that are more than CDf away from Gray.
PixLim=(xdim*yDim) * CTh / 100.0 * 2 [two counts, for U+V for YUV, and Max(R,G,B)-Mid(R,G,B) and Mid(R,G,B)-Min(R.G,B) for RGB]
YUV

if(Abs(u-128)>CDf) Then Cnt=Cnt+1
if(Abs(v-128)>CDf) Then Cnt=Cnt+1

... Later

return (Cnt > Pixlim) # If color return True


RGB

if(Max(R,G,B)-Mid(R,G,B)>CDf) Then Cnt = Cnt + 1
if(Mid(R,G,B)-Min(R,G,B)>CDf) Then Cnt = Cnt + 1

... Later

return (Cnt > Pixlim) # If color return True


EDIT: Or perhaps more like so (more precise control for CDf).
Let Cdf = allowed maximum difference from Gray that will NOT be counted (default 0),
Let CTh = % Threshold to calculate number of differences that are more than CDf away from Gray.
PixLim=(xdim*yDim) * CTh / 100.0

YUV

dif = Abs(u-128) + Abs(v-128)
if(dif > CDf) Then Cnt=Cnt+1

... Later

return (Cnt > Pixlim) # If color return True


RGB



dif = Max(R,G,B)-Mid(R,G,B) + (Mid(R,G,B)-Min(R,G,B)
if(dif > CDf) Then Cnt=Cnt+1

... Later

return (Cnt > Pixlim) # If color return True

[StainlessS]

@Martin53,

For your version StrReplace,

s=StrReplace("abc","","1")

ie replace "" with "1",
Causes silent crash due to stack exhaustion.
(Problem due to FindStr("abc","") returning result 1 [bit of a daft result, 0 not found would perhaps be more sensible]).
Should return source string s where StrLen(find)==0

[StainlessS]

Here, two updates to scripts that are currently supplied with RT_Stats.
Scripts are demo color detectors, added a marker to indicate detection location.
Next version RT will have indicator in the Security Camera Motion Detection script, to show where motion was detected.
Uses RectMarker function first posted here:- http://forum.doom9.org/showthread.php?p=1733583#post1733583

Luma Y Detector

#RT_YInRAngeLocate() Tester

OBJECT_W = 20 # Object to find, size in pixels
OBJECT_H = 40 # Object to find, size in pixels

AREA_X = 0 # Area of frame to search (0,0,0,0=full frame {As Crop})
AREA_Y = 0
AREA_W = 0
AREA_H = 0

LUMA_MIN = 235 # Luma minimum of object to find
LUMA_MAX = 255

# Fake input clip, for testing
a = BlankClip(length=481,width=480,height=480)
b = BlankClip (width=OBJECT_W, height=OBJECT_H, color=$FFFFFF)
a.Animate (0, a.FrameCount () - 1, "Overlay", b, 0, 0, b, 480, 480)

ConvertToYV12() # Any colorspace ok so long as LUMA_MIN/MAX OK (RGB -> Luma-Y)

MarkerColor = $FFFF00 # Detection marker color
MARKER_TYPE = False # False = Corners only marker, True=Full Rect Marker
Msk = Last.RectMarker(OBJECT_W+8,OBJECT_H+8,MARKER_TYPE)
Mrk = Msk.BlankClip(Color=MarkerColor)

DEBUG=false
NOTFOUND="NOT FOUND" # Last few frames should show not found, depends upon OBJECT_W/H (as partially disappears off frame).
sep = ", "
ket = "] "
COORDINATES="Y_Coordinates.txt"
RT_FileDelete(COORDINATES) # Delete coordinates file

ScriptClip ("""
Bingo=RT_YInRangeLocate(debug=DEBUG,Baffle_W=OBJECT_W,Baffle_H=OBJECT_H,x=AREA_X,y=AREA_Y,w=AREA_W,h=AREA_H,Lo=LUMA_MIN,hi=LUMA_MAX)
(Bingo) ? WriteFile (COORDINATES,"current_frame","KET", "YIRL_X", "sep", "YIRL_Y", "sep", "YIRL_W", "sep", "YIRL_H")
\ : WriteFile (COORDINATES,"current_frame","KET", "NOTFOUND")
(Bingo) ? Overlay(Mrk,x=YIRL_X-4,y=YIRL_Y-4,Mask=Msk,Mode="Blend",output="YV12")
\ : NOP
Return Last
""")

Return Last

Function RectMarker(clip c,int W, Int H,Bool "Hit") {
# req RT_Stats & mt_tools_2, Returns YV12 frame WxH same FPS as c and without audio
Hit=Default(hit,False)
InFix = (Hit)
\ ? RT_string("(((x<=1|x>=%d|y<=1|y>=%d)&(x<4|x>=%d)&(y<4|y>=%d)|(x==0|x==%d|y==0|y==%d)))?255:0", W-2,H-2,W-4,H-4,W-1,H-1)
\ : RT_string("((x<=1|x>=%d|y<=1|y>=%d)&(x<4|x>=%d)&(y<4|y>=%d))?255:0", W-2,H-2,W-4,H-4)
c.Blankclip(width=W,height=H,Length=1,pixel_type="YV12").Killaudio
return mt_lutspa(relative = false,yExpr=mt_Polish(InFix), chroma = "128")
}


RGB Detector

# RGBInRangeLocate_Test

RED =$AA # Object Color to look for
GREEN =$CC
BLUE =$FF

OBJECT_W = 10 # Object Width
OBJECT_H = 10 # Object Height

AREA_X = 0 # Area of frame to search (0,0,0,0=full frame, As Crop)
AREA_Y = 0
AREA_W = 0
AREA_H = 0

R_MIN = RED R_MAX = RED
G_MIN = GREEN G_MAX = GREEN
B_MIN = BLUE B_MAX = BLUE

# Overlay Converts RGB->YUV->RGB (can set inexact color). Use Layer Instead, MUST set ALPHA
RGBA=((255 * 256 + RED)*256+GREEN)*256+BLUE

COORDINATES="RGB_Coordinates.txt"

# Fake input clip, for testing
a = BlankClip(length=480,width=480,height=480)
b = a.BlankClip(width=OBJECT_W, height=OBJECT_H, color=RGBA)
a.Animate (0, a.FrameCount () - 1, "Layer", b,"add",257, 0, 0, b,"add",257, 479, 479)

MarkerColor = $FFFF00 # Detection marker color
MARKER_TYPE = False # False = Corners only marker, True=Full Rect Marker
Msk = Last.RectMarker(OBJECT_W+8,OBJECT_H+8,MARKER_TYPE)
Mrk = Msk.BlankClip(Color=MarkerColor)


NOTFOUND="NOT FOUND" # Will be NOT FOUND, As OBJECT moves out of frame at end of clip (can no longer find size OBJECT_W x OBJECT_H)
sep = ", "
Ket = "] "
RT_FileDelete(COORDINATES) # Delete coordinates file
DEBUG=False
ScriptClip ("""
Bingo=RT_RgbInRangeLocate(x=AREA_X,y=AREA_Y,w=AREA_W,h=AREA_H,
\ RLo=R_MIN,Rhi=R_MAX,GLo=G_MIN,Ghi=G_MAX,BLo=B_MIN,Bhi=B_MAX,Baffle_W=OBJECT_W,Baffle_H=OBJECT_H,debug=DEBUG)
(Bingo) ? WriteFile (COORDINATES,"current_frame", "Ket", "RGBIRL_X", "sep", "RGBIRL_Y")
\ : WriteFile (COORDINATES,"current_frame", "Ket", "NOTFOUND")

(Bingo) ? Overlay(Mrk,x=RGBIRL_X-4,y=RGBIRL_Y-4,Mask=Msk,Mode="Blend")
\ : NOP
Return Last
""")

Return Last

Function RectMarker(clip c,int W, Int H,Bool "Hit") {
# req RT_Stats & mt_tools_2, Returns YV12 frame WxH same FPS as c and without audio
Hit=Default(hit,False)
InFix = (Hit)
\ ? RT_string("(((x<=1|x>=%d|y<=1|y>=%d)&(x<4|x>=%d)&(y<4|y>=%d)|(x==0|x==%d|y==0|y==%d)))?255:0", W-2,H-2,W-4,H-4,W-1,H-1)
\ : RT_string("((x<=1|x>=%d|y<=1|y>=%d)&(x<4|x>=%d)&(y<4|y>=%d))?255:0", W-2,H-2,W-4,H-4)
c.Blankclip(width=W,height=H,Length=1,pixel_type="YV12").Killaudio
return mt_lutspa(relative = false,yExpr=mt_Polish(InFix), chroma = "128")
}

[jmac698]

Just to let you know I need a 16bit version of planetcrop. I'm happy to report that imagesource reads 16bit PNG fine. It's chugging away at a set now. I'm going to have to find a way to split the 16bits into high and low parts and a way to encode it back into 16 bit pngs. I was thinking some incantations of imagemagick might do this. I could do it as video also but I don't know if there's any codecs that handle 16bit. In fact I think my images are really 12 bit though.

I'm working on the mod myself but feel free to help if you want a challenge :)

ps
About that statistics analysis - that's backed up as I write a CHDK script to aquire all the data. I have to do some testing to see if setting out of the ordinary ISO ranges actually work on the camera. There's also some weirdness about ISO not being true ISO but inflated market values.

ps2
it took a while but processing finished, perfect alignment at defaults gj :)

and here's some frames to play with
http://wikisend.com/download/839422/MoonHDR.7z

[StainlessS]

Hi Jmac, I see you there.
Have DL your clip and will have a play.

[StainlessS]

JM, can you post what you got so far so I can take a peek, thanx.

EDIT: I'm guess'in that 'most significant BIT' alone would be sufficient for detection.
(more than 8 would be overkill)

[jmac698]

Yes my thoughts exactly, though theoretically it could be a dim image, practically I would never expose a planet so dimly. It could be dim mind you, if I use a low ISO (which means less initial noise), but not THAT dim.

So the idea would be, detect on upper 8 bits, apply the crop to high and low images, then output stacked. Should be simple. Then imagemagick to stack/unstack. I do have a question though, what does autolevel=1 do? I was expecting to make the brightest pixel = 255. Speaking of which, that feature may take some work to work in 16bit.

Btw this pre-aligning *really* helps. The stacking programs I have either crash or take forever to align badly aligned images. But I couldn't stack the images I have; one remaining problem is rotation, I need to de-rotate what I have. I'm not using the "drifting across view" method any more, just handheld, so the alignment is a bit messier. I don't think there's any rotation detection plugins existing for Avisynth.

ps however, if I convert from polar to cartesian and make a "map" of the planet, then it's a simple shift finding to find the rotation angle, that could be simple.

update:
incantation is (create stacked high/low from 16bit PNG)

convert IMG_7853.png ( +clone -evaluate and 255 -evaluate multiply 256 ) -evaluate and 65280 -append -depth 8 test.png

[martin53]

@Martin53,

For your version StrReplace,

s=StrReplace("abc","","1")

ie replace "" with "1",
Causes silent crash due to stack exhaustion.
(Problem due to FindStr("abc","") returning result 1 [bit of a daft result, 0 not found would perhaps be more sensible]).
Should return source string s where StrLen(find)==0

Nice find! (OK, bad pun)
For me, the blue supplement is satisfying, may everybody decide for himself.

Function StrReplace(string s,string find,string replace) # Repeated, string replacements
# Based On:- http://forum.doom9.org/showthread.php?p=1540504#post1540504 By Vampiredom, Gavino, IanB
{
# use RT_Stats RT_StrAddStr() if present to avoid 2.58/2.6a3 string concatenation bug
function StrAddStr(a,b,c){Try{return RT_StrAddStr(a,b,c)}catch(msg){return a+b+c}}
find=(find==""?s:find)
i=s.FindStr(find) f=StrLen(find) r=StrLen(replace) k=r-min(r,f-1)
return i==0?s:replace.FindStr(find)==0
\ ? StrReplace(StrAddStr(s.LeftStr(i-1),replace,s.MidStr(i+f)),find,replace)
\ : StrAddStr(s.LeftStr(i-1),replace.LeftStr(k),StrReplace(replace.MidStr(k+1)+s.MidStr(i+f),find,replace))
}

[StainlessS]

OK,

#M53
s=StrReplace("abc","","1") # Returns '1'

#RT
s=RT_StrReplace("abc","","1") # Returns 'abc'


Hi M53, I thought that you had been eaten by crocodiles again.

(Clarification, I though it again, NOT you had been eaten by crocodiles for a second time, that would have been very unlucky)


EDIT: Here, 1st scene is Martin53 on Safari (as always, camera in-hand):- https://www.youtube.com/watch?v=ndzOp777_UE

[martin53]

StainlessS, have you got a bit of spare time?
These extensions to RT_Stats could be useful in scripts:

* RT_FloatAsRGB32() taking an arbitrary parameter count and generating not a 1x1 clip, but a 1x(parameter count) clip.

Possible use: AVS functions could return vectors of float values, packed as a clip.

* RT_IntAsRGB32() doing about the same as RT_FloatAsRGB32()

Possible use: improved readability and less complicated clip graph than
StackHorizontal(Blankclip(width=1,height=1,pixel_type="RGB32",color=myint1),Blankclip(width=1,height=1,pixel_type="RGB32",color=myint2),...,...)

* A new stats function that takes one clip (and another one for the stats comparing two clips) (i.e. all the stuff under 'Runtime Clip Comparison Functions' in the RT_Stats manual), and is called outside RTE and does not return one float, but a RGB32 clip with the float stats values vector of all the results, horizontally stacked.

Possible use: Inside the plugins, the various values like max, min, average etc. can easily be evaluated together, without much speed penalty. The clip profits from the AviSynth frame cache mechanism and it becomes possible to obtain these cached statistic values much faster from RTE scripts with RT_RGB32AsFloat(). I have an RTE script here that checks, say, 50 frames backward and forward for each frame (it searches for the one with the most edges i.e. AverageLuma of an edge function). If these values were cached, I expect a significant speed improvement, especially for HD footage.

[StainlessS]

StainlessS, have you got a bit of spare time?
These extensions to RT_Stats could be useful in scripts:

* RT_FloatAsRGB32() taking an arbitrary parameter count and generating not a 1x1 clip, but a 1x(parameter count) clip.

Possible use: AVS functions could return vectors of float values, packed as a clip.

* RT_IntAsRGB32() doing about the same as RT_FloatAsRGB32()

Possible use: improved readability and less complicated clip graph than
StackHorizontal(Blankclip(width=1,height=1,pixel_type="RGB32",color=myint1),Blankclip(width=1,height=1,pixel_type="RGB32",color=myint2),...,...)

* A new stats function that takes one clip (and another one for the stats comparing two clips) (i.e. all the stuff under 'Runtime Clip Comparison Functions' in the RT_Stats manual), and is called outside RTE and does not return one float, but a RGB32 clip with the float stats values vector of all the results, horizontally stacked.

Possible use: Inside the plugins, the various values like max, min, average etc. can easily be evaluated together, without much speed penalty. The clip profits from the AviSynth frame cache mechanism and it becomes possible to obtain these cached statistic values much faster from RTE scripts with RT_RGB32AsFloat(). I have an RTE script here that checks, say, 50 frames backward and forward for each frame (it searches for the one with the most edges i.e. AverageLuma of an edge function). If these values were cached, I expect a significant speed improvement, especially for HD footage.

I'll see what I can do.

[martin53]

OK,
Hi M53, I thought that you had been eaten by crocodiles again.

EDIT: Here, 1st scene is Martin53 on Safari (as always, camera in-hand):- https://www.youtube.com/watch?v=ndzOp777_UE

:D
No, I just am in the (early) progress of digitizing a fair number of old books etc, so i'm not bored
and some months ago I had decided to switch over to Avisynth+, but I felt it hadn't settled everything, so decided to wait for a while. Things changed not towards simplicity since there are these different MT approaches and all that kinda stuff that thwarts you all the time when you'd like to concentrate on the video algorithms.

[StainlessS]

M,
Have you considered trying the DBase functions for your previously suggested funcs ?
You could eg create a DBase with same number of records as a clip, and maybe a bool as first field to indicate whether
other fields are valid, and have a function that fills remaining DBase fields with Ymin, YMax AverageLuma etc. (int and float values) from RT_YStats.
You could even do a prescan to set DB once, for multipass use, or so that settings could be tweaked without the creating DB every time.
RT_DBase read/write is pretty fast, about 265 MicroSecond to set a Float and about 165 Microseconds to read back, just to access
Avisynth 'True' or 'False' constants takes about 500 microseconds on my 2.4Ghz core duo.

EDIT: Just doing away with eg StackHorizontal is not gonna make anything a great deal faster, I think.
EDIT: A plug function that fills DBase record with YStats results might be best bet.

[martin53]

M,
Have you considered trying the DBase functions for your previously suggested funcs ?

I did, though I didn't benchmark and am impressed about the DB speed! I am a bit fond of the script simplicity I imagine with the clip approach; it seems kinda elegant to me since dealing with clips is the Avisynth competence. Also, it deals flawlessly with multi instances.

I agree that the DB funcs are very capable. One could write a RTE function with GScript that prescans the clip when called the first time - but you remember: string processing and GScript in RTE is a don't (I learned when I refactorized SRestore). So I think I'd use an additional boolean flag field to reflect if the frame is already analysed. If not, the RTE analysing function would derive the values just for current_frame and set the flag (quite similar to clip frame fetching). - All that is possible in AviSynth language today. But I feel it creates fat AVS files, is tricky to write as generic script snippet for multi-instances, and harder to understand/review/maintain than it should be.

[StainlessS]

Hi guys, just a (sitrep and) heads up warning of future change in RT_Stats plugin v2.0.

RT_QwikScan current version is a DBase utility function, v2.0 will be changed to Array utility function.
Original RT_QwikScan was implemented with mixture of Int and Float in frame FingerPrint, but changed later to soley Float.
I wanted to increase the number of fingerprint component values from 4 to about 16, but that would require 16*256 (instead of 4*256)
fields in Locator DBase, making the use of DBase very unwieldy with a 4KB Type string for RT_DBaseAlloc function.
Changing to Array utility greatly simplifies the creation of the FingerPrint and Locator files. (Some info here:- http://forum.doom9.org/showthread.php?p=1693002#post1693002)

Increasing number of FingerPrint components reduces the time taken with higher values of LumaTol to more acceptable durations.
Below are some timings that were first done in post linked a few lines above, current timings done with same JurassicPark DVD clip,
converted to UT_Video Codec AVI file, original tests were probably done on RoboCropped clip, current one still has a little letterboxing
top and bottom (AR=1.85 I think). Also, CPU, Hard drive both different, and code probably changed too for the current RT_stats (as opposed to
v2.0 RT_Stats).

Here, new numbers (Search for 10, frames, 3000->13000 in 1000 frame steps, at multiple values for LumaTol).

Test Script for both OLD v1.45 and Upcoming RT_Stats v2.0

AVISource("JurassicPark.avi")

RTVER2 = (RT_Version > 1.99) # New Version RT_Stats ?

EXT = (RTVER2) ? ".ARR" : ".DB"

FINGER = "JPARK" + EXT
PREV = "JPARK_PREV" + EXT # Prev LOCATOR, ie BackWards Search.
NEXT = "JPARK_NEXT" + EXT # Next LOCATOR, ie ForWards Search (not used in this script).

RT_FileDelete(FINGER) RT_FileDelete(PREV) RT_FileDelete(NEXT) # Uncomment to Create QwikScan DBs/Arrays, comment out to Re-Run

(Exist(FINGER)) ? NOP : RT_QwikScanCreate(Last, FINGER , PREV,"", debug=true) # We dont need NEXT Locator for this backwards scan test.

START=RT_TimerHP()
QwikScanTest(Last,FINGER,PREV,0.0) # LUMATOL = 0.0
QwikScanTest(Last,FINGER,PREV,4.0) # LUMATOL = 4.0
QwikScanTest(Last,FINGER,PREV,8.0) # LUMATOL = 8.0
QwikScanTest(Last,FINGER,PREV,12.0) # LUMATOL = 12.0
QwikScanTest(Last,FINGER,PREV,16.0) # LUMATOL = 16.0
TIM=RT_TimerHP()-START
RT_DebugF("TOTAL TIME = %.3f secs (%.2f mins)",TIM,TIM/60.0,name="QwikScanTest: ")
return MessageClip("QwikScanTest: DONE")

Function QwikScanTest(clip c,String FINGER,String PREV,Float LumaTol) { # Test script searching ~175,000 frames per FindFrame:
c myName="QwikScanTest: "
SearchStart=FrameCount-1 # Search direction from end to beginning, using PREV (ie Backward search).
RT_DebugF("",name=myName)
RT_DebugF("LumaTol = %.2f",LumaTol,name=myName)
GScript("""
Tot = 0.0 TotSearched=0
For(FindFrame=3000,13000,1000) {
S=RT_TimerHP()
Result = RT_QwikScan(Last,SearchStart,Last,FindFrame,FINGER,PREV,lumatol=LUMATOL,Flags=2,LD=0.0,maxdistance=SearchStart-0)
T=RT_TimerHP-S
Tot = Tot + T
Searched = SearchStart - Result + 1
TotSearched = TotSearched + Searched
RT_Debugf("%6d] Got Frame %6d in %10f secs",Findframe,Result,T,name=myName)
}
RT_Debugf("Avg Search time = %10f secs : Avg Searched = %.3f FPS",Tot/10.0,TotSearched/Tot,name=myName)
""")
return Last
}

Above Script Updated

Updated timings moved to following post (Too big)



Perhaps new FingerPrint ranges could be improved, but above about first guess at best settings.

EDIT: Above searches are using LD=0.0 (LumaDifference), ie exact match.
EDIT: As previously documented, RT_QwikScan is to be considered Permanently Experimental.
EDIT: Approx time to scan file using MatchFrames.avs (LumaDifference) would be about 16 to 18 Mins per frame search.
EDIT: Above tests done on 2.4GHz Core Duo (duel core,only 1 core used), pretty full SATA 2.
EDIT: The only real difference in script is that the argument names PrevDB and NextDB have dropped the DB suffix,
Files will be Array and will still work if using "DB" suffix, just suffix would be misleading. Can just supply Prev and Next args without
Optional Arg Names, and use "" where no Prev/Next Locator Array is needed (Prev/Next == Backward/Forward scan).

[StainlessS]

Above post script updated, timings moved here (too big for previous post).

Current Timings for v1.45 RT_Stats

00001433 18:34:01.671 RT_QwikScanCreate: Total time = 860.29 seconds (14.34 mins)

00001434 18:34:01.671 QwikScanTest:
00001435 18:34:01.671 QwikScanTest: LumaTol = 0.00
00001436 18:34:01.750 QwikScanTest: 3000] Got Frame 3000 in 0.053955 secs
00001437 18:34:01.796 QwikScanTest: 4000] Got Frame 4000 in 0.060242 secs
00001438 18:34:01.828 QwikScanTest: 5000] Got Frame 5000 in 0.027161 secs
00001439 18:34:01.859 QwikScanTest: 6000] Got Frame 6000 in 0.026855 secs
00001440 18:34:01.890 QwikScanTest: 7000] Got Frame 7000 in 0.032837 secs
00001441 18:34:01.921 QwikScanTest: 8000] Got Frame 8000 in 0.030457 secs
00001442 18:34:01.953 QwikScanTest: 9000] Got Frame 9000 in 0.032959 secs
00001443 18:34:01.984 QwikScanTest: 10000] Got Frame 10000 in 0.025818 secs
00001444 18:34:02.000 QwikScanTest: 11000] Got Frame 11000 in 0.024231 secs
00001445 18:34:02.031 QwikScanTest: 12000] Got Frame 12000 in 0.028748 secs
00001446 18:34:02.062 QwikScanTest: 13000] Got Frame 13000 in 0.028381 secs
00001447 18:34:02.062 QwikScanTest: Avg Search time = 0.037164 secs : Avg Searched = 5151582.500 FPS
00001448 18:34:02.062 QwikScanTest:
00001449 18:34:02.062 QwikScanTest: LumaTol = 4.00
00001450 18:34:02.125 QwikScanTest: 3000] Got Frame 3000 in 0.057007 secs
00001451 18:34:02.296 QwikScanTest: 4000] Got Frame 4000 in 0.168701 secs
00001452 18:34:05.140 QwikScanTest: 5000] Got Frame 5000 in 2.847839 secs
00001453 18:34:13.359 QwikScanTest: 6000] Got Frame 6000 in 8.218628 secs
00001454 18:34:15.218 QwikScanTest: 7000] Got Frame 7000 in 1.866821 secs
00001455 18:34:24.531 QwikScanTest: 8000] Got Frame 8000 in 9.311768 secs
00001456 18:34:25.296 QwikScanTest: 9000] Got Frame 9000 in 0.752258 secs
00001457 18:34:25.984 QwikScanTest: 10000] Got Frame 10000 in 0.699585 secs
00001458 18:34:30.796 QwikScanTest: 11000] Got Frame 11000 in 4.800659 secs
00001459 18:34:31.546 QwikScanTest: 12000] Got Frame 12000 in 0.761841 secs
00001460 18:34:35.250 QwikScanTest: 13000] Got Frame 13000 in 3.689514 secs
00001461 18:34:35.250 QwikScanTest: Avg Search time = 3.317462 secs : Avg Searched = 57711.285 FPS
00001462 18:34:35.250 QwikScanTest:
00001463 18:34:35.250 QwikScanTest: LumaTol = 8.00
00001464 18:34:49.171 QwikScanTest: 3000] Got Frame 3000 in 13.921143 secs
00001465 18:34:49.968 QwikScanTest: 4000] Got Frame 4000 in 0.803589 secs
00001466 18:35:04.390 QwikScanTest: 5000] Got Frame 5000 in 14.421509 secs
00001467 18:36:06.218 QwikScanTest: 6000] Got Frame 6000 in 61.818726 secs
00001468 18:36:18.453 QwikScanTest: 7000] Got Frame 7000 in 12.233459 secs
00001469 18:37:49.343 QwikScanTest: 8000] Got Frame 8000 in 90.886597 secs
00001470 18:37:56.453 QwikScanTest: 9000] Got Frame 9000 in 7.107300 secs
00001471 18:38:03.734 QwikScanTest: 10000] Got Frame 10000 in 7.283325 secs
00001472 18:38:48.187 QwikScanTest: 11000] Got Frame 11000 in 44.446167 secs
00001473 18:38:58.250 QwikScanTest: 12000] Got Frame 12000 in 10.061401 secs
00001474 18:39:23.031 QwikScanTest: 13000] Got Frame 13000 in 24.780884 secs
00001475 18:39:23.031 QwikScanTest: Avg Search time = 28.776409 secs : Avg Searched = 6653.193 FPS
00001476 18:39:23.031 QwikScanTest:
00001477 18:39:23.031 QwikScanTest: LumaTol = 12.00
00001478 18:41:19.312 QwikScanTest: 3000] Got Frame 3000 in 116.273315 secs
00001479 18:42:06.531 QwikScanTest: 4000] Got Frame 4000 in 47.215332 secs
00001480 18:42:55.093 QwikScanTest: 5000] Got Frame 5000 in 48.573608 secs
00001481 18:49:03.343 QwikScanTest: 6000] Got Frame 6000 in 368.208862 secs
00001482 18:50:08.765 QwikScanTest: 7000] Got Frame 7000 in 65.419678 secs
00001483 18:55:48.031 QwikScanTest: 8000] Got Frame 8000 in 339.256104 secs
00001484 18:56:07.890 QwikScanTest: 9000] Got Frame 9000 in 19.856689 secs
00001485 18:56:25.640 QwikScanTest: 10000] Got Frame 10000 in 17.749023 secs
00001486 18:58:35.125 QwikScanTest: 11000] Got Frame 11000 in 129.474365 secs
00001487 18:59:09.906 QwikScanTest: 12000] Got Frame 12000 in 34.775391 secs
00001488 19:00:21.500 QwikScanTest: 13000] Got Frame 13000 in 71.595459 secs
00001489 19:00:21.500 QwikScanTest: Avg Search time = 125.839783 secs : Avg Searched = 1521.419 FPS
00001490 19:00:21.500 QwikScanTest:
00001491 19:00:21.500 QwikScanTest: LumaTol = 16.00
00001492 19:05:16.984 QwikScanTest: 3000] Got Frame 3000 in 295.473145 secs
00001493 19:07:06.281 QwikScanTest: 4000] Got Frame 4000 in 109.297119 secs
00001494 19:07:33.468 QwikScanTest: 5000] Got Frame 5000 in 27.183594 secs
00001495 19:19:05.687 QwikScanTest: 6000] Got Frame 6000 in 692.185547 secs
00001496 19:21:22.656 QwikScanTest: 7000] Got Frame 7000 in 136.964844 secs
00001497 19:31:21.406 QwikScanTest: 8000] Got Frame 8000 in 598.720703 secs
00001498 19:31:55.656 QwikScanTest: 9000] Got Frame 9000 in 34.247559 secs
00001499 19:32:27.296 QwikScanTest: 10000] Got Frame 10000 in 31.631348 secs
00001500 19:38:06.203 QwikScanTest: 11000] Got Frame 11000 in 338.902344 secs
00001501 19:39:10.000 QwikScanTest: 12000] Got Frame 12000 in 63.796387 secs
00001502 19:42:01.140 QwikScanTest: 13000] Got Frame 13000 in 171.128418 secs
00001503 19:42:01.140 QwikScanTest: Avg Search time = 249.953094 secs : Avg Searched = 765.964 FPS
00001504 19:42:01.156 QwikScanTest: TOTAL TIME = 4079.293 secs (67.99 mins)


Upcoming RT_Stats v2.0 Timings

00001433 18:10:50.390 RT_QwikScanCreate: Total time = 791.58 seconds (13.19 mins)

00001434 18:10:50.390 QwikScanTest:
00001435 18:10:50.390 QwikScanTest: LumaTol = 0.00
00001436 18:10:50.781 QwikScanTest: 3000] Got Frame 3000 in 0.291931 secs
00001437 18:10:50.843 QwikScanTest: 4000] Got Frame 4000 in 0.068787 secs
00001438 18:10:50.890 QwikScanTest: 5000] Got Frame 5000 in 0.046021 secs
00001439 18:10:51.015 QwikScanTest: 6000] Got Frame 6000 in 0.132996 secs
00001440 18:10:51.218 QwikScanTest: 7000] Got Frame 7000 in 0.199158 secs
00001441 18:10:51.437 QwikScanTest: 8000] Got Frame 8000 in 0.214050 secs
00001442 18:10:51.515 QwikScanTest: 9000] Got Frame 9000 in 0.074219 secs
00001443 18:10:51.703 QwikScanTest: 10000] Got Frame 10000 in 0.192444 secs
00001444 18:10:51.921 QwikScanTest: 11000] Got Frame 11000 in 0.224243 secs
00001445 18:10:52.125 QwikScanTest: 12000] Got Frame 12000 in 0.203918 secs
00001446 18:10:52.218 QwikScanTest: 13000] Got Frame 13000 in 0.090942 secs
00001447 18:10:52.218 QwikScanTest: Avg Search time = 0.173871 secs : Avg Searched = 1101133.375 FPS
00001448 18:10:52.218 QwikScanTest:
00001449 18:10:52.218 QwikScanTest: LumaTol = 4.00
00001450 18:10:52.718 QwikScanTest: 3000] Got Frame 3000 in 0.494995 secs
00001451 18:10:52.765 QwikScanTest: 4000] Got Frame 4000 in 0.050171 secs
00001452 18:10:52.812 QwikScanTest: 5000] Got Frame 5000 in 0.048035 secs
00001453 18:10:53.234 QwikScanTest: 6000] Got Frame 6000 in 0.411987 secs
00001454 18:10:53.484 QwikScanTest: 7000] Got Frame 7000 in 0.258301 secs
00001455 18:10:53.796 QwikScanTest: 8000] Got Frame 8000 in 0.299316 secs
00001456 18:10:53.968 QwikScanTest: 9000] Got Frame 9000 in 0.180054 secs
00001457 18:10:54.531 QwikScanTest: 10000] Got Frame 10000 in 0.557739 secs
00001458 18:10:55.125 QwikScanTest: 11000] Got Frame 11000 in 0.598877 secs
00001459 18:10:55.343 QwikScanTest: 12000] Got Frame 12000 in 0.208557 secs
00001460 18:11:00.390 QwikScanTest: 13000] Got Frame 13000 in 5.059814 secs
00001461 18:11:00.390 QwikScanTest: Avg Search time = 0.816785 secs : Avg Searched = 234400.828 FPS
00001462 18:11:00.406 QwikScanTest:
00001463 18:11:00.406 QwikScanTest: LumaTol = 8.00
00001464 18:11:01.000 QwikScanTest: 3000] Got Frame 3000 in 0.595642 secs
00001465 18:11:01.328 QwikScanTest: 4000] Got Frame 4000 in 0.340088 secs
00001466 18:11:03.312 QwikScanTest: 5000] Got Frame 5000 in 1.975830 secs
00001467 18:11:05.531 QwikScanTest: 6000] Got Frame 6000 in 2.218445 secs
00001468 18:11:05.890 QwikScanTest: 7000] Got Frame 7000 in 0.365906 secs
00001469 18:11:06.578 QwikScanTest: 8000] Got Frame 8000 in 0.687744 secs
00001470 18:11:06.812 QwikScanTest: 9000] Got Frame 9000 in 0.229126 secs
00001471 18:11:07.921 QwikScanTest: 10000] Got Frame 10000 in 1.102112 secs
00001472 18:11:09.187 QwikScanTest: 11000] Got Frame 11000 in 1.262756 secs
00001473 18:11:10.484 QwikScanTest: 12000] Got Frame 12000 in 1.306335 secs
00001474 18:11:11.515 QwikScanTest: 13000] Got Frame 13000 in 1.029785 secs
00001475 18:11:11.515 QwikScanTest: Avg Search time = 1.111377 secs : Avg Searched = 172268.281 FPS
00001476 18:11:11.515 QwikScanTest:
00001477 18:11:11.515 QwikScanTest: LumaTol = 12.00
00001478 18:11:12.046 QwikScanTest: 3000] Got Frame 3000 in 0.528748 secs
00001479 18:11:14.109 QwikScanTest: 4000] Got Frame 4000 in 2.060547 secs
00001480 18:11:21.250 QwikScanTest: 5000] Got Frame 5000 in 7.143982 secs
00001481 18:11:35.296 QwikScanTest: 6000] Got Frame 6000 in 14.047180 secs
00001482 18:11:39.703 QwikScanTest: 7000] Got Frame 7000 in 4.407166 secs
00001483 18:11:52.484 QwikScanTest: 8000] Got Frame 8000 in 12.780212 secs
00001484 18:11:53.031 QwikScanTest: 9000] Got Frame 9000 in 0.539673 secs
00001485 18:11:54.390 QwikScanTest: 10000] Got Frame 10000 in 1.363708 secs
00001486 18:12:14.984 QwikScanTest: 11000] Got Frame 11000 in 20.584717 secs
00001487 18:12:29.234 QwikScanTest: 12000] Got Frame 12000 in 14.246704 secs
00001488 18:12:35.500 QwikScanTest: 13000] Got Frame 13000 in 6.279663 secs
00001489 18:12:35.500 QwikScanTest: Avg Search time = 8.398230 secs : Avg Searched = 22797.066 FPS
00001490 18:12:35.515 QwikScanTest:
00001491 18:12:35.515 QwikScanTest: LumaTol = 16.00
00001492 18:12:37.234 QwikScanTest: 3000] Got Frame 3000 in 1.723083 secs
00001493 18:12:42.578 QwikScanTest: 4000] Got Frame 4000 in 5.337830 secs
00001494 18:12:55.343 QwikScanTest: 5000] Got Frame 5000 in 12.768433 secs
00001495 18:13:36.156 QwikScanTest: 6000] Got Frame 6000 in 40.813232 secs
00001496 18:13:47.625 QwikScanTest: 7000] Got Frame 7000 in 11.472595 secs
00001497 18:14:44.906 QwikScanTest: 8000] Got Frame 8000 in 57.274414 secs
00001498 18:14:51.265 QwikScanTest: 9000] Got Frame 9000 in 6.364258 secs
00001499 18:15:04.093 QwikScanTest: 10000] Got Frame 10000 in 12.816406 secs
00001500 18:16:12.296 QwikScanTest: 11000] Got Frame 11000 in 68.203369 secs
00001501 18:17:00.859 QwikScanTest: 12000] Got Frame 12000 in 48.566162 secs
00001502 18:17:53.093 QwikScanTest: 13000] Got Frame 13000 in 52.229004 secs
00001503 18:17:53.093 QwikScanTest: Avg Search time = 31.756878 secs : Avg Searched = 6028.772 FPS
00001504 18:17:53.093 QwikScanTest: TOTAL TIME = 422.690 secs (7.04 mins)


Perhaps I could just use 4 component fingerprint where LumaTol=0.0, seems quite sufficient.

[StainlessS]

Post #1 of 2

Have now decided upon new 16 component FingerPrint to replace old 4 component FingerPrint in RT_QwikScan.

Here timing test on JurassicPark (non-RoboCropped, would be faster if small amount of letterboxing removed as it would 'water down' fingerprinting).


AVISource("JurassicPark.avi") #.Trim(0,-100000)

RTVER2 = (RT_Version > 1.99) # New Version RT_Stats ?

EXT = (RTVER2) ? ".ARR" : ".DB"

FINGER = "JPARK" + EXT
PREV = "JPARK_PREV" + EXT # Prev LOCATOR, ie BackWards Search.
NEXT = "JPARK_NEXT" + EXT # Next LOCATOR, ie ForWards Search (not used in this script).

RT_FileDelete(FINGER) RT_FileDelete(PREV) RT_FileDelete(NEXT) # Uncomment to Create QwikScan DBs/Arrays, comment out to Re-Run

(Exist(FINGER)) ? NOP : RT_QwikScanCreate(Last, FINGER , PREV,"", debug=true) # We dont need NEXT Locator for this backwards scan test.

START=RT_TimerHP()
QwikScanTest(Last,FINGER,PREV,0.0)
QwikScanTest(Last,FINGER,PREV,1.0)
QwikScanTest(Last,FINGER,PREV,2.0)
QwikScanTest(Last,FINGER,PREV,3.0)
QwikScanTest(Last,FINGER,PREV,4.0)
QwikScanTest(Last,FINGER,PREV,5.0)
QwikScanTest(Last,FINGER,PREV,6.0)
QwikScanTest(Last,FINGER,PREV,7.0)
QwikScanTest(Last,FINGER,PREV,8.0)
QwikScanTest(Last,FINGER,PREV,9.0)
QwikScanTest(Last,FINGER,PREV,10.0)
QwikScanTest(Last,FINGER,PREV,11.0)
QwikScanTest(Last,FINGER,PREV,12.0)
QwikScanTest(Last,FINGER,PREV,13.0)
QwikScanTest(Last,FINGER,PREV,14.0)
QwikScanTest(Last,FINGER,PREV,15.0)
QwikScanTest(Last,FINGER,PREV,16.0)
TIM=RT_TimerHP()-START
RT_DebugF("TOTAL TIME = %.3f secs (%.2f mins)",TIM,TIM/60.0,name="QwikScanTest: ")
return MessageClip("QwikScanTest: DONE")

Function QwikScanTest(clip c,String FINGER,String PREV,Float LumaTol) { # Test script searching ~175,000 frames per FindFrame:
c myName="QwikScanTest: "
SearchStart=FrameCount-1 # Search direction from end to beginning, using PREV (ie Backward search).
RT_DebugF("",name=myName)
RT_DebugF("LumaTol = %.2f",LumaTol,name=myName)
GScript("""
Tot = 0.0 TotSearched=0
For(FindFrame=3000,13000,1000) {
S=RT_TimerHP()
Result = RT_QwikScan(Last,SearchStart,Last,FindFrame,FINGER,PREV,lumatol=LUMATOL,Flags=2,LD=0.0,maxdistance=SearchStart-0)
T=RT_TimerHP-S
Tot = Tot + T
Searched = SearchStart - Result + 1
TotSearched = TotSearched + Searched
RT_Debugf("%6d] Got Frame %6d in %10f secs",Findframe,Result,T,name=myName)
}
RT_Debugf("Avg Search time = %10f secs : Avg Searched = %.3f FPS",Tot/10.0,TotSearched/Tot,name=myName)
""")
return Last
}


Timing test Part 1

00000004 11:50:39 [1992] RT_QwikScanCreate: RT_QwikScanCreate() by StainlessS
00000051 12:05:19 RT_QwikScanCreate: Total time = 879.56 seconds (14.66 mins)
00000052 12:05:19 QwikScanTest:
00000053 12:05:19 QwikScanTest: LumaTol = 0.00
00000054 12:05:19 QwikScanTest: 3000] Got Frame 3000 in 0.346375 secs
00000055 12:05:19 QwikScanTest: 4000] Got Frame 4000 in 0.051880 secs
00000056 12:05:19 QwikScanTest: 5000] Got Frame 5000 in 0.076843 secs
00000057 12:05:20 QwikScanTest: 6000] Got Frame 6000 in 0.126953 secs
00000058 12:05:20 QwikScanTest: 7000] Got Frame 7000 in 0.132751 secs
00000059 12:05:20 QwikScanTest: 8000] Got Frame 8000 in 0.130676 secs
00000060 12:05:20 QwikScanTest: 9000] Got Frame 9000 in 0.090881 secs
00000061 12:05:20 QwikScanTest: 10000] Got Frame 10000 in 0.267456 secs
00000062 12:05:20 QwikScanTest: 11000] Got Frame 11000 in 0.157593 secs
00000063 12:05:21 QwikScanTest: 12000] Got Frame 12000 in 0.162415 secs
00000064 12:05:21 QwikScanTest: 13000] Got Frame 13000 in 0.124023 secs
00000065 12:05:21 QwikScanTest: Avg Search time = 0.166785 secs : Avg Searched = 1147917.250 FPS
00000066 12:05:21 QwikScanTest:
00000067 12:05:21 QwikScanTest: LumaTol = 1.00
00000068 12:05:21 QwikScanTest: 3000] Got Frame 3000 in 0.349548 secs
00000069 12:05:21 QwikScanTest: 4000] Got Frame 4000 in 0.012817 secs
00000070 12:05:21 QwikScanTest: 5000] Got Frame 5000 in 0.020447 secs
00000071 12:05:21 QwikScanTest: 6000] Got Frame 6000 in 0.292786 secs
00000072 12:05:21 QwikScanTest: 7000] Got Frame 7000 in 0.063477 secs
00000073 12:05:22 QwikScanTest: 8000] Got Frame 8000 in 0.217041 secs
00000074 12:05:22 QwikScanTest: 9000] Got Frame 9000 in 0.085693 secs
00000075 12:05:22 QwikScanTest: 10000] Got Frame 10000 in 0.131409 secs
00000076 12:05:22 QwikScanTest: 11000] Got Frame 11000 in 0.136658 secs
00000077 12:05:22 QwikScanTest: 12000] Got Frame 12000 in 0.071411 secs
00000078 12:05:22 QwikScanTest: 13000] Got Frame 13000 in 0.146118 secs
00000079 12:05:22 QwikScanTest: Avg Search time = 0.152740 secs : Avg Searched = 1253466.000 FPS
00000080 12:05:22 QwikScanTest:
00000081 12:05:22 QwikScanTest: LumaTol = 2.00
00000082 12:05:22 QwikScanTest: 3000] Got Frame 3000 in 0.171509 secs
00000083 12:05:22 QwikScanTest: 4000] Got Frame 4000 in 0.009888 secs
00000084 12:05:22 QwikScanTest: 5000] Got Frame 5000 in 0.012207 secs
00000085 12:05:22 QwikScanTest: 6000] Got Frame 6000 in 0.099548 secs
00000086 12:05:23 QwikScanTest: 7000] Got Frame 7000 in 0.072693 secs
00000087 12:05:23 QwikScanTest: 8000] Got Frame 8000 in 0.161621 secs
00000088 12:05:23 QwikScanTest: 9000] Got Frame 9000 in 0.056641 secs
00000089 12:05:23 QwikScanTest: 10000] Got Frame 10000 in 0.108093 secs
00000090 12:05:23 QwikScanTest: 11000] Got Frame 11000 in 0.092224 secs
00000091 12:05:23 QwikScanTest: 12000] Got Frame 12000 in 0.028442 secs
00000092 12:05:24 QwikScanTest: 13000] Got Frame 13000 in 1.214905 secs
00000093 12:05:24 QwikScanTest: Avg Search time = 0.202777 secs : Avg Searched = 944164.813 FPS
00000094 12:05:24 QwikScanTest:
00000095 12:05:24 QwikScanTest: LumaTol = 3.00
00000096 12:05:24 QwikScanTest: 3000] Got Frame 3000 in 0.105103 secs
00000097 12:05:24 QwikScanTest: 4000] Got Frame 4000 in 0.004822 secs
00000098 12:05:24 QwikScanTest: 5000] Got Frame 5000 in 0.043945 secs
00000099 12:05:25 QwikScanTest: 6000] Got Frame 6000 in 0.165405 secs
00000100 12:05:25 QwikScanTest: 7000] Got Frame 7000 in 0.232666 secs
00000101 12:05:25 QwikScanTest: 8000] Got Frame 8000 in 0.128418 secs
00000102 12:05:25 QwikScanTest: 9000] Got Frame 9000 in 0.072754 secs
00000103 12:05:25 QwikScanTest: 10000] Got Frame 10000 in 0.239380 secs
00000104 12:05:25 QwikScanTest: 11000] Got Frame 11000 in 0.201599 secs
00000105 12:05:26 QwikScanTest: 12000] Got Frame 12000 in 0.096985 secs
00000106 12:05:26 QwikScanTest: 13000] Got Frame 13000 in 0.769226 secs
00000107 12:05:26 QwikScanTest: Avg Search time = 0.206030 secs : Avg Searched = 929256.625 FPS
00000108 12:05:26 QwikScanTest:
00000109 12:05:26 QwikScanTest: LumaTol = 4.00
00000110 12:05:27 QwikScanTest: 3000] Got Frame 3000 in 0.238403 secs
00000111 12:05:27 QwikScanTest: 4000] Got Frame 4000 in 0.041504 secs
00000112 12:05:27 QwikScanTest: 5000] Got Frame 5000 in 0.049438 secs
00000113 12:05:27 QwikScanTest: 6000] Got Frame 6000 in 0.263123 secs
00000114 12:05:27 QwikScanTest: 7000] Got Frame 7000 in 0.137268 secs
00000115 12:05:27 QwikScanTest: 8000] Got Frame 8000 in 0.189087 secs
00000116 12:05:27 QwikScanTest: 9000] Got Frame 9000 in 0.166199 secs
00000117 12:05:28 QwikScanTest: 10000] Got Frame 10000 in 0.145508 secs
00000118 12:05:28 QwikScanTest: 11000] Got Frame 11000 in 0.192322 secs
00000119 12:05:28 QwikScanTest: 12000] Got Frame 12000 in 0.148560 secs
00000120 12:05:28 QwikScanTest: 13000] Got Frame 13000 in 0.384705 secs
00000121 12:05:28 QwikScanTest: Avg Search time = 0.195612 secs : Avg Searched = 978750.875 FPS
00000122 12:05:28 QwikScanTest:
00000123 12:05:28 QwikScanTest: LumaTol = 5.00
00000124 12:05:29 QwikScanTest: 3000] Got Frame 3000 in 0.280396 secs
00000125 12:05:29 QwikScanTest: 4000] Got Frame 4000 in 0.054749 secs
00000126 12:05:29 QwikScanTest: 5000] Got Frame 5000 in 0.365051 secs
00000127 12:05:29 QwikScanTest: 6000] Got Frame 6000 in 0.365234 secs
00000128 12:05:30 QwikScanTest: 7000] Got Frame 7000 in 0.207642 secs
00000129 12:05:30 QwikScanTest: 8000] Got Frame 8000 in 0.349792 secs
00000130 12:05:30 QwikScanTest: 9000] Got Frame 9000 in 0.080261 secs
00000131 12:05:30 QwikScanTest: 10000] Got Frame 10000 in 0.232544 secs
00000132 12:05:30 QwikScanTest: 11000] Got Frame 11000 in 0.219299 secs
00000133 12:05:31 QwikScanTest: 12000] Got Frame 12000 in 0.199646 secs
00000134 12:05:31 QwikScanTest: 13000] Got Frame 13000 in 0.278564 secs
00000135 12:05:31 QwikScanTest: Avg Search time = 0.263318 secs : Avg Searched = 727087.000 FPS
00000136 12:05:31 QwikScanTest:
00000137 12:05:31 QwikScanTest: LumaTol = 6.00
00000138 12:05:31 QwikScanTest: 3000] Got Frame 3000 in 0.126953 secs
00000139 12:05:31 QwikScanTest: 4000] Got Frame 4000 in 0.116028 secs
00000140 12:05:32 QwikScanTest: 5000] Got Frame 5000 in 0.448853 secs
00000141 12:05:32 QwikScanTest: 6000] Got Frame 6000 in 0.518555 secs
00000142 12:05:32 QwikScanTest: 7000] Got Frame 7000 in 0.190247 secs
00000143 12:05:33 QwikScanTest: 8000] Got Frame 8000 in 0.596313 secs
00000144 12:05:33 QwikScanTest: 9000] Got Frame 9000 in 0.124817 secs
00000145 12:05:33 QwikScanTest: 10000] Got Frame 10000 in 0.258606 secs
00000146 12:05:34 QwikScanTest: 11000] Got Frame 11000 in 0.433838 secs
00000147 12:05:34 QwikScanTest: 12000] Got Frame 12000 in 0.247864 secs
00000148 12:05:35 QwikScanTest: 13000] Got Frame 13000 in 0.598877 secs
00000149 12:05:35 QwikScanTest: Avg Search time = 0.366095 secs : Avg Searched = 522965.406 FPS
00000150 12:05:35 QwikScanTest:
00000151 12:05:35 QwikScanTest: LumaTol = 7.00
00000152 12:05:35 QwikScanTest: 3000] Got Frame 3000 in 0.435547 secs
00000153 12:05:35 QwikScanTest: 4000] Got Frame 4000 in 0.157654 secs
00000154 12:05:36 QwikScanTest: 5000] Got Frame 5000 in 0.804016 secs
00000155 12:05:37 QwikScanTest: 6000] Got Frame 6000 in 0.545349 secs
00000156 12:05:37 QwikScanTest: 7000] Got Frame 7000 in 0.176636 secs
00000157 12:05:37 QwikScanTest: 8000] Got Frame 8000 in 0.412476 secs
00000158 12:05:37 QwikScanTest: 9000] Got Frame 9000 in 0.084412 secs
00000159 12:05:38 QwikScanTest: 10000] Got Frame 10000 in 0.352600 secs
00000160 12:05:38 QwikScanTest: 11000] Got Frame 11000 in 0.585999 secs
00000161 12:05:38 QwikScanTest: 12000] Got Frame 12000 in 0.367310 secs
00000162 12:05:39 QwikScanTest: 13000] Got Frame 13000 in 0.618896 secs
00000163 12:05:39 QwikScanTest: Avg Search time = 0.454089 secs : Avg Searched = 421624.063 FPS
00000164 12:05:39 QwikScanTest:
00000165 12:05:39 QwikScanTest: LumaTol = 8.00
00000166 12:05:39 QwikScanTest: 3000] Got Frame 3000 in 0.167358 secs
00000167 12:05:40 QwikScanTest: 4000] Got Frame 4000 in 0.343262 secs
00000168 12:05:41 QwikScanTest: 5000] Got Frame 5000 in 1.232056 secs
00000169 12:05:42 QwikScanTest: 6000] Got Frame 6000 in 0.883545 secs
00000170 12:05:42 QwikScanTest: 7000] Got Frame 7000 in 0.198364 secs
00000171 12:05:43 QwikScanTest: 8000] Got Frame 8000 in 0.599060 secs
00000172 12:05:43 QwikScanTest: 9000] Got Frame 9000 in 0.187134 secs
00000173 12:05:43 QwikScanTest: 10000] Got Frame 10000 in 0.431396 secs
00000174 12:05:44 QwikScanTest: 11000] Got Frame 11000 in 0.830872 secs
00000175 12:05:44 QwikScanTest: 12000] Got Frame 12000 in 0.319458 secs
00000176 12:05:45 QwikScanTest: 13000] Got Frame 13000 in 0.725037 secs
00000177 12:05:45 QwikScanTest: Avg Search time = 0.591754 secs : Avg Searched = 323538.063 FPS

[StainlessS]

Post #2 of 2

Timing Test Part 2


00000178 12:05:45 QwikScanTest:
00000179 12:05:45 QwikScanTest: LumaTol = 9.00
00000180 12:05:45 QwikScanTest: 3000] Got Frame 3000 in 0.222961 secs
00000181 12:05:46 QwikScanTest: 4000] Got Frame 4000 in 0.335266 secs
00000182 12:05:48 QwikScanTest: 5000] Got Frame 5000 in 2.186646 secs
00000183 12:05:50 QwikScanTest: 6000] Got Frame 6000 in 2.281433 secs
00000184 12:05:51 QwikScanTest: 7000] Got Frame 7000 in 1.033875 secs
00000185 12:05:53 QwikScanTest: 8000] Got Frame 8000 in 1.965820 secs
00000186 12:05:53 QwikScanTest: 9000] Got Frame 9000 in 0.115051 secs
00000187 12:05:54 QwikScanTest: 10000] Got Frame 10000 in 0.577332 secs
00000188 12:05:55 QwikScanTest: 11000] Got Frame 11000 in 1.148682 secs
00000189 12:05:55 QwikScanTest: 12000] Got Frame 12000 in 0.512390 secs
00000190 12:05:57 QwikScanTest: 13000] Got Frame 13000 in 1.153259 secs
00000191 12:05:57 QwikScanTest: Avg Search time = 1.153271 secs : Avg Searched = 166010.344 FPS
00000192 12:05:57 QwikScanTest:
00000193 12:05:57 QwikScanTest: LumaTol = 10.00
00000194 12:05:57 QwikScanTest: 3000] Got Frame 3000 in 0.277588 secs
00000195 12:05:57 QwikScanTest: 4000] Got Frame 4000 in 0.470886 secs
00000196 12:06:01 QwikScanTest: 5000] Got Frame 5000 in 3.773743 secs
00000197 12:06:04 QwikScanTest: 6000] Got Frame 6000 in 3.006226 secs
00000198 12:06:05 QwikScanTest: 7000] Got Frame 7000 in 0.659546 secs
00000199 12:06:09 QwikScanTest: 8000] Got Frame 8000 in 4.409241 secs
00000200 12:06:09 QwikScanTest: 9000] Got Frame 9000 in 0.171204 secs
00000201 12:06:10 QwikScanTest: 10000] Got Frame 10000 in 0.336792 secs
00000202 12:06:12 QwikScanTest: 11000] Got Frame 11000 in 2.671021 secs
00000203 12:06:13 QwikScanTest: 12000] Got Frame 12000 in 0.721191 secs
00000204 12:06:15 QwikScanTest: 13000] Got Frame 13000 in 1.599243 secs
00000205 12:06:15 QwikScanTest: Avg Search time = 1.809668 secs : Avg Searched = 105795.648 FPS
00000206 12:06:15 QwikScanTest:
00000207 12:06:15 QwikScanTest: LumaTol = 11.00
00000208 12:06:15 QwikScanTest: 3000] Got Frame 3000 in 0.489136 secs
00000209 12:06:16 QwikScanTest: 4000] Got Frame 4000 in 0.612000 secs
00000210 12:06:21 QwikScanTest: 5000] Got Frame 5000 in 4.916809 secs
00000211 12:06:27 QwikScanTest: 6000] Got Frame 6000 in 6.184570 secs
00000212 12:06:28 QwikScanTest: 7000] Got Frame 7000 in 1.295044 secs
00000213 12:06:33 QwikScanTest: 8000] Got Frame 8000 in 5.124390 secs
00000214 12:06:34 QwikScanTest: 9000] Got Frame 9000 in 0.240662 secs
00000215 12:06:34 QwikScanTest: 10000] Got Frame 10000 in 0.464783 secs
00000216 12:06:36 QwikScanTest: 11000] Got Frame 11000 in 2.268311 secs
00000217 12:06:38 QwikScanTest: 12000] Got Frame 12000 in 1.496033 secs
00000218 12:06:40 QwikScanTest: 13000] Got Frame 13000 in 2.119751 secs
00000219 12:06:40 QwikScanTest: Avg Search time = 2.521149 secs : Avg Searched = 75939.594 FPS
00000220 12:06:40 QwikScanTest:
00000221 12:06:40 QwikScanTest: LumaTol = 12.00
00000222 12:06:40 QwikScanTest: 3000] Got Frame 3000 in 0.301514 secs
00000223 12:06:41 QwikScanTest: 4000] Got Frame 4000 in 0.603210 secs
00000224 12:06:44 QwikScanTest: 5000] Got Frame 5000 in 3.602295 secs
00000225 12:06:58 QwikScanTest: 6000] Got Frame 6000 in 13.220520 secs
00000226 12:06:59 QwikScanTest: 7000] Got Frame 7000 in 1.423706 secs
00000227 12:07:06 QwikScanTest: 8000] Got Frame 8000 in 7.343750 secs
00000228 12:07:07 QwikScanTest: 9000] Got Frame 9000 in 0.227539 secs
00000229 12:07:07 QwikScanTest: 10000] Got Frame 10000 in 0.453613 secs
00000230 12:07:12 QwikScanTest: 11000] Got Frame 11000 in 4.815796 secs
00000231 12:07:14 QwikScanTest: 12000] Got Frame 12000 in 1.819641 secs
00000232 12:07:17 QwikScanTest: 13000] Got Frame 13000 in 3.590393 secs
00000233 12:07:17 QwikScanTest: Avg Search time = 3.740198 secs : Avg Searched = 51188.469 FPS
00000234 12:07:17 QwikScanTest:
00000235 12:07:17 QwikScanTest: LumaTol = 13.00
00000236 12:07:18 QwikScanTest: 3000] Got Frame 3000 in 0.677856 secs
00000237 12:07:19 QwikScanTest: 4000] Got Frame 4000 in 1.226013 secs
00000238 12:07:24 QwikScanTest: 5000] Got Frame 5000 in 4.723145 secs
00000239 12:07:45 QwikScanTest: 6000] Got Frame 6000 in 21.395813 secs
00000240 12:07:47 QwikScanTest: 7000] Got Frame 7000 in 1.615479 secs
00000241 12:07:57 QwikScanTest: 8000] Got Frame 8000 in 9.870605 secs
00000242 12:07:57 QwikScanTest: 9000] Got Frame 9000 in 0.573364 secs
00000243 12:07:58 QwikScanTest: 10000] Got Frame 10000 in 0.660889 secs
00000244 12:08:09 QwikScanTest: 11000] Got Frame 11000 in 11.117920 secs
00000245 12:08:11 QwikScanTest: 12000] Got Frame 12000 in 2.308838 secs
00000246 12:08:17 QwikScanTest: 13000] Got Frame 13000 in 5.714478 secs
00000247 12:08:17 QwikScanTest: Avg Search time = 5.988440 secs : Avg Searched = 31970.764 FPS
00000248 12:08:17 QwikScanTest:
00000249 12:08:17 QwikScanTest: LumaTol = 14.00
00000250 12:08:18 QwikScanTest: 3000] Got Frame 3000 in 0.886475 secs
00000251 12:08:20 QwikScanTest: 4000] Got Frame 4000 in 1.673950 secs
00000252 12:08:27 QwikScanTest: 5000] Got Frame 5000 in 6.839844 secs
00000253 12:08:53 QwikScanTest: 6000] Got Frame 6000 in 26.627686 secs
00000254 12:08:55 QwikScanTest: 7000] Got Frame 7000 in 1.999268 secs
00000255 12:09:09 QwikScanTest: 8000] Got Frame 8000 in 13.562744 secs
00000256 12:09:10 QwikScanTest: 9000] Got Frame 9000 in 1.127197 secs
00000257 12:09:10 QwikScanTest: 10000] Got Frame 10000 in 0.562378 secs
00000258 12:09:23 QwikScanTest: 11000] Got Frame 11000 in 12.665771 secs
00000259 12:09:28 QwikScanTest: 12000] Got Frame 12000 in 4.609619 secs
00000260 12:09:35 QwikScanTest: 13000] Got Frame 13000 in 7.313232 secs
00000261 12:09:35 QwikScanTest: Avg Search time = 7.786817 secs : Avg Searched = 24587.070 FPS
00000262 12:09:35 QwikScanTest:
00000263 12:09:35 QwikScanTest: LumaTol = 15.00
00000264 12:09:37 QwikScanTest: 3000] Got Frame 3000 in 1.915161 secs
00000265 12:09:40 QwikScanTest: 4000] Got Frame 4000 in 2.822021 secs
00000266 12:10:14 QwikScanTest: 5000] Got Frame 5000 in 34.144043 secs
00000267 12:12:58 QwikScanTest: 6000] Got Frame 6000 in 163.648926 secs
00000268 12:13:11 QwikScanTest: 7000] Got Frame 7000 in 13.373291 secs
00000269 12:14:30 QwikScanTest: 8000] Got Frame 8000 in 79.159424 secs
00000270 12:14:34 QwikScanTest: 9000] Got Frame 9000 in 3.889893 secs
00000271 12:14:38 QwikScanTest: 10000] Got Frame 10000 in 3.750244 secs
00000272 12:15:25 QwikScanTest: 11000] Got Frame 11000 in 47.346069 secs
00000273 12:15:46 QwikScanTest: 12000] Got Frame 12000 in 20.660645 secs
00000274 12:16:19 QwikScanTest: 13000] Got Frame 13000 in 33.497070 secs
00000275 12:16:19 QwikScanTest: Avg Search time = 40.420677 secs : Avg Searched = 4736.561 FPS
00000276 12:16:19 QwikScanTest:
00000277 12:16:19 QwikScanTest: LumaTol = 16.00
00000278 12:16:26 QwikScanTest: 3000] Got Frame 3000 in 6.523193 secs
00000279 12:16:51 QwikScanTest: 4000] Got Frame 4000 in 24.800171 secs
00000280 12:17:20 QwikScanTest: 5000] Got Frame 5000 in 28.942017 secs
00000281 12:20:53 QwikScanTest: 6000] Got Frame 6000 in 213.503906 secs
00000282 12:21:10 QwikScanTest: 7000] Got Frame 7000 in 17.306396 secs
00000283 12:23:00 QwikScanTest: 8000] Got Frame 8000 in 109.623169 secs
00000284 12:23:05 QwikScanTest: 9000] Got Frame 9000 in 5.133057 secs
00000285 12:23:10 QwikScanTest: 10000] Got Frame 10000 in 4.757690 secs
00000286 12:24:08 QwikScanTest: 11000] Got Frame 11000 in 58.195313 secs
00000287 12:24:36 QwikScanTest: 12000] Got Frame 12000 in 27.926880 secs
00000288 12:25:20 QwikScanTest: 13000] Got Frame 13000 in 44.354736 secs
00000289 12:25:20 QwikScanTest: Avg Search time = 54.106651 secs : Avg Searched = 3538.474 FPS
00000290 12:25:20 QwikScanTest: TOTAL TIME = 1201.369 secs (20.02 mins)


Here the old 4 Component FingerPrint
https://s20.postimg.cc/z70c67wjh/Qwik_Scan_OLD_zpsasksjiyc.png (https://postimg.cc/image/iw089wk1l/)

And new 16 Component FingerPrint
https://s20.postimg.cc/9cqjgfwjh/Qwik_Scan_NEW_zpsukgwzvjw.png (https://postimg.cc/image/6ine2zud5/)

And script used to create graphic

SSS="""
{0,47},
{48,207},
{208,255},

{0,63},
{64,191},
{192,255},

{0,79},
{80,175},
{176,255},

{0,95},
{96,159},
{160,255},

{0,111},
{144,255},

{32, 95},
{160,223},

"""


GSCript("""
c=RT_Undefined
Lines = RT_TxtQueryLines(SSS)
For(i=0,Lines-1) {
S=RT_TxtGetLine(SSS,i)
S=EatWhite(S)
if(S!="" && !IsHash(S)) {
S=EatOBrace(S).EatWhite
Assert(IsDigit(S),RT_String("Not a Digit [Start=%s]",S))
lo=RT_NumberValue(S) S=EatDigits(S).EatWhite
Assert(IsComma(S),RT_String("Not a Comma %s",S))
S=EatComma(S).EatWhite
Assert(IsDigit(S),RT_String("Not a Digit [End=%s]",S))
hi=RT_NumberValue(S)
S=EatDigits(S).EatWhite.EatCBrace.EatWhite.EatComma.EatWhite.EatHash
Assert(S=="",RT_String("Not End of String [S=%s]",S))
c = Doit(lo,hi,c)
}
}
return c


Function Doit(int lo,int hi,clip "c") {
wmul=2 Wid=512 Hit=20 ExtH=4 YOff=(Hit-ExtH)/2
c2=BlankClip(Width=Wid,Height=Hit,Length=1)
c3=c2.BlankClip(Width=(hi-lo+1)*wmul,color=$FFFFFF)
c2=c2.Overlay(c3,x=lo*wmul,opacity=0.5)
lmin = Max(lo-16,0) lmax = Min(lo+15,255) lw=(lmax-lmin+1)*wmul
c3=c2.BlankClip(Width=lw,Height=ExtH,color=$FF00FF)
c2=c2.Overlay(c3,x=lmin*wmul,y=YOff,opacity=0.5)
hmin = Max(hi-15,0) hmax = Min(hi+16,255) hw=(hmax-hmin+1)*wmul
c3=c2.BlankClip(Width=hw,Height=ExtH,color=$FF00FF)
c2=c2.Overlay(c3,x=hmin*wmul,y=YOff,opacity=0.5)
hs=String(hi)
c2=c2.RT_Subtitle("%d",lo,align=5,x=lo*wmul,y=2).RT_Subtitle("%s",hs,align=5,x=(hi*wmul)-StrLen(hs)*10,y=2)
c = (c.IsClip) ? StackVertical(c,c2) : c2
return c
}
""")


Function IsDigit(String S) {return (RT_Ord(S)>=48 && RT_Ord(S)<=57)}
Function IsComma(String S) {return (RT_Ord(S)==RT_Ord(","))}
Function IsOBrace(String S) {return (RT_Ord(S)==RT_Ord("{"))}
Function IsCBrace(String S) {return (RT_Ord(S)==RT_Ord("}"))}
Function IsHash(String S) {return (RT_Ord(S)==RT_Ord("#"))}
Function EatComma(String S) {return (IsComma(S))?MidStr(S,2):S}
Function EatOBrace(String S) {return (IsOBrace(S))?MidStr(S,2):S}
Function EatCBrace(String S) {return (IsCBrace(S))?MidStr(S,2):S}
Function EatWhite(String S) {return (RT_Ord(S)==32 || RT_Ord(S)==9) ?EatWhite(MidStr(S,2)):S}
Function EatDigits(String S) {return (IsDigit(S)) ?EatDigits(MidStr(S,2)): S}
Function EatHash(String S) {return (IsHash(S)) ?"":S}


The timing test is just testing effectiveness of FingerPrinting, ie how good it is at examining just frames that are similar to the one we are searching for,
and skipping over those that are not similar, without testing them.
With higher values of LumaTol there are many more frames that will match the FingerPrint and so FingerPrinting is less effective.

The FingerPrint components are Luma histogram ranges with LumaTol tolerance (which expands or contracts the range, luma pixel
population must be between those of the expanded and contracted ranges to match, and for every range). The Magenta line in graphic shows expansion/contraction range (at maximum LumaTol=16.0).

As can probably be seen from the graphics, we are tracking the 'hump' in the middle of a typical luma histogram.

EDIT: Above tests done on 2.4 Ghz Core Duo, Sata 2. More modern machines might produce speeds in the region of 10 times the above speeds.

EDIT: Re-did tests on RoboCropped source and overall time was about 1.5 mins less (~20 min -> 18.5 mins).

EDIT: Did search on JurassicPark frame 173,000, using MatchFrames.avs (perhaps next best frame locator along with LocateFrames.avs
http://forum.doom9.org/showthread.php?t=164766&highlight=Matchframes),
Took 999.935 secs (16.67 mins) to locate the frame (RT_QwikScan average search distance about 175,000 frames, searching from last frame downwards, Matchframes searches only forward and so chose frame 173,000 to be about same distance.
EDIT: MatchFrames was using not yet available RT_Stats v2.0 RT_FrameDifference, which is significantly faster than v1.45 RT_.

EDIT: Any clip that has had luma histogram equalization applied will likely fail, all fingerprints will be pretty much identical and so almost
every frame will be scanned, ie slow.

[StainlessS]

RT_Stats v2.00 Beta01.

See MediaFire/SendSpace in my sig below.

Post 1 of 4


REMOVED:-
RT_ArrayGetNelMax
RT_ArrayGetNels
RT_LumaMovement
RT_LumaSceneChange

CHANGED/ADDED ARGS:
RT_ArrayGetDim1Max "s[Current]b"
RT_ArraySetAttrib "si.+"
RT_ArraySetID "si.+"
RT_ArraySetStrAttrib "sis+"
RT_DBaseRecordsMax "s[Current]b"
RT_DBaseSet "si.+"
RT_DBaseSetAttrib "si.+"
RT_DBaseSetField "sii.+"
RT_DBaseSetID "si.+"
RT_DBaseSetStrAttrib "sis+"
RT_FrameDifference "cc[n]i[n2]i[ChromaWeight]f[x]i[y]i[w]i[h]i[x2]i[y2]i[altscan]b[ChromaI]b[Matrix]i"
RT_QwikScanCreate "cs[Prev]s[Next]s[Matrix]i[Debug]b"
#############################################################################

ADDED:
RT_ArrayCheckID "si.+"
RT_DBaseCheckID "si.+"
RT_DBaseCmpField "siii[Field2]i[Field3]i[Sig]b"
RT_DBaseCocktailSort "s[RecS]i[RecE]i[Field]i[Field2]i[Field3]i[Sig]b[Ascend]b[Debug]b"
RT_DBaseGetStrAttrib "si"
RT_DBaseGetTypeString "s"
RT_DBaseQuickSort "s[RecS]i[RecE]i[Field]i[Field2]i[Field3]i[Sig]b[Ascend]b[Debug]b"
RT_DBaseRecordSwap "sii"
RT_FileDuplicate "ss[Overwrite]b"
RT_ForceProcess "c[Video]b[Audio]b"
RT_FrameMovement "cc[n]i[n2]i[ChromaWeight]f[x]i[y]i[w]i[h]i[x2]i[y2]i[altscan]b[ChromaI]b[Matrix]i[blkw]i[blkh]i[OlapX]i[OLapY]i[BlkTh]f[Prefix]s"
RT_GetPid ""
RT_QueryColor "c[n]i[x]i[y]i[w]i[h]i[altscan]b[CDf]i[CTh]f[ChromaI]b"

[StainlessS]

Post 2 of 4



RT_ArrayCheckID(String Filename,Int Start_ix,var ID1, ... ,var IDn)
This function allows you to compare Array ID's with some known ID's, eg match clip Width with some stored ID width, ie is it
the clip that Array was written for. Allows you to simultaneously check multiple ID's, starting at Start_ix in the Array ID array.
Filename, Array Filename, no default.
Start_Ix, ID starting array element index range 0 -> RT_ArrayGetID(Filename,-1) - 1.
IDx, Int or Float, The value that ID(Start_ix + n) will be checked against (type [int/float] and value must match exactly).
Error if ID1 -> IDn not of type Int or type Float.
Returns -1, all ID's compared the same, otherwise the index (0 relative) of the ID comparison that failed.
NOTE, if eg Start_ix = 4, and return value=4, then ID1 was compared with Array ID[4] and that comparison failed to match.

***
***
***
*
RT_DBaseCheckID(String Filename,Int StartIx,var ID1, ... ,var IDn)
This function allows you to compare DBase ID's with some known ID's, eg match clip Width with some stored ID width, ie is it
the clip that DBase was written for. Allows you to simultaneously check multiple ID's, starting at StartIx in the DBase ID array.
Filename, DBase Filename, no default.
StartIx, ID starting array element index range 0 -> RT_DBaseGetID(Filename,-1) - 1.
IDx, Int or Float, The value that ID(StartIx + x) will be checked against (type [int:float] and value must match exactly).
Returns -1, all ID's compared the same, otherwise the index (0 relative) of the ID comparison that failed.
NOTE, if eg StartIx = 4, and return value=4, then ID1 was compared with DBase ID[4] (the first provided arg failed to match DB ID[4])
and that comparison failed to match.

***
***
***

RT_DBaseCmpField(String FileName,int Record_1, int Record_2,int Field,int "Field2",int "Field3",bool "Sig"=true)
Does comparison on same field of two records.
Filename, DBase Filename, no default.
Record_1, first record to use in comparison.
Record_2, second record to use in comparison.
Field, Primary field to use in comparison.
Field2, Optional secondary field to use in comparison (where Primary fields are same).
Field3, Optional ternary field to use in comparison. (where Primary and Secondary fields are same)
Sig, default true. String fields only. If True then does case significant comparison, else case insigificant.
Returns:-
-1, if Record_1 field is less than Record_2 field. (For field type Bool when Record_1 field is false and Record_2 field is true).
1, if Record_1 field is greater than Record_2 field. (For field type Bool when Record_1 field is true and Record_2 field is false).
0, if Record_1 field is same as Record_2 field.

***
***
***

RT_DBaseCocktailSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary Field3.
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
Debug, Default false. Writes time taken to sort to DebugView.

DBase Sorting routine using Cocktail Shaker Sort Algorithm which is a bi-directional BubbleSort, in even passes it bubbles upwards,
and on odd passes bubbles downwards. Result is that if a DBase is already sorted and you add a record to the end of it, then
this sort routine should take no more than 3 passes to update the sort. Cocktail sort was chosen over BubbleSort as it does not
matter if result is to be sorted Ascending or Descending, the bi-directional nature of the algorithm takes care of both.
This, like BubbleSort is very slow on unsorted data, and so should only be used where the data is already sorted, otherwise use
RT_DBaseQuickSort().
When sorting on Bools, True will be sorted as if value 1, and false as if value 0.

***
***
***

RT_DBaseGetStrAttrib(String Filename,Int ix)
Function to return a user String Attribute, or to inquire the number of String attributes, or the maximum string length, currently implemented
in a DBase. RT_DBaseAlloc() initializes all String Attributes to "", ie zero length.
Filename, DBase Filename, no default.
Ix, -1, inquires the number of String attributes currently implemented. (10)
-2, inquires the Maximum string length allowed in string attributes. (1024)
else, String Attribute element index range 0 -> RT_DBaseGetStrAttrib(Filename,-1) - 1.

***
***
***

RT_DBaseGetTypeString(String Filename)
Filename, DBASE Filename, no default.
Returns string that can be used via RT_DBaseAlloc() to recreate similar DBase (same fields and sizes).

***
***
***

RT_DBaseQuickSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary Field3.
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
Debug, Default false. Writes time taken to sort to DebugView.

DBase Sorting routine using QuickSort Algorithm by C.A.R Hoare, modification of Bentley-McIlroy partitioning which was
modified by Robert Sedgewick (Three-Way Partitioning), and here, further modified.
This sort routine is intended for use where records are not already sorted, where already sorted then maybe use RT_DBaseCocktailSort()
instead. This version QuickSort (unlike original C.A.R. Hoare), copes quite well with many similar values in the sort fields and
should be reasonably good/fast considering it is a file sort.
When sorting on Bools, True will be sorted as if value 1, and false as if value 0.

***
***
***

RT_DBaseRecordSwap(String FileName,int Record_1, int Record_2)
Swaps two records in DBase.
Filename, DBase Filename, no default.
Record_1, first record to swap.
Record_2, second record to swap.
Returns 1 if records swapped, else 0 where Record_1 == Record_2 (aint no point in swapping).

Example DBase utility script function:- Req GScript.

Function DBase_BubbleSort(String DB,int "RecS",int "RecE",int "Field",int "Field2",int "Field3",Bool "Sig",Bool "Ascend",Bool "BubbleSink",Bool "Debug") {
/*
DBase_BubbleSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,Bool "BubbleSink"=false,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary Field3.
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
BubbleSink, Default True. Bubble sort direction, True = Downwards (Assume records added at end, less work if previously sorted)
Standard BubbleSort 'bubbles float', this one by default bubbles downwards ie 'bubbles sink'.
This does not affect result, only bubblesort algorithm, if already sorted and new record added to end
of DBase, then BubbleSink=True will sort in no more than 2 passes.
Debug, Default false. Writes time taken to sort to DebugView.
*/
myName="DBase_BubbleSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
BubbleSink= Default (BubbleSink,True) Debug=Default(Debug,False)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
True=True False=False # Locals
nel=RecE-RecS+1 j=0 Swapped=1
incr = (Ascend) ? 1 : -1
GScript("""
While(Swapped>0) {
Swapped = 0
j = j + 1
for(i=0,nel-1-j) {
a = (BubbleSink) ? RecE-1-i : RecS+i
b = a+1
x=RT_DBaseCmpField(DB,a,b,Field,Field2,Field3,sig)
if(x == incr) {Swapped = Swapped + RT_DBaseRecordSwap(DB,a,b)}
}
}
""")
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
}

***
***
***

RT_FileDuplicate(String Filename1,String Filename2,bool "Overwrite"=False)
Makes a copy of the source Filename1, to a second destination Filename2, automatically overwriting existing Filename2 if
Overrwrite is true (default false). Fiails in existing Filanem2 if Overrwrite= false (default).

[StainlessS]

Post 3 of 4



***
***
***

RT_ForceProcess(clip, bool "Video"=True,bool "Audio"=false)
If Video (default true), Force read every frame of clip from 0 to FrameCount-1.
If Audio (default false), Force read every sample of clip from 0 to NumberOfSamples-1.
Useful where a clip outputs some kind of file for use in a second filter, this function would in such a case
forcibly read and therefore write the file, so that it may be available to other filters or to a second pass of the filter that
initially wrote the file.
Returns only after completed force reading of video frames and/or audio samples.
Returns 0.

***
***
***

RT_FrameMovement(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0,
int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
bool "AltScan"=false,Bool "ChromaI"=False,int "Matrix"=(c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600?3:2),
int "BlkW"=64,int "BlkH"=BlkW, Int "OLapX"=BlkW/2, Int "OLapY"=BlkH/2,Float "BlkTh"=1.0,String "Prefix"="")

Returns FLOAT value (0.0 -> 255.0) movement detection between clip c frame n area x,y,w,h, and clip c2 frame n2 area x2,y2,w,h.
The frames are sectioned into blocks of BlkW x BlkH with block overlaps of OLapX and OLapY, and all blocks are differenced, the result
is the greatest difference for any one block comparison.
Is more sensitive to localized movement. Best control of localized movement sensitivity is by changing blk size, lower is more sensitive.
If Altscan is true, then only every other horizontal scanline in BLKH will be scanned.

Args:-
c, c2, comparison clips, usually the same clip.
n, default current_frame, frame from clip c.
n2, default current_frame, frame from clip c2.
ChromaWeight, default 1.0/3.0. Range 0.0 -> 1.0.
Difference method control when comparing blocks.
Y8, returns same as RT_LumaDifference. ChromaWeight ignored.
YUV,
Weighting applied YUV chroma:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
RGB,
If ChromaWeight > 0.0, then returns same as RT_RGBDifference() [average RGB pixel channel difference].
If ChromaWeight == 0.0, then returns same as RT_LumaDifference() using Matrix arg to convert RGB to YUV-Y Luma.
x,y,w,h, all default 0 (full frame c).
x2,y2. x2 defaults x, y2 defaults y, for clip c2.
Altscan, default false. If true then scan only every other scanline starting at clip c y coord, and clip c2 y2 coord.
ChromaI, default false. If YV12 and ChromaI, then do YV12 interlaced chroma scanning.
Ignored if not YV12. Both YV12 clips must be same, cannot difference one that is progressive and one that has interlaced chroma.
Matrix, default (c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600)?3:2
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709.
Default = (c.Width > 1100 OR c.Height>600 OR c2.Width > 1100 OR c2.Height>600) then 3(PC709) else 2(PC601). YUV not used
Matrix only Used if RGB and ChromaWeight == 0.0, where returns same as RT_LumaDifference().
BlkW, default 64, Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
BlkH, default BlkW, Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
OlapX, default BlkW/2, Horizontal block overlap. 4 -> BlkW/2
OlapY, default BlkH/2, Vertical block overlap. 4 -> BlkH/2
BlkTh, default 1.0. Governs extra information returned when Prefix != "", as local variables.
Prefix, default "" is do not set additional local variables.
If set eg "FM_", then sets local variables as below.
"FM_Xoff" = X Coord of block in Clip c that was greatest.
"FM_Yoff" = Y Coord of block in Clip c that was greatest.
"FM_PercAboveTh" = Percentage of Blocks that were above BlkTh.
"FM_BlkAveDf" = Average Block difference.
"FM_nAboveTh" = Number of Blocks that were above BlkTh.
"FM_TotalBlks" = Total Number of Blocks.

NOTE, 'x2' and 'y2' default to 'x' and 'y' respectively.
c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

***
***
***

RT_GetPid()
Returns Int Process ID of the current process.

***
***
***

RT_QueryColor(clip c,int "n"=current_frame,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
\ bool "altscan"=false,int "CDf"=5,float "CTh"=100.0,bool "ChromaI"=false)

Returns, if(CTh==100.0) THEN returns percentage of pixels whose color difference from Gray for frame n, area x,y,w,h is greater than CDf, as a float,
OTHERWISE if (CTh<100.0) returns True/False depending upon whether or not percentage of pixels different from gray by at least CDf, exceeds
CTh percentage threshold, True if it does, ie color.

Altscan (default false), scans every other line if true (eg if y is odd then only odd lines).
height h, is specified same as for altscan=false although it does not matter if height is specified as either odd or
even, internally h is reduced by one if altscan is true and height is even.

CDf, (default 5, range 0 -> 255), Maximum allowed pixel difference from Gray that will NOT be counted (0, is any difference
is counted).

CTh, (default 100.0), % Threshold to calculate number of pixels that are more than CDf away from Gray.
If CTh==100.0%(default), then entire area is scanned and returns percentage of pixels that are more than CDf distant from gray.
If CTh < 100.0%, then may exit early without scanning entire area but only if CTh% pixels are found that are more than
CDf distant from gray, where it will return the percentage of pixels that were scanned and found to be more than CDf distant
from gray. If return value is less than CTh (and CTh != 100.0) then scanned entire area and did not exit early, where you
might interpret as being a grayscale frame/area. This weird arrangement is so that you can cause the function to exit early
where it thinks that the area is indeed color, the result in such case has to be compared with CTh to see how it is to be
interpreted. The purpose of early exit is to get a speedy answer, the comparison with CTh% pixels is only done at the end of
each scanline and not at every pixel. In most cases where there is low CDf and CTh, color frame and no lettebox border, there
should be an early exit within a few scanlines (grayscale frames, will scan entire area).

ChromaI, Default false. If true then scan chroma as for Interlaced YV12. (Only valid for YV12, otherwise ignored).

How it works(roughly, YUV and RGB are processed a little bit differently):-

Let Pixels_h = Actual number of scanlines processed (altscan)
Let Pixels = (w*Pixels_h)
Let PixLim = int(Pixels * CTh / 100.0) # Exit early pixel count limit
Let Cnt = 0
For each valid(altscan) pixel in area:-
if(RGB) {
if(Max(R,G,B)-Mid(R,G,B) > CDf || Mid(R,G,B)-Min(R,G,B) > CDf) Cnt=Cnt+1
} Elseif(YUV) {
if(Abs(u-128)>CDf || Abs(v-128)>CDf) Cnt=Cnt+1
}
At the End of each scanline:-
if(Cnt > Pixlim) then exit pixel loop # exit early if limit exceeded

At function end:-
bret = (cnt > lim)
result min(max((cnt * 100.0 / pixels),0.0),100.0);
if(CTh==100.0) return result;
else return bret;

[StainlessS]

Post 4 of 4

Auto generated by one of the included scripts.



RT_Stats_ORDERED_Function_List v2.00Beta01


There follows a list of all function names together with CPP style argument specifiers that inform
Avisynth the argument types and optional names. Optional arguments have square brackets surrounding
their name as in [name] and are followed by a type specifier character that gives the type.
Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args,
followed by optional 'arg1' of type bool and optional 'arg2' of type int.

# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
###################################


RT_ArrayAlloc "s[Type]i[Dim1]i[Dim2]i[Dim3]i[StringLenMax]i"
RT_ArrayAppend "s."
RT_ArrayCheckID "si.+"
RT_ArrayExtend "s[Add]i"
RT_ArrayGet "si[ix2]i[ix3]i"
RT_ArrayGetAttrib "si"
RT_ArrayGetDim "s[Dim]i"
RT_ArrayGetDim1Max "s[Current]b"
RT_ArrayGetElSize "s"
RT_ArrayGetID "si"
RT_ArrayGetStrAttrib "si"
RT_ArrayGetType "s"
RT_ArraySet "s.i[ix2]i[ix3]i"
RT_ArraySetAttrib "si.+"
RT_ArraySetID "si.+"
RT_ArraySetStrAttrib "sis+"
RT_ArrayTypeName "i"
RT_AverageLuma "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_AvgLumaDif "c[n]i[slices]b[matrix]i"
RT_BitAND "ii"
RT_BitASL "ii"
RT_BitASR "ii"
RT_BitCHG "ii"
RT_BitCLR "ii"
RT_BitLSL "ii"
RT_BitLSR "ii"
RT_BitNOT "i"
RT_BitOR "ii"
RT_BitROL "ii"
RT_BitROR "ii"
RT_BitSET "ii"
RT_BitSetCount "i"
RT_BitTST "ii"
RT_BitXOR "ii"
RT_Call "s[Hide]b[Debug]b"
RT_ChanAve "c[n]i[Prefix]s"
RT_ColorSpaceXMod "c"
RT_ColorSpaceYMod "c[Laced]b"
RT_DBaseAlloc "sis[StringLenMax]i"
RT_DBaseAppend "s.+"
RT_DBaseCheckID "si.+"
RT_DBaseCmpField "siii[Field2]i[Field3]i[Sig]b"
RT_DBaseCocktailSort "s[RecS]i[RecE]i[Field]i[Field2]i[Field3]i[Sig]b[Ascend]b[Debug]b"
RT_DBaseExtend "s[Add]i"
RT_DBaseFields "s"
RT_DBaseFieldSize "si"
RT_DBaseFieldType "si"
RT_DBaseGetAttrib "si"
RT_DBaseGetField "sii"
RT_DBaseGetID "si"
RT_DBaseGetStrAttrib "si"
RT_DBaseGetTypeString "s"
RT_DBaseQuickSort "s[RecS]i[RecE]i[Field]i[Field2]i[Field3]i[Sig]b[Ascend]b[Debug]b"
RT_DBaseRecords "s"
RT_DBaseRecordSize "s"
RT_DBaseRecordsMax "s[Current]b"
RT_DBaseRecordSwap "sii"
RT_DBaseSet "si.+"
RT_DBaseSetAttrib "si.+"
RT_DBaseSetField "sii.+"
RT_DBaseSetID "si.+"
RT_DBaseSetStrAttrib "sis+"
RT_DBaseTypeName "i"
RT_Debug "s+[name]b[repeat]b"
RT_DebugF "s.*[name]s[tabsz]i"
RT_FileDelete "s"
RT_FileDuplicate "ss[Overwrite]b"
RT_FileFindStr "ss[sig]b[pos]i[start]i[lines]i"
RT_FilenameSplit "s[get]i"
RT_FileQueryLines "s"
RT_FileRename "ss"
RT_FindStr "ss[sig]b[pos]i"
RT_FloatAsRGBA "f"
RT_ForceProcess "c[Video]b[Audio]b"
RT_FrameDifference "cc[n]i[n2]i[ChromaWeight]f[x]i[y]i[w]i[h]i[x2]i[y2]i[altscan]b[ChromaI]b[Matrix]i"
RT_FrameMovement "cc[n]i[n2]i[ChromaWeight]f[x]i[y]i[w]i[h]i[x2]i[y2]i[altscan]b[ChromaI]b[Matrix]i[blkw]i[blkh]i[OlapX]i[OLapY]i[BlkTh]f[Prefix]s"
RT_FSelFolder "[title]s[dir]s[debug]b"
RT_FSelOpen "[title]s[dir]s[filt]s[fn]s[multi]b[debug]b"
RT_FSelSaveAs "[title]s[dir]s[filt]s[fn]s[debug]b"
RT_FunctionExist "s"
RT_GetCropDAR "cf[x]f[y]f[w]f[h]f"
RT_GetDAR "cf"
RT_GetFileExtension "s"
RT_GetFileTime "si"
RT_GetFullPathName "s"
RT_GetLastError ""
RT_GetLastErrorString ""
RT_GetPid ""
RT_GetProcessName "[Parent]b[debug]b"
RT_GetSAR "cf"
RT_GetSystemEnv "s"
RT_GetWorkingDir ""
RT_GraphLink "cc+b*"
RT_Hex "i[width]i"
RT_HexValue "s[pos]i"
RT_IncrGlobal "s[Init]i"
RT_InternalFunctions ""
RT_LocalTimeString "[file]b"
RT_LumaCorrelation "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaDifference "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_LumaPixelsDifferent "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[Thresh]i"
RT_LumaPixelsDifferentCount "cc[n]i[delta]i[x]i[y]i[w]i[h]i[n2]i[delta2]i[x2]i[y2]i[Interlaced]b[matrix]i[Thresh]i"
RT_NumberString "i[base]i[width]i"
RT_NumberValue "s[base]i[pos]i"
RT_Ord "s[pos]i"
RT_PluginDir ""
RT_PluginFunctions ""
RT_PluginParam "s"
RT_QueryBorderCrop "c[Samples]i[Thresh]f[Laced]b[xMod]i[yMod]i[wMod]i[hMod]i[Relative]b[Prefix]s[RLBT]i[Debug]b[Ignore]f[Matrix]i[ScanPerc]f"
"[Baffle]i[ScaleAutoThreshRGB]b[ScaleAutoThreshYUV]b[ATM]f[Start]i[End]i[LeftAdd]i[TopAdd]i[RightAdd]i[BotAdd]i[LeftSkip]i"
"[TopSkip]i[RightSkip]i[BotSkip]i"

RT_QueryColor "c[n]i[x]i[y]i[w]i[h]i[altscan]b[CDf]i[CTh]f[ChromaI]b"
RT_QueryLumaMinMax "c[Samples]i[Ignore]f[Prefix]s[Debug]b[X]i[Y]i[W]i[H]i[Matrix]i[Start]i[End]i"
RT_QuoteStr "s"
RT_QwikScan "ciciss[LumaTol]f[Flags]i[LC]f[LD]f[FD]f[PD]f[PC]i[XP]i[ChromaWeight]f[PDThresh]i[MaxDistance]i[Inclusive]b[Prefix]s"
RT_QwikScanCreate "cs[Prev]s[Next]s[Matrix]i[Debug]b"
RT_QwikScanEstimateLumaTol "cc[n]i[n2]i[matrix]i"
RT_ReadTxtFromFile "s[Lines]i[Start]i"
RT_RGB32AsFloat "c[n]i[delta]i[x]i[y]i"
RT_RGB32AsInt "c[n]i[delta]i[x]i[y]i"
RT_RgbChanAve "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanInRange "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[lo]i[hi]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMax "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMedian "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMin "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanMinMaxDifference "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanPNorm "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mu]f[d]i[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanStats "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[Chan]i[lo]i[hi]i[flgs]i[prefix]s[mu]f[d]i[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_RgbChanStdev "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Chan]i[mask]c[maskmin]i[maskmax]i"
RT_RgbInRange "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[Rlo]i[Rhi]i[Glo]i[Ghi]i[Blo]i[Bhi]i"
RT_RGBInRangeLocate "c[n]i[delta]i[x]i[y]i[w]i[h]i[baffle]i[thresh]f[Rlo]i[Rhi]i[Glo]i[Ghi]i[Blo]i[Bhi]i[prefix]s[debug]b[baffle_w]i[baffle_h]i"
RT_ScriptDir ""
RT_ScriptFile ""
RT_ScriptName ""
RT_SignalDAR "f"
RT_SignalDAR2 "ii"
RT_Stats "c"
RT_StrAddStr "ss+"
RT_String "s.*[esc]i"
RT_StrPad "si[c]s"
RT_StrReplace "sss[sig]b"
RT_StrReplaceDeep "sss[sig]b"
RT_StrReplaceMulti "sss[sig]b"
RT_Subtitle "cs.*[align]i[x]i[y]i[vcent]b[expx]b[expy]b[esc]i"
RT_Timer ""
RT_TimerHP ""
RT_TxtAddStr "ss+"
RT_TxtFindStr "ss[sig]b[pos]i[start]i[lines]i"
RT_TxtGetLine "s[Line]i"
RT_TxtQueryLines "s"
RT_TxtSort "s[mode]i"
RT_TxtWriteFile "ss[append]b"
RT_Undefined ""
RT_VarExist "s"
RT_VarIsSame "..+[sig]b"
RT_VarType "."
RT_Version ""
RT_VersionDll ""
RT_VersionString ""
RT_WriteFile "ss.*[Append]b"
RT_WriteFileList "ss[append]b"
RT_YankChain "c[n]i[delta]i"
RT_YDifference "c[n]i[delta]i[x]i[y]i[w]i[h]i[x2]i[y2]i[Interlaced]b[matrix]i"
RT_YInRange "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[lo]i[hi]i[mask]c[maskmin]i[maskmax]i"
RT_YInRangeLocate "c[n]i[delta]i[x]i[y]i[w]i[h]i[baffle]i[thresh]f[lo]i[hi]i[matrix]i[prefix]s[debug]b[baffle_w]i[baffle_h]i"
RT_YPlaneMax "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMedian "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMin "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneMinMaxDifference "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPlaneStdev "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mask]c[maskmin]i[maskmax]i"
RT_YPNorm "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[matrix]i[mu]f[d]i[p]i[u]i[mask]c[maskmin]i[maskmax]i"
RT_YStats "c[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[matrix]i[lo]i[hi]i[flgs]i[prefix]s[mu]f[d]i[p]i[u]i[mask]c[maskmin]i[maskmax]i"

[StainlessS]

In case someone would like to have a play with DBase sorting scripts (or maybe alter for use with AVSLib Arrays or whatever).

Here, Bubblesort as already given in docs, Cocktail Shaker sort and QuickSort (as implemented in RT_stats v2.0), and an Insert sort.


Function DBase_BubbleSort(String DB,int "RecS",int "RecE",
\ int "Field",int "Field2",Int "Field3",
\ Bool "Sig",Bool "Ascend",Bool "BubbleSink",Bool "Debug") {
/*
DBase_BubbleSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,Bool "BubbleSink"=false,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary field3
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
BubbleSink, Default True. Bubble sort direction, True = Downwards (Assume records added at end, less work if previously sorted)
Standard BubbleSort 'bubbles float', this one by default bubbles downwards ie 'bubbles sink'.
This does not affect result, only bubblesort algorithm, if already sorted and new record added to end
of DBase, then BubbleSink=True will sort in no more than 2 passes.

*/
myName="DBase_BubbleSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
BubbleSink= Default (BubbleSink,True) Debug=Default(Debug,False)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
True=True False=False # Locals
nel=RecE-RecS+1 j=0 Swapped=1
incr = (Ascend) ? 1 : -1
GScript("""
While(Swapped>0) {
Swapped = 0
j = j + 1
for(i=0,nel-1-j) {
a = (BubbleSink) ? RecE-1-i : RecS+i
b = a+1
x = RT_DBaseCmpField(DB,a,b,Field,Field2,field3,sig)
if(x == incr) {Swapped = Swapped + RT_DBaseRecordSwap(DB,a,b)}
}
}
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
""")
}



Function DBase_CocktailSort(String DB,int "RecS",int "RecE",
\ int "Field",int "Field2",Int "Field3",
\ Bool "Sig",Bool "Ascend",Bool "Debug") {
/*
DBase_CocktailSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=False)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary field3
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
*/
myName="DBase_CocktailSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
Debug=Default(Debug,False)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
True=True False=False # Locals
incr = (Ascend) ? 1 : -1
low =recS high=recE
swapped = 1
GScript("""
while(swapped>0) {
swapped = 0
for (i=low,high-1) {
x = RT_DBaseCmpField(DB,i,i+1,Field,Field2,field3,sig)
if(x == incr) {
swapped = swapped + RT_DBaseRecordSwap(DB,i,i+1)
}
}
if(swapped>0) {
high = high - 1
swapped = 0
for (i=high-1,low,-1) {
x = RT_DBaseCmpField(DB,i,i+1,Field,Field2,field3,sig)
if(x == incr) {
swapped = swapped + RT_DBaseRecordSwap(DB,i,i+1)
}
}
low=low+1
}
}
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
""")
}



Function DBase_InsertSort(String DB,int "RecS",int "RecE",
\ int "Field",int "Field2",Int "Field3",
\ Bool "Sig",Bool "Ascend",Bool "Debug") {
/*
DBase_InsertSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=False)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary field3
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
*/
myName="DBase_InsertSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
Debug=Default(Debug,False)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
True=True False=False # Locals
nel=RecE-RecS+1
incr = (Ascend) ? 1 : -1
GScript("""
for(i=1,nel-1) {
j = i
x = incr
while(j > 0 && x==incr) {
b = RecS + j
a = b - 1
x = RT_DBaseCmpField(DB,a,b,Field,Field2,field3,sig)
if(x==incr) {RT_DBaseRecordSwap(DB,a,b)}
j = j - 1
}
}
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
""")
}

[StainlessS]

Too big for prev post, here Quicksort in script.

Function DBase_QuickSort(String DB,int "RecS",int "RecE",
\ int "Field",int "Field2",Int "Field3",
\ Bool "Sig",Bool "Ascend",Bool "Debug") {
/*
DBase_QuickSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary field3
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.

QuickSort Three-Way Partitioning.
Modification of Bentley-McIlroy partitioning which was modified by Robert Sedgewick, and further modified.
Chosen (modified) pivot is low + (high-low)/2, which is swapped with low record and comparisons done on that and not
on a copy of pivot record in memory (possibly up to max 262,144KB, 1024 fields of 256KB each).

Any comments should be taken as if sort ascending = true.

Three or less records use Static sort (modified), otherwise,

At first four partitions are created:
some of the elements that are equal to the pivot;
the element that are "less" than the pivot;
the elements that are "greater" than the pivot;
the rest of the elements that are equal to the pivot.

later, outer elements equal to (and including) pivot are swapped into centre (final position) and then
the smaller sized greater than or lesser than partition is recursively sorted, and the greater sized
iteratively sorted (modified) .
*/
myName="DBase_QuickSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
Debug=Default(Debug,False)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
True=True False=False # Locals
GScript("""
Function QuickSortLo(String DB,int low,int high,int Field,int Field2,Int Field3,Bool Sig,int incr) {
while(high > low) {
nel = high - low + 1
if(nel <= 3) { # less than 3 elements, static sort.
if(RT_DBaseCmpField(DB,low,low+1,Field,Field2,Field3,Sig) == incr) {RT_DBaseRecordSwap(DB,low,low+1)}
if(nel == 3) {
if(RT_DBaseCmpField(DB,low+1,low+2,Field,Field2,Field3,Sig)== incr) {RT_DBaseRecordSwap(DB,low+1,low+2)}
if(RT_DBaseCmpField(DB,low, low+1,Field,Field2,Field3,Sig)== incr) {RT_DBaseRecordSwap(DB,low,low+1)}
}
high=low # terminate while(high > low)
} else {
pivot = low+(high-low)/2 # pivot element that we will use
RT_DBaseRecordSwap(DB,low,pivot) # Swap pivot 'out of the way'
pivot = low # We moved pivot to low
i=low+1 # Start at Pivot + 1
j=high
index1=i-1
index2=j+1
DOBREAK=False
while (i <= j && !DOBREAK) {
x=incr
while(x == incr && i <= high) {
x=RT_DBaseCmpField(DB,pivot,i,Field,Field2,Field3,Sig)
if(x==incr) {i=i+1}
}
x=incr
while(x == incr && j>pivot) {
x=RT_DBaseCmpField(DB,j,pivot,Field,Field2,Field3,Sig)
if(x == incr) {j=j-1}
}
if(i < j) {
if(RT_DBaseCmpField(DB,i,pivot,Field,Field2,Field3,Sig)!=0) {
if(RT_DBaseCmpField(DB,j,pivot,Field,Field2,Field3,Sig)!=0) {
RT_DBaseRecordSwap(DB,i,j) # db[i] != db[pivot] and db[j] != db[pivot]
} else {
RT_DBaseRecordSwap(DB,i,j) # db[i] != db[pivot] and db[j] == db[pivot]
index1=index1+1
RT_DBaseRecordSwap(DB,index1,i) # swap original db[j]<->db[index1]
}
} else if(RT_DBaseCmpField(DB,j,pivot,Field,Field2,Field3,Sig)!=0) { # db[i] == db[pivot] db[j] != db[pivot]
RT_DBaseRecordSwap(DB,i,j)
index2=index2-1
RT_DBaseRecordSwap(DB,j,index2) # swap original db[i]<->db[index2]
} else { # db[i] == db[pivot] and db[j] == db[pivot]
index1=index1+1
RT_DBaseRecordSwap(DB,index1,i) # swap db[i] with less than pivot group
index2=index2-1
RT_DBaseRecordSwap(DB,j,index2) # swap db[j] with greater than pivot group
}
i=i+1
j=j-1
} else if(i == j) {
i=i+1
j=j-1
DOBREAK=true
}
} # End while (i <= j && !DOBREAK)
# Here if ascend: DB[low]->DB[Index1]==Pivot : DB[Index1+1]->DB[j]<Pivot : DB[i]->DB[index2-1]>Pivot : DB[index2]->DB[high]==Pivot
# Swap low same as pivot group (including original pivot) into center, swapping with less than pivot middle group.
for(k = low,index1) {RT_DBaseRecordSwap(DB,k,j) j=j-1}
# Swap high same as pivot group into center, swapping with greater than pivot middle group.
for(k = high,index2,-1) {RT_DBaseRecordSwap(DB,i,k) i=i+1}
# Here if ascend, DB[low]->DB[j] < pivot : DB[j+1]->DB[i-1] same as pivot : DB[i]->DB[high] > pivot
nelLo = j-low + 1
nelHi = high-i+ 1
# Recurse smaller partition, iterate larger (center, 'same as pivot' partition already in final position).
if(nelHi>NelLo) {
QuickSortLo(DB,low,j ,Field,Field2,Field3,Sig,incr) # Recursive
low=i # iterate greater than pivot partition
}
else {
QuickSortLo(DB,i ,high,Field,Field2,Field3,Sig,incr) # Recursive
high=j # iterate less than pivot partition
}
} # End if(nel<=3)
} # End While(low < high)
} # End Function QuickSortLo
incr = (Ascend) ? 1 : -1
QuickSortLo(DB,RecS,RecE,Field,Field2,Field3,Sig,incr)
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
""")
}


EDIT: Oops, had used RT_DBaseCmpField(DB,j,pivot,Field,Field2,Field3,Sig) without the Sig arg, fixed.

[StainlessS]

Note, the "SecurityCamExtractMotion.AVS" previously posted here:- http://forum.doom9.org/showthread.php?p=1628675#post1628675
is much better, included in current RT_Stats 2.00 Beta01 (MediaFire or Sendspace in sig below).

Marker shows block with greatest amount of motion.

https://s20.postimg.cc/ws8glsgal/SCMD_zpsjj5cduzt.png (https://postimg.cc/image/hjij80mm1/)

https://s20.postimg.cc/5ix37af7h/SCMD2_zpsiuzq505m.png (https://postimg.cc/image/uc6n7xy7t/)

MotDetect.avs not given (included in RT_Stats v2.00 and useless without it)

############################################
# SecurityCamExtractMotion.AVS, v2.0, By StainlessS.
# Requires RT_Stats v2.0, FrameSel v1.03, GScript, Grunt, mt_tools_2.
#
# Clip, Planar, YUY2 or RGB clip, YV411 not supported due to non support in Overlay.
# (RGB, will likely require different thresholds)
#
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN MotDetect.
#
# MotDetect() creates four .txt command files for KEEP frames (movement), DROP frames (not movement), DUPE frames and HOLD frames.
# HOLD frames are combination of all KEEP frames and, DROP frames where block with greatest difference is in close proximity to previous
# KEEP frame block with greatest difference (ie there seems to be possible movement very near previously identified movement).
# Using SelectFrames() you can select frames types of each of the 4 command files, or using the Reject mode of FrameSel, you can
# instead return the frames NOT in the command file, so using the 4 created command files, you can extract 8 variations of frame types
# after a single detection scan.
#
############################################
Th = 6.5 # Greater or equal KEEP (movement), below is DROP(or DUPE, or possibly HOLD).
# Need check against source, block size and almost all other settings.
#
dTH = 3.0 # Below or equal is DUPE, 0.0 <= dTh < Th (above dTh and below Th is DROP [or possibly HOLD]).
#
BLKW = 64 # Suggest about 16 -> 64 (even only, mod 4 else probably slower). Metrics will change with block size.
BLKH = BLKW # Localized movement, adjust for smallest size of moving objects to detect, makes only little difference to speed,
# bigger, may be a bit faster.
# For human target, suggest box encompassing head + shoulders of most distant human.
# RT_FrameMovement returns greatest average pixel difference for any one of the blocks.
# Block searches may be overlapped by up to half a block both horizontally and vertically and so total x,y,w,h area
# of each pair of frames is compared up to about 4 times due to the half block overlaps.

#
X = 16 # Area Of Interest, avoid eg passing traffic or timecode (or could use eg Overlay to blot out stuff for dclip).
Y = 16 # Suggest cut out eg sky unless looking for flying saucers (less to search, faster).
W = -16 # As for Crop. Best MOD 4 only
H = -16
#
CW = 0.0 # ChromaWeight, 0.0->1.0. 0.0 = Luma ONLY, about 0.1 suggested for color (luma ONLY faster for YUV).
# For RGB, Matrix used where CW==0.0 to convert to Luma-Y. If CW != 0.0, then returns average pixel differences,
# (apdR+apdG+apdB)/3.0.
# Perhaps always CW=0.0 [luma only] for YUV night captures.
# Can convert RGB to YUV for Luma-Y detect and SelectFrames() using original RGB, if required.
#
AltScan=false # If True, Scan only every other raster line starting from Y (maybe faster, especially YUV Luma scan only).
#
ChromaI=False # If True, YV12 chroma is interlaced (ignored if not YV12 or if Luma ONLY scan [CW==0.0]).
#
Matrix=(Width>1100||Height>600)?3:2 # Matrix used for RGB conversion to Luma-Y.
# Default, If (Width>1100 || Height>600) then 3(PC.709), else 2(PC.601). (0=Rec.601, 1=Rec.709)
# NOTE, default PClevels conversions will likely require different thresholds to YUV because of the
# greater differences in Y (after RGB-Luma-Y conversion), probably about (255.0/219.0) times greater.
#
OLapX= BlkW/2 # X Block Overlap. 0->BLKW/2 (bigger more accurate, slower)
OLapY= BlkH/2 # Y Block Overlap. 0->BLKH/2
# Where YUV, CW==0.0 and AltScan==True, might try eg OLapY=(BlkH/2)-1, where would be faster due to AltScan and,
# also in overlapped 'Y_Block' scan will then scan alternate scanlines that were not scanned in previous non overlapped
# scan (if that makes any sense, ie even scanlines sampled in first block, odd lines sampled in overlapped block scan).

LAG = 1 # Lag (default 1, ie no lag. 1 or more). Compares current frame n with frame n-Lag. Detection is always
# initially compared(n,n-1), if neither DUPE nor KEEP detected and Lag>1, then will redetect using comparison(n,n-Lag)
# where n-lag frame is limited to no earlier than the most recent KEEP frame.
# A moderate Lag (up to about 50 or so), may be more sensitive to very small amount of movement,
# it introduces a time delay between compared frames. There is no upper limit to Lag arg.
# NOTE, Lag > 1 will likely produce more KEEP and HOLD frames (ie identify more movement), and also a bit slower
# due to 2nd detection scan where initial scan identified DROP frame (ie DROP may be converted to KEEP or HOLD).
# Lag will not make any huge difference to results, small amount of movement at beginning of sequence of movements may be
# detected, use lag > 1 only where every possible movement frame is required.

SHOWDCLIP=True # If True, then shows DClip in MotDetect rather than source clip (Only useful if dclip pre-processed in some fashion)
#
dclip=Last # Detection clip.
# Or some kind of denoise or whatever, used for detection (visible in ShowMetrics if SHOWDCLIp==True).
# Use SPATIAL denoise ONLY.
#dclip=dclip.MaxContrast(Samples=64,Gamma=1.0).VagueDenoiser(threshold=0.8,method=1,nsteps=6,chromaT=0,interlaced=false) # SLOWISH
#dclip=dclip.MaxContrast(Samples=64,Gamma=1.0).hqdn3d(2.0, 0.0,0.0,0.0,0) # A bit FASTER

###
### Show Metrics ONLY
DIM=True # Dim/darken area outside of the x,y,w,h search area. A little faster with Dim==False when showing metrics only.
SHOWMISS=true # Show block 'Marker' for greatest moving DROP block (miss), (markers never shown for dupes).
MISSCOLOR =$AAFF00 # Color, block 'Marker' for greatest DROP block (when SHOWMISS==True).
HITCOLOR =$FFFF44 # Color, block 'Marker' for greatest KEEP block (hit).
HOLDCOLOR =$FF8800 # Color, block 'Marker' for HOLD block (Miss converted to HIT due to Miss proximity to previous Hit).
VERBOSE=True # Show additional metrics, Th, dTh, BLK Args
###
### Frame Extraction ONLY
SHOW =True # Show FrameSel() Info on frame when extracting frames (ie shows '66] Using 12345').
SHOWTIME=True # Show original clip Time on Frame.
#
# # Name of command frames file returned By AutoGetFrames and SelectFrames. (Keep.Txt, Drop.txt, Dupe.txt, Hold.txt)
ReturnFN="Keep.Txt"
#ReturnFN="Hold.Txt"
#ReturnFN="Drop.Txt"
#ReturnFN="Dupe.Txt"
#
REJECT = False # If True, then return frames NOT in RetunFN rather than frames in ReturnFN.
###

##########################################
# Choose 1 of below options:- (uncomment)
##########################################

# Show metrics (Multi-Instance capable) to get best Threshold 'th' (will create valid frames files if playing straight through).
MotDetect(Last,Th,dTH,BLKW,BLKH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OLapX,OLapY,LAG,dclip,Dim,SHOWMISS,MISSCOLOR,HITCOLOR,HOLDCOLOR,SHOWDCLIP,VERBOSE)

# Make Frames Cmd files for SelectFrames(), Must Play all way through (no metrics shown ie faster).
#MotDetect(Last,Th,dTH,BLKW,BLKH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OLapX,OLapY,LAG,dclip,Hide=True)

# Make Frames command files and then extract ReturnFN frames (Keep.txt default) automatically in 1 pass (Auto scans entire file before display).
# Fastest possible scanning, faster than eg VDub video analysis pass as no video shown at all during scan, Speed FPS result shown via DebugView.
#AutoGetFrames(Last,Th,dTh,BlkW,BLKH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OlapX,OlapY,Lag,dclip,show=SHOW,showTime=SHOWTIME,ReturnFN=ReturnFN,Reject=REJECT)

# Return Frames using EXISTING ReturnFN frames Command file (after making ReturnFN using one of above options)
#SelectFrames(Last,ReturnFN,show=SHOW,showTime=SHOWTIME,reject=REJECT)

[luigizaninoni]

a 64-bit version of the dll does not exist, does it ? I'd like to try AWB script with AVS+ 64-bit, but rt_stats 64 would be required. And CallCmd too

[StainlessS]

Sorry, I'm still using XP32 and not likely to go Windows 64 at any time in future (Linux probable next step).

EDIT: I will though take a look at all source and see if I can spot any problem areas that may result on 64 bit compile.
EDIT: I currently have Linux Mint 64 bit installed, but requires M$ compiler as RT_ is CPP interface dll.
(Does M$ VS Express run under Wine ? )

[StainlessS]

Here some string parsing functions,


Function ChrIsNul(String S) {return RT_Ord(S)== 0} # End of String
Function ChrIsTAB(String S) {return RT_Ord(S)== 9} # TAB
Function ChrIsLF(String S) {return RT_Ord(S)==10} # Line Feed (New Line)
Function ChrIsCR(String S) {return RT_Ord(S)==13} # Carriage Return
Function ChrIsSPACE(String S) {return RT_Ord(S)==32} # SPACE
Function ChrIsBang(String S) {return RT_Ord(S)==33} # !
Function ChrIsDQuote(String S) {return RT_Ord(S)==34} # "
Function ChrIsHash(String S) {return RT_Ord(S)==35} # #
Function ChrIsDollar(String S) {return RT_Ord(S)==36} # $
Function ChrIsPercent(String S) {return RT_Ord(S)==37} # %
Function ChrIsAmp(String S) {return RT_Ord(S)==38} # &
Function ChrIsSQuote(String S) {return RT_Ord(S)==39} # '
Function ChrIsOParen(String S) {return RT_Ord(S)==40} # (
Function ChrIsCParen(String S) {return RT_Ord(S)==41} # )
Function ChrIsStar(String S) {return RT_Ord(S)==42} # *
Function ChrIsPlus(String S) {return RT_Ord(S)==43} # +
Function ChrIsComma(String S) {return RT_Ord(S)==44} # ,
Function ChrIsHyphen(String S) {return RT_Ord(S)==45} # -
Function ChrIsDot(String S) {return RT_Ord(S)==46} # .
Function ChrIsSlash(String S) {return RT_Ord(S)==47} # /
Function ChrIsColon(String S) {return RT_Ord(S)==58} # :
Function ChrIsSColon(String S) {return RT_Ord(S)==59} # ;
Function ChrIsLT(String S) {return RT_Ord(S)==60} # <
Function ChrIsEqual(String S) {return RT_Ord(S)==61} # =
Function ChrIsGT(String S) {return RT_Ord(S)==62} # >
Function ChrIsQuery(String S) {return RT_Ord(S)==63} # ?
Function ChrIsAt(String S) {return RT_Ord(S)==64} # @
Function ChrIsOSquare(String S) {return RT_Ord(S)==91} # [
Function ChrIsBSlash(String S) {return RT_Ord(S)==92} # /
Function ChrIsCSquare(String S) {return RT_Ord(S)==93} # ]
Function ChrIsHat(String S) {return RT_Ord(S)==94} # ^
Function ChrIsUScore(String S) {return RT_Ord(S)==95} # _
Function ChrIsOCurly(String S) {return RT_Ord(S)==123} # {
Function ChrIsPipe(String S) {return RT_Ord(S)==124} # |
Function ChrIsCCurly(String S) {return RT_Ord(S)==125} # }
Function ChrIsTilde(String S) {return RT_Ord(S)==126} # ~
#
Function ChrIsWhite(String S) {C=RT_Ord(S) return C==32||C>=8&&C<=13} # SPC, BackSPC, TAB, LF, VT, FF, CR
Function ChrIsDigit(String S) {C=RT_Ord(S) return C>=48&&C<=57}
Function ChrIsUCase(String S) {C=RT_Ord(S) return C>=65&&C<=90}
Function ChrIsLCase(String S) {C=RT_Ord(S) return C>=97&&C<=122}
Function ChrIsAlpha(String S) {C=RT_Ord(S) return C>=65&&C<=90||C>=97&&C<=122}
Function ChrIsHex(String S) {C=RT_Ord(S) return C>=48&&C<=57||C>=65&&C<=70||C>=97&&C<=102}

Below require GScript

GScript("""
Function ChrEatWhite(String S) {i=1 C=RT_Ord(S,i) While(C==32||C>=8&&C<=13) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatAlpha(String S) {i=1 C=RT_Ord(S,i) While(C>=65&&C<=90||C>=97&&C<=122) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatDigits(String S) {i=1 C=RT_Ord(S,i) While(C>=48&&C<=57) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatHex(String S) {i=1 C=RT_Ord(S,i) While(C>=48&&C<=57||C>=65&&C<=70||C>=97&&C<=102) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
""")

[StainlessS]

Post #1 of 2.

RT_Stats v2.0Beta02, new version.:- http://www.mediafire.com/file/twc5yzpza9pzo6s/RT_Stats_25%2626_dll_v2.00Beta02_20161002.zip


v2.00Beta1, 06 July 2016. Fixed bug in RT_DebugF(). Arg string containing '\r\n' did not always remove '\r'.
Switch to VS 2008 Compiler.
String replace fn's no longer return error on empty source string, return "".
Added, RT_ForceProcess.
RT_QueryBorderCrop, changed defaults samples=40, Thresh=-40.0, Ignore=0.4.
wmod,hmod limited max 16. Fixed crop centering.
RT_LumaDifference, RT_LumaCorrelation, RT_LumaPixelsDifferent, RT_LumaPixelsDifferentCount,
clips do not have to be same dimensions.
Lots of other changes.
v2.00Beta2, 2nd Oct 2016, Added args Thresh_w, Thresh_h and Rescan to RT_YInRangeLocate, RT_RGBInRangeLocate.
Added Debug arg to RT_ForceProcess.
Replaced RT_DBaseCockTailSort() with RT_DBaseInsertSort(), spotted an obvious improvement to the InsertSort
algo that I had used (was initially no better than CockTailSort).
Modified RT_DBaseQuickSort() algo, added Ins and RandPivot args.
Avoided what I believe is a bug in the supposedly Optimal Sedgewick three-way-partitioning quicksort algo.


Changes since v2.00 Beta 01

Added debug arg.

RT_ForceProcess(clip, bool "Video"=True,bool "Audio"=false,Bool "Debug"=True)
If Video (default true), Force read every frame of clip from 0 to FrameCount-1.
If Audio (default false), Force read every sample of clip from 0 to NumberOfSamples-1.
Useful where a clip outputs some kind of file for use in a second filter, this function would in such a case
forcibly read and therefore write the file, so that it may be available to other filters or to a second pass of the filter that
initially wrote the file.
Debug, default true. Write Progress to DebugView.
Returns only after completed force reading of video frames and/or audio samples.
Returns 0.


Added function

RT_RandInt(int "RandMax"=$7FFFFFFF)
RandMax default $7FFFFFFF, ie max valid +ve int. Range 1 <= Abs(RandMax) <= $7FFFFFF. (Error if RandMax==0 or $80000000)
Returns:,
RandMax > 0, then result 0 -> RandMax - 1
RandMax < 0, then result -(0 -> Abs(RandMax) - 1)
NOTE, There is no setable seed and and so sequences are NOT repeatable (uses system crypographic random number generator).
Use builtin Rand if require repeatable sequence.


Added args Thresh_w, thresh_h and Rescan.

Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(Width>1100||Height>600?3:2),String "Prefix"="YIRL_",bool "Debug"=false, \
int "Baffle_W"=Baffle,int "Baffle_H"=Baffle,Float "Thresh_w"=Thresh,Float "Thresh_h"=Thresh,Bool "Rescan")

Function to scan frame n+delta for an area where percentage of pixels in luma range Lo to Hi (inclusive) breaks a threshold over at
least Baffle consecutive number of scanlines (all four sides).
Returns True on Successful location else false. On success sets Local var Coords.

RT_YInRangeLocate tests scanlines from outer most scanlines (h and v), towards inner most, stops where it finds Baffle consecutive scanlines
that contain more than Thresh percent of pixels in range Lo to Hi. Object interiors are not tested and so could be hollow.
Would also give +ve result for a disjoint shape like this
--
| |
--
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold. Default is any pixel in the range Lo to Hi will break threshold.
Lo: Default 128, lower bound of pixel values to search for.
Hi: Default 255, upper bound of pixel values to search for.
Matrix: Default for RGB is 3(PC709) if width > 1100 or height > 600 Else 2(PC601) : YUV not used
Prefix: Default "YIRL_", prefix for Local Var coords set on successful location of object, eg YIRL_X, YIRL_Y, YIRL_W, YIRL_H.
Debug: Default false. If true outputs info to debugView.
Baffle_W, Default Baffle. Thickness of Left and Right side vertical edges, silently limited to width W of search area.
Baffle_H, Default Baffle. Thickness of Top and Bottom horizontal edges, silently limited to height H of search area.

RT_Stats v2.00 Beta 02, Added args Thresh_w and Thresh_h, and Bool ReScan.
Say we are scanning to find top horizontal edge. We will scan and count number of white pixels in a horizontal scanline.
If this percenage count EXCEEDS Thresh_w, and the total number of similarly exceeding consecutive scanlines exceeds Baffle_h
(NOTE Baffle_h NOT Baffle_w, although scanlines are horizontal, we are counting the thickness of consecutive lines vertically)
then we have found the top edge. Same thing for bottom edge using same variables Thesh_w and Baffle_h.
Scanning vertical edges uses Thresh_h and Baffle_w.
NOTE, for RT_Stats v2.00 Beta 02, you can also supply eg Thresh_w = -12.0 (minus) which is taken as a pixel count of +12, rather
than as a percentage. This Thresh_x -ve pixel count will have a teeny weeny bit subtracted before use, as the pixel count must be
greater (something like Thresh_w=Abs(Thresh_w)- 1.0/scanline_width). NOTE, the RT_YInRangeLocator() default of 0.0 detects more
than 0.0 pixels in scanline percentage count, ie any pixel in range (lo->hi) breaks the threshold and counted against
baffle.
The new Bool ReScan arg, forces a horizontal edge (top and bottom) rescan if a new vertical edge is found, and a vertical edge (left and right)
rescan if a new horizontal edge is found. If top and bottom scan broke threshold and passed baffle, followed by left and right scans doing
the same, the top and bottom pixel objects that caused the detections could well have 'moved' outside of the detected area, ie left and right
edges moving inwards may leave those objects outside because in the left/right scan they did not satisfy thresh and baffle requirements,
and detection will likely be bad. Rescan forces a further scan until all four sides scanned sides are a detection (or not). (Will not be
any great overhead if the initial detect was good as it will just recan the same scanlines accessed immediately prior). The internal routine
that RT_YInRangeLocate() uses, returns percentage of pixels that are in the required range, each time an edge is detected, the number of
required pixels is recalculated as a percentage of the reduced scanline lengths (whether given as percentage, or as a -ve pixel count).


Function RT_RgbInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "RLo"=128,int "RHi"=255,int "GLo"=128,int "GHi"=255,int "BLo"=128,int "BHi"=255, \
String "Prefix"="RGBIRL_",bool "Debug"=false,int "Baffle_W"=Baffle,int "Baffle_H"=Baffle,Float "Thresh_w"=Thresh,Float "Thresh_h"=Thresh,Bool "Rescan")

Function to scan frame n+delta for an area where percentage of pixels in all 3 RGB channels range cLo to cHi (inclusive, c==R or G or B)
breaks a threshold over at least Baffle consecutive number of scanlines (all four sides).
Returns True on Successful location else false. On success sets Local var Coords.

RT_RgbInRangeLocate() tests scanlines from outer most scanlines (h and v), towards inner most, stops where it finds Baffle consecutive scanlines
that contain more than Thresh percent of pixels in range cLo to cHi. Object interiors are not tested and so could be hollow.
Would also give +ve result for a disjoint shape like this
--
| |
--
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold.
Default is any pixel with all three channels in range cLo to cHi, will break threshold.
RLo: Default 128, lower bound of pixel values of Red Channel to search for.
RHi: Default 255, upper bound of pixel values to Red Channel search for.
GLo: Default 128, lower bound of pixel values of Green Channel to search for.
GHi: Default 255, upper bound of pixel values to Green Channel search for.
BLo: Default 128, lower bound of pixel values of Blue Channel to search for.
BHi: Default 255, upper bound of pixel values to Blue Channel search for.
Prefix: Default "RGBIRL_", prefix for Local Var coords set on successful location of object, eg RGBIRL_X, RGBIRL_Y, RGBIRL_W, RGBIRL_H.
Debug: Default false. If true outputs info to debugView.
Baffle_W, Default Baffle. Thickness of Left and Right side vertical edges, silently limited to width W of search area.
Baffle_H, Default Baffle. Thickness of Top and Bottom horizontal edges, silently limited to height H of search area.

RT_Stats v2.00 Beta 02, Added args Thresh_w and Thresh_h, and Bool ReScan.
Say we are scanning to find top horizontal edge. We will scan and count number of white pixels in a horizontal scanline.
If this percenage count EXCEEDS Thresh_w, and the total number of similarly exceeding consecutive scanlines exceeds Baffle_h
(NOTE Baffle_h NOT Baffle_w, although scanlines are horizontal, we are counting the thickness of consecutive lines vertically)
then we have found the top edge. Same thing for bottom edge using same variables Thesh_w and Baffle_h.
Scanning vertical edges uses Thresh_h and Baffle_w.
NOTE, for RT_Stats v2.00 Beta 02, you can also supply eg Thresh_w = -12.0 (minus) which is taken as a pixel count of +12, rather
than as a percentage. This Thresh_x -ve pixel count will have a teeny weeny bit subtracted before use, as the pixel count must be
greater (something like Thresh_w=Abs(Thresh_w)- 1.0/scanline_width). NOTE, the RT_YInRangeLocator() default of 0.0 detects more
than 0.0 pixels in scanline percentage count, ie any pixel R,G,B components are in range (Xlo->Xhi) breaks the threshold and counted against
baffle.
The new Bool ReScan arg, forces a horizontal edge (top and bottom) rescan if a new vertical edge is found, and a vertical edge (left and right)
rescan if a new horizontal edge is found. If top and bottom scan broke threshold and passed baffle, followed by left and right scans doing
the same, the top and bottom pixel objects that caused the detections could well have 'moved' outside of the detected area, ie left and right
edges moving inwards may leave those objects outside because in the left/right scan they did not satisfy thresh and baffle requirements,
and detection will likely be bad. Rescan forces a further scan until all four sides scanned sides are a detection (or not). (Will not be
any great overhead if the initial detect was good as it will just recan the same scanlines accessed immediately prior). The internal routine
that RT_YInRangeLocate() uses, returns percentage of pixels that are in the required range, each time an edge is detected, the number of
required pixels is recalculated as a percentage of the reduced scanline lengths (whether given as percentage, or as a -ve pixel count).


Replacing RT_DBaseCocktailSort.

RT_DBaseInsertSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary Field3.
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
Debug, Default false. Writes time taken to sort to DebugView.

DBase Sorting routine using InsertSort Sort Algorithm.
This, like BubbleSort is very slow on unsorted data, and so should only be used where the data is already sorted (or nearly sorted), otherwise use
RT_DBaseQuickSort().
When sorting on Bools, True will be sorted as if value 1, and false as if value 0.


Added Ins and RandPivot args.

RT_DBaseQuickSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=false,int "Ins",Bool "RandPivot")
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary Field3.
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
Debug, Default false. Writes time taken to sort to DebugView.
ins, default -1. if (RecE-RecS+1) <= ins, then use InsertSort instead of QuickSort on recursive partitions.
Default -1, does not use InsertSort, QuickSort only.
RandPivot, default false. If default false then for Pivot use RecS+(RecE-RecS)/2 ELSE use RecS+RT_RandInt(RecE-RecS+1).


DBase Sorting routine using QuickSort Algorithm by C.A.R Hoare, modification of Bentley-McIlroy partitioning which was
modified by Robert Sedgewick (Three-Way Partitioning), and here, further modified.
This sort routine is intended for use where records are not already sorted, where already sorted then maybe use RT_DBaseInsertSort()
instead. This version QuickSort (unlike original C.A.R. Hoare), copes quite well with many similar values in the sort fields and
should be reasonably good/fast considering that it is a file sort.
When sorting on Bools, True will be sorted as if value 1, and false as if value 0.

[StainlessS]

Post #2 of 2.

Modified InsertSort algo, (script equivalent, optimization change to original posted version in blue)

Function DBase_InsertSort(String DB,int "RecS",int "RecE",
\ int "Field",int "Field2",Int "Field3",
\ Bool "Sig",Bool "Ascend",Bool "Debug") {
/*
DBase_InsertSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=False)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary field3
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
*/
myName="DBase_InsertSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
Debug=Default(Debug,False)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
incr = (Ascend) ? 1 : -1 low=RecS high=RecE
GScript("""
for(i=low+1,high) {
j = i x = incr
while(j > low && x==incr) {
x = RT_DBaseCmpField(DB,j-1,j,Field,Field2,field3,sig)
if(x==incr) {RT_DBaseRecordSwap(DB,j - 1, j) j = j - 1 }
else {j = low } # Swap not needed, Abort inner pass, everything before is already sorted.
}
}
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
""")
}


Modified quicksort used (script version).

Function DBase_QuickSort(String DB,int "RecS",int "RecE",
\ int "Field",int "Field2",Int "Field3",
\ Bool "Sig",Bool "Ascend",Bool "Debug",int "ins",bool "RandPivot") {
/*
DBase_QuickSort(String DB,int "RecS"=0,int "RecE"=DB.Records-1,int "Field"=0,int "Field2"=Field,Int "Field3"=Field2,
\ Bool "Sig"=true,Bool "Ascend"=true,bool "Debug"=false)
Sort DBase records, primarily by field Field, and optionally by secondary field Field2, and ternary field3
String DB, Filename of DBase file.
RecS. default 0. First record to sort.
RecE. default DB.Records-1. Last record to sort.
Field, Default 0. Primary sort field.
Field2, Default Field(not used). Secondary sort field. (if Field of both records SAME, then also compares secondary Field2 fields)
Field3, Default Field2(not used). Ternary sort field (if also Secondary Field2 fields SAME then compare Field3 fields).
Sig, Default True. String fields are Case Significant.
Ascend, default True. Sort Ascending.
ins, default -1. (high-low+1) <= ins, then use InsertSort instead of QuickSort on recursive partitions.
Default -1, does not use InsertSort, quickSort only.
RandPivot, default false. If default false then for Pivot use low+(high-low)/2 ELSE use low+RT_RandInt(high-low+1).

QuickSort Three-Way Partitioning.
Modification of Bentley-McIlroy partitioning which was modified by Robert Sedgewick, and further modified (ssS).
Chosen Pivot is either low+RandInt(high-low+1) OR low + (high-low)/2, depending upon status of RandPivot bool.
which is swapped with low record and comparisons done on that and not
on a copy of pivot record in memory (possibly up to max 262,144KB, 1024 fields of 256KB each).

Any comments should be taken as if sort ascending = true.

ins or less records use Static sort, otherwise,

At first four partitions are created:
some of the elements that are equal to the pivot;
the element that are "less" than the pivot;
the elements that are "greater" than the pivot;
the rest of the elements that are equal to the pivot.

later, outer elements equal to (and including) pivot are swapped into centre (final position) and then
the smaller sized greater than or lesser than partition is recursively sorted, and the greater sized
iteratively sorted (modified) .
*/
myName="DBase_QuickSort: " Assert(Exist(DB),RT_String("%s Dbase Does NOT exist '%s'",myName,DB))
Records = RT_DBaseRecords(DB) Fields = RT_DBaseFields(DB)
RecS = Default (RecS,0) RecE = Default (RecE,Records-1)
Field = Default (Field,0) Field2 = Default (Field2,Field) Field3 = Default (Field3,Field2)
Sig = Default (Sig,True) Ascend = Default (Ascend,True)
Debug=Default(Debug,False) ins=Default(ins,-1) RandPivot=Default(RandPivot,false)
Field3 = (Field3==Field2) ? -1 : Field3 # If Field3 == Field2 then switch OFF ternary comparison field
Field2 = (Field2==Field) ? -1 : Field2 # If Field2 == Field then switch OFF secondary comparison field
Assert(RecE>=0 && RecE < Records, RT_String("%s Invalid RecE(%d)", myName,RecE))
Assert(RecS>=0 && RecS <= RecE, RT_String("%s Invalid RecS(%d)", myName,RecS))
Assert(Field>=0 && Field < Fields, RT_String("%s Invalid Field(%d)", myName,Field))
Assert(Field2>=-1 && Field2 < Fields, RT_String("%s Invalid Field2(%d)", myName,Field2))
Assert(Field3!=Field && Field3>=-1 && Field3 < Fields,RT_String("%s Invalid Field3(%d)", myName,Field3))
TStart = RT_TimerHP()
True=True False=False # Locals
GScript("""
Function QuickSortLo(String DB,int low,int high,int Field,int Field2,Int Field3,Bool Sig,int incr,ins,randPivot) {
while(high > low) { # while 2 or more records
Escape=False
nel = high - low + 1
if(nel <= ins) { # ins or less records, Insert sort.
for(i=low+1,high) {
j = i x = incr
while(j > low && x==incr) {
x = RT_DBaseCmpField(DB,j-1,j,Field,Field2,field3,sig)
if(x==incr) {RT_DBaseRecordSwap(DB,j - 1, j) j = j - 1}
else {j=low} # Swap not needed, Abort inner pass, everything before is already sorted.
}
}
high=low # terminate while(high > low)
} else {
# pivot element that we will use for partitioning (Threeway, below : equal : above)
pivot = (randPivot) ? low+RT_RandInt(nel) : low+(high-low)/2
RT_DBaseRecordSwap(DB,low,pivot) # Swap pivot 'out of the way'
pivot = low # We moved pivot to low
i=low+1 # Start at Pivot + 1
j=high
p=i-1
q=j+1
DOBREAK=False
while (i <= j && !DOBREAK) {
iStat=incr
while(iStat == incr && i <= high) { # Until DB[i] >= DB[pivot]
iStat=RT_DBaseCmpField(DB,pivot,i,Field,Field2,Field3,Sig)
if(iStat==incr) {i=i+1}
}
jStat=incr
while(jStat == incr && j > pivot) { # Until DB[j] <= DB[Pivot]
jStat=RT_DBaseCmpField(DB,j,pivot,Field,Field2,Field3,Sig)
if(jStat == incr) {j=j-1}
}
if(i < j) {
if(iStat==0 && jStat == 0) { # db[i] == db[pivot] and db[j] == db[pivot] (both correct side)
p=p+1 RT_DBaseRecordSwap(DB,p,i) # swap db[i] with less than pivot group
q=q-1 RT_DBaseRecordSwap(DB,j,q) # swap db[j] with greater than pivot group
} else { # One or both DV[i],DB[j] NOT equal to DB[Pivot]
RT_DBaseRecordSwap(DB,i,j) # Swap DB[i] & DB[j] (move the not equal one/both to correct side)
if(iStat==0 && jStat!=0) {
q=q-1 RT_DBaseRecordSwap(DB,j,q) # swap original db[i], now db[j]<->db[q]
} else if(iStat!=0 && jStat==0) {
p=p+1 RT_DBaseRecordSwap(DB,p,i) # swap original db[j], now db[i]<->db[p]
} # Else if both DB[i]!=DB[Pivot] and DB[j]!=DB{Pivot] then have already swapped i,j to correct sides.
}
i=i+1
j=j-1
} else if(i == j) {
i=i+1
j=j-1
DOBREAK=True
# If i WAS equal to j, then possible that ALL items are same as pivot,
# we check for number above and below pivot to see if we can avoid Partition by 4 to Partition by 3 moves.
# If all items are same as pivot, then we are done and can escape.
if(j<p+1 && q-1<i) {
Escape=True
high = low # DONE
}
}
} # End while (i <= j && !DOBREAK)
if(!Escape) {
# Here if ascend: DB[low]->DB[p]==Pivot : DB[p+1]->DB[j]<Pivot : DB[i]->DB[q-1]>Pivot : DB[q]->DB[high]==Pivot
# Swap low same as pivot group (including original pivot) into center, swapping with less than pivot middle group.
for(k = low,p) {RT_DBaseRecordSwap(DB,k,j) j=j-1}
# Swap high same as pivot group into center, swapping with greater than pivot middle group.
for(k = high,q,-1) {RT_DBaseRecordSwap(DB,i,k) i=i+1}
# Here if ascend, DB[low]->DB[j] < pivot : DB[j+1]->DB[i-1] same as pivot : DB[i]->DB[high] > pivot
nelLo = j-low + 1
nelHi = high-i+ 1
# Recurse smaller partition, iterate larger (center, 'same as pivot' partition already in final position).
if(nelHi>nelLo) {
if(nelLo>=2) {
QuickSortLo(DB,low,j ,Field,Field2,Field3,Sig,incr,ins,randPivot) # Recursive
}
low=i # iterate greater than pivot partition
} else {
if(nelHi>=2) {
QuickSortLo(DB,i ,high,Field,Field2,Field3,Sig,incr,ins,randPivot) # Recursive
}
high=j # iterate less than pivot partition
}
}
} # End if(nel <= ins) { # ins or less records, Insert sort.
} # End While(low < high)
} # End Function QuickSortLo
incr = (Ascend) ? 1 : -1
QuickSortLo(DB,RecS,RecE,Field,Field2,Field3,Sig,incr,ins,randPivot)
if(Debug) {T = RT_TimerHP() - TStart RT_DebugF("Time = %.2fSecs (%.2fMins)",T,T/60.0,name=myName)}
""")
}



The Sedgewick three-way-partition algo that I found (PDF at top of google search):- https://www.google.com/search?q=sedgewick+optimal&source=hp&ei=dy7BYa6SDevUgwf-tIKoAg&iflsig=ALs-wAMAAAAAYcE8h-w3hcWRRZArHrNYSzNZAp0zX75W&ved=0ahUKEwjuqYq63_P0AhVr6uAKHX6aACUQ4dUDCAk&uact=5&oq=sedgewick+optimal&gs_lcp=Cgdnd3Mtd2l6EANQ0QRYwipgwixoAXAAeACAAZ4BiAGxBJIBAzAuNJgBAKABAqABAbABAA&sclient=gws-wiz

Seems to be bugged, and although it arrives at the correct result, the partitioning is flawed (probably resulting in more comparisons and swaps than necessary).
We avoid this suspected bug.

Here on website GeeksForGeeks:- http://www.geeksforgeeks.org/3-way-quicksort/
is the Sedgewick algo. See my last post for link to a runnable demo (runs CPP source over Internet and shows results).
The demo shows the partitioning error.
The 3rd line of demo result shows the partitioning error [the 3 way partitioning]

4 4 4 4 4 4 9 9 4 9 9 4 // partition output ( ??? )

EDIT: Should be

4 4 4 4 4 4 4 4 9 9 9 9 // partition output ( correct )


Here a direct link to the demo page:- http://code.geeksforgeeks.org/5vCqyw

click on "> Run" near the bottom of that page.

EDIT: RT_DBaseQuicksort sorts 100,000 randomly sorted records of 1 field which are single character strings, in about 9 seconds on 2.4Ghz core duo, sata 2.0.
The RT_DBaseInsertSort sorts a similar but already sorted DBase in less than a second (I think, EDIT: actually 1.2secs).
Not bad considering there will be many similar values and that it is a file sort.
The original C.A.R. Hoare quicksort would cope quite badly with many similar values (common in real world problems).

EDIT: RT_DBaseQuicksort on 100,000 records with 3 randomly generated sort fields,
Field 1 is 3 character string Letters (A-Z,a-z),
2nd sort field is int (0->32767),
3rd sort field float (0.0->32767.0),
Sort ascending, not case significant, default Ins and RandPivot.
Sort Time 26.734 secs.

EDIT: Almost exact same time (26.292secs) if int field 1st, then float field, then string field.

[StainlessS]

RT_Stats v2.0 Beta 3, New Version, See MediaFire in sig below this post.

Addition


RT_Sleep(Float time)
time in seconds, 0.0 -> 60.0. Silently limited to that range. Accurate to about 0.001 seconds.
RT_Sleep(0.0) would not sleep for any specific time, it would just give opportunity to any waiting
tasks to do a bit of processing, before returning to the current task.


Mod Additions in Blue

RT_QwikScanCreate(clip c,string ARR,string "Prev"="",string "Next"="",int "matrix"=(Width>1100||Height>600?3:2),bool "Debug"=False,
\ int "X"=0,int "Y"=0,int "W"=0,int "H"=0)
Creates an Array set used to quickly scan a clip for frame matching, Arrays used by RT_QwikScan.
c, clip used to create Array frame fingerprint info.
Prev, Array filename, used to find frames prior to a 'start frame'.
Next, Array filename, used to find frames after a 'start frame'.
Either Prev or Next (or both) must be given.
Matrix, used to convert RGB to Luma_Y for Fingerprint creation.
Debug, True sends a little info to DebugView window (google).
v2.0, changed from using DBase files to Array files.
Ints, x,y,w,h, are coords to use, avoid eg logos, as for crop(). (eg 32,32,-32,-32, 32 pixels skipped on all sides)

There are some values stored in the Array ID's that can be recovered using RT_ArrayGetID(ARR_name,ID_Index) as type int.
RT_QwikScan will use below values as set in the Array ID's.

ID_Index: Value
6 matrix
7 framecount
8 X
9 Y
10 W
11 H

[StainlessS]

Post #1 of 2.

RT_Stats v2.0 Beta 4, new version (See MediaFire in sig below this post).


v2.00Beta1, 06 July 2016. Fixed bug in RT_DebugF(). Arg string containing '\r\n' did not always remove '\r'.
Switch to VS 2008 Compiler.
String replace fn's no longer return error on empty source string, return "".
Added, RT_ForceProcess.
RT_QueryBorderCrop, changed defaults samples=40, Thresh=-40.0, Ignore=0.4.
wmod,hmod limited max 16. Fixed crop centering.
RT_LumaDifference, RT_LumaCorrelation, RT_LumaPixelsDifferent, RT_LumaPixelsDifferentCount,
clips do not have to be same dimensions.
Lots of other changes.
v2.00Beta2, 2nd Oct 2016, Added args Thresh_w, Thresh_h and Rescan to RT_YInRangeLocate, RT_RGBInRangeLocate.
Added Debug arg to RT_ForceProcess.
Replaced RT_DBaseCockTailSort() with RT_DBaseInsertSort(), spotted an obvious improvement to the InsertSort
algo that I had used (was initially no better than CockTailSort).
Modified RT_DBaseQuickSort() algo, added Ins and RandPivot args.
Avoided what I believe is a bug in the supposedly Optimal Sedgewick three-way-partitioning quicksort algo.
v2.0 Beta 4. Bug fix in ARR_Read_Header(), (kicks oneself).
RT_QwikScan() added FM FrameMovement mode.


From RT_FrameMovement(), in brief.

Returns FLOAT value (0.0 -> 255.0) movement detection between clip c frame n area x,y,w,h, and clip c2 frame n2 area x2,y2,w,h.
The frames are sectioned into blocks of BlkW x BlkH with block overlaps of OLapX and OLapY, and all blocks are differenced, the result
is the greatest difference for any one block comparison.
Is more sensitive to localized movement. Best control of localized movement sensitivity is by changing blk size, lower is more sensitive.

RT_QwikScan uses new FM mode to locate frame with least (block maximum) local difference to find frame, eg spot small mouth movement and ignore general clip noise.

From AVS script included in zip (script is ~540 lines, not included here)

Function SirPlant(clip src,clip rep,
\ string srcARR,string srcNxtARR, string repARR,string repNxtARR,
\ Float "Th"=0.90,Float "Th2"=0.75,Float "LumaTol"=8.0,Int "MatchLen"=60,Int "SearchStep"=MatchLen/4,
\ Float "CW"=1.0/3.0,Int "BlkW"=64,Int "BlkH"=BlkW,Int "OLapX"=BlkW/2,int "OLapY"=BlkH/2,
\ Bool "Strict"=True,Int "InsMode"=0,String "Project"="Default",Bool "Debug"=True,Int "Verbosity"=1)

Req:-
RT_Stats v2.0 Beta 4, http://forum.doom9.org/showthread.php?p=1787898#post1787898 (c) StainlessS
Prune, http://forum.doom9.org/showthread.php?t=162446 (c) StainlessS
GScript, http://forum.doom9.org/showthread.php?t=147846 (c) Gavino
DebugView, https://technet.microsoft.com/en-us/sysinternals/debugview.aspx (c) SysInternals (M$).

Function to generate a Prune() Command file to substitute/supplant sequences in src source clip with similar sequences from rep replacement clip.
Both clips must have scenes similarly ordered (same size, Framerate, Colorspace etc). Can Also insert unique ranges in rep clip, into
the src source clip at approximately the correct position (or both replace and insert, with all unique ranges in the single result clip,
perhaps easier to hand edit into correct ordering).

NOTE, RT_QwikScanCreate() which is used to create the srcARR, srcNxtARR, and repARR, and repNxtARR Arrays, sets the Matrix (RGB to Luma-Y)
and X, Y, W, and H args used for RT_LumaCorrelation differencing. This function acquires those values from the ARR array ID's.
The X, Y, W, and H args provided to RT_QwikScanCreate() will be used to avoid eg Logos on one clip that do not exist on other (as for Crop).

For every rep replacement frame, it searches the entire src source clip (from current src clip position) for a frame that best matches (initially
via RT_QwikScan FM mode to locate BEST MATCH, and then via RT_LumaCorrelation Th arg to see if match is good enough), this is Sync1 frame, if
Sync1+1 ie Sync2 frames (also via Th) also match, then tries to match remainder of frames (both forwards and backwards using Th2) and the
resulting range length is compared to MatchLen. If sufficiently long, then that range will be replaced. That is basically it. The function creates
a Prune() Command file which is used to splice the sequences together. Note, Prune will Trim/Splice BOTH video and Audio.
(It also alternately does a reverse scan to find current position src frame in rep clip, in between above rep scan failures, doing about same as above).

Args:-
src, source clip.

rep, replacement/substitute clip (probably better quality clip of similar dimensions/colorspace and spatially in sync with src).

srcArr, RT_Stats Array filename, for array as created by RT_QwikScanCreate(), eg
RT_QwikScanCreate(src,"MySrc.ARR","","MySrcNxt.ARR",Matrix=Matrix,x=X,y=Y,w=W,h=H)
where
Matrix defaults (src.Width>1100||src.Height>600)?3:2) # 3=PC709, 2=PC601
X,Y,W,H are crop style coords (ignore eg logo).

srcNxtARR, RT_Stats Array filename, for array as created by RT_QwikScanCreate() as in above example.
Used for fast forward searching and locating of frames using RT_QwikScan().

repArr, RT_Stats Array filename, for array as created by RT_QwikScanCreate(), eg
RT_QwikScanCreate(rep,"MySrc.ARR","","MySrcNxt.ARR",Matrix=Matrix,x=X,y=Y,w=W,h=H)
where
Matrix defaults (src.Width>1100||src.Height>600)?3:2) # 3=PC709, 2=PC601
X,Y,W,H are crop style coords (ignore eg logo).

repNxtARR, RT_Stats Array filename, for array as created by RT_QwikScanCreate() as in above example.
Used for fast forward searching and locating of frames using RT_QwikScan().

Th, Default 0.90. LumaCorrelation threshold, greater or equal to this are considered similar frames. Th is used to find 'Sync' frames, and
Th should be greater than Th2. Range 0.0 < Th <= 1.0.
0.0 = not correlated, 1.0 = full correlation.

Th2, Default 0.75. LumaCorrelation threshold, greater or equal to this are considered similar frames, used for all remaining frames that are not
'Sync' frames. Range 0.0 < Th2 <= Th.

LumaTol, Default 8.0. Range 0.0->16.0. See RT_Stats RT_QwikScan docs. Suggest 16.0 if significant differences in brightness/contrast between clips,
lower is faster but may not find some frames if significant difference in brightness between src and rep clips.
Small increase in LumaTol results in significant speed decrease.
If you end up with duplicate ranges in result clip, then LumaTol is likely set too low, it cannot find the frames via RT_QwikScan fingerprint,
and so sees the frames as not matching.
When Verbosity=2, and Sync1 and Sync2 are shown high (In DebugView), but below Th, then Th is set too high to match frames.

MatchLen, Default = 60. Minimum number of frames (of source clip, when Strict=False) that must match, otherwise not replaced. Range 16 or more.

SearchStep, Default MatchLen/4 (1 -> MatchLen). When frame search fails, the search frame number is incremented by this, and search retried. When matching frame
is found, will scan both forwards and backwards, re-doing frames that were stepped over. Higher faster, but possibility of missing some
matches.

CW Default 1.0/3.0 (0.0 -> 1.0),
ChromaWeight, 0.0 = Luma differencing only, else weighting for pair of chroma channels.
If RGB, 0.0 converts to Luma-Y using Matrix arg as used in creating Scan Arrays. If 0.0 < CW, then instead it differences using
(AvergeRedPixelDifference + AvergeGreenPixelDifference + AvergeBluePixelDifference) / 3.0.
The CW ChromaWeight arg is used during the (new in RT_Stats v2.0 Beta 4) RT_QwikScan FM FrameMovement mode, See Blk args below.
CW==0.0 will be a little faster for YUV, and perhaps a little slower for RGB, suggest keep at default or close, maybe 0.2 or 0.1.

BlkW, Default 64 (8 <= BlkW, even only, Silently limited to frame dimensions).
Used in the new FM FrameMovement mode of RT_QwikScan.
RT_FrameMovement(), splits a frame into BlkW x BlkH blocks (repeated with overlaps of OLapX and OLapY), the result of RT_FrameMovement
is the greatest average pixel difference between similar positioned blocks in each frame, and so is more sensitive to localised movement
like lips moving, where the rest of the frame stays almost exactly the same, allows us to find the correct Sync frames whilst ignoring
most of the noise between clips.
Once we find a Sync Frame using RT_QwikScan FM mode (and LumaTol arg), we compare found frames with RT_LumaCorrelation Th arg, if greater
or equal, then Sync Frame + 1 also has to match greater or equal to Th. When the pair of Sync Frames match then we do a bi-directional
scan to establish the extent of matching frames, these additional frames have to be greater or equal to Th2 to be considered part of a
matching range. Lower BlkW will be a bit slower. See Also RT_QwikScan.

BlkH, Default BklW. (8 <= BlkH, even only, Silently limited to frame dimensions). See Above BlkW.

OLapX, Default BklW/2. (0 <= BlkW/2, Silently limited to frame dimensions). Block Overlap, See Above BlkW.

OLapY, Default BlkH/2. (0 <= BlkH/2, Silently limited to frame dimensions). Block Overlap, See Above BlkW.

Strict, Default True. If True, then compared ranges must match 1:1, if replacement rep clip has extra frames or fewer frames in places,
then matched ranges will break at those points.
If False, then will try to see if match will be re-established if allowing for fewer or more frames in the replacement clip.
NOTE, if set false, then audio must use audio from the 'Prune'ed clips to maintain audio sync, if the src and rep clips are NOT
of similar volume, then the effect will not be good (change in volume where ranges are replaced, the alternative is out of sync audio).
Suggest audio pre-normalizing of both clips, if Strict=False (Can just use original source audio if Strict=True).

InsMode, Default 0 (0->2).
0) Replace matching ranges.
1) Insert unique extra ranges in rep clip into src, at approximate correct postition (matching ranges not replaced).
2) Replace and Insert extras mode.
When inserting extra ranges into the source clip, we can never be always correct in the ordering (but we do try to predict correct order),
when we have a unique sequence in both clips, we do not know which should be inserted first, indeed one of the sequences may have to
be inserted midway in the other to be correct, we have no way of telling, but do try to match end of sequence frames to foretell
the correct ordering.

Project, Default "Default". Empty string "" will be converted to the default.
Project is the name prefix given to output command and log files. (Below output files using default project).
"Default_Command.txt" # Prune Command file to create result clip.
"Default_UnUsed.txt" # Prune Command file to create clip of 'unused' frames from rep replacement clip.
"Default_SirPlant.LOG" # The log file.

Debug, Default True. Send info to DebugView (Google).

Verbosity, Default 1. (0 -> 2) higher more info when Debug=true.

[StainlessS]

Post #2 of 2.

From DebugView log, also creates file log

00000006 05:50:13.546 SirPlant: ######
00000007 05:50:13.546 SirPlant: #
00000008 05:50:13.546 SirPlant: # Local Start Time = 2016-11-29 05:50:13.546
00000009 05:50:13.546 SirPlant: #
00000010 05:50:13.546 SirPlant: # Th=0.900 Th2=0.750 LumaTol=8.000
00000011 05:50:13.546 SirPlant: #
00000012 05:50:13.546 SirPlant: # MatchLen=60 SearchStep=15 Strict=True Project='Default'
00000013 05:50:13.546 SirPlant: #
00000014 05:50:13.546 SirPlant: # CW=0.33 BlkW=64 BlkH=64 OLapX=32 OLapY=32
00000015 05:50:13.546 SirPlant: #
00000016 05:50:13.546 SirPlant: # InsMode=2 (REPLACE & INSERT EXTRAS)
00000017 05:50:13.546 SirPlant: #
00000018 05:50:13.546 SirPlant: ######
00000019 05:50:13.546 SirPlant:
00000020 05:50:17.968 SirPlant: 0] *** ALIGNMENT SUCCEED *** rep[0,809]->src[99,908]
00000021 05:50:17.968 SirPlant: 0] StartFailCor=-1.000 StartCor=0.939 Sync1= 0.939 Sync2= 0.951 MinCor=0.915 EndCor=0.970 EndFailCor=0.336
00000022 05:50:17.968 SirPlant: 0] Writing Intervening src[0,98] Len=99
00000023 05:50:17.968 SirPlant: 99] Writing matching REPLACEMENT range rep[0,809] -> src[99,908] Len=810
00000024 05:50:19.203 SirPlant: 909] *** ALIGNMENT SUCCEED *** rep[962,1127]->src[909,1074]
00000025 05:50:19.203 SirPlant: 909] StartFailCor=-1.000 StartCor=0.960 Sync1= 0.960 Sync2= 0.960 MinCor=0.955 EndCor=0.967 EndFailCor=0.085
00000026 05:50:19.203 SirPlant: 909] *** Writing seemingly EXTRA range rep[810,961] Len=152
00000027 05:50:19.203 SirPlant: 1061] Writing matching REPLACEMENT range rep[962,1127] -> src[909,1074] Len=166
00000028 05:50:20.765 SirPlant: 1227] *** ALIGNMENT SUCCEED *** rep[1458,1609]->src[1075,1226]
00000029 05:50:20.765 SirPlant: 1227] StartFailCor=-1.000 StartCor=0.980 Sync1= 0.980 Sync2= 0.984 MinCor=0.946 EndCor=0.982 EndFailCor=0.238
00000030 05:50:20.765 SirPlant: 1227] *** Writing seemingly EXTRA range rep[1128,1457] Len=330
00000031 05:50:20.765 SirPlant: 1557] Writing matching REPLACEMENT range rep[1458,1609] -> src[1075,1226] Len=152
00000032 05:50:24.343 SirPlant: 1709] *** ALIGNMENT SUCCEED *** rep[1610,2196]->src[4166,4752]
00000033 05:50:24.343 SirPlant: 1709] StartFailCor=-1.000 StartCor=0.989 Sync1= 0.989 Sync2= 0.989 MinCor=0.978 EndCor=0.981 EndFailCor=0.305
00000034 05:50:24.343 SirPlant: 1709] Writing Intervening src[1227,4165] Len=2939
00000035 05:50:24.343 SirPlant: 4648] Writing matching REPLACEMENT range rep[1610,2196] -> src[4166,4752] Len=587
00000036 05:51:27.312 SirPlant: 5235] *** ALIGNMENT SUCCEED *** rep[2197,2733]->src[7146,7682]
00000037 05:51:27.312 SirPlant: 5235] StartFailCor=-1.000 StartCor=0.974 Sync1= 0.974 Sync2= 0.976 MinCor=0.952 EndCor=0.974 EndFailCor=-0.114
00000038 05:51:27.312 SirPlant: 5235] Writing Intervening src[4753,7145] Len=2393
00000039 05:51:27.312 SirPlant: 7628] Writing matching REPLACEMENT range rep[2197,2733] -> src[7146,7682] Len=537
00000040 05:51:31.718 SirPlant: 8165] *** ALIGNMENT SUCCEED *** rep[3570,3880]->src[7683,7993]
00000041 05:51:31.718 SirPlant: 8165] StartFailCor=-1.000 StartCor=0.988 Sync1= 0.988 Sync2= 0.987 MinCor=0.954 EndCor=0.957 EndFailCor=-0.036
00000042 05:51:31.718 SirPlant: 8165] *** Writing seemingly EXTRA range rep[2734,3569] Len=836
00000043 05:51:31.718 SirPlant: 9001] Writing matching REPLACEMENT range rep[3570,3880] -> src[7683,7993] Len=311
00000049 05:51:42.265 SirPlant: 9312] *** ALIGNMENT SUCCEED *** rep[5138,5390]->src[8725,8977]
00000050 05:51:42.265 SirPlant: 9312] StartFailCor=0.189 StartCor=0.930 Sync1= 0.930 Sync2= 0.930 MinCor=0.755 EndCor=0.758 EndFailCor=0.740
00000051 05:51:42.312 SirPlant: 9312] !!! Following two sections may need hand edit for order !!!
00000052 05:51:42.312 SirPlant: 9312] Writing Intervening src[7994,8724] Len=731
00000053 05:51:42.312 SirPlant: 10043] *** Writing seemingly EXTRA range rep[3881,5137] Len=1257
00000054 05:51:42.312 SirPlant: 11300] Writing matching REPLACEMENT range rep[5138,5390] -> src[8725,8977] Len=253
00000055 05:51:56.265 SirPlant: 11553] *** ALIGNMENT SUCCEED *** rep[5391,6601]->src[8992,10202]
00000056 05:51:56.265 SirPlant: 11553] StartFailCor=-1.000 StartCor=0.993 Sync1= 0.993 Sync2= 0.992 MinCor=0.977 EndCor=0.992 EndFailCor=0.332
00000057 05:51:56.265 SirPlant: 11553] Writing Intervening src[8978,8991] Len=14
00000058 05:51:56.265 SirPlant: 11567] Writing matching REPLACEMENT range rep[5391,6601] -> src[8992,10202] Len=1211
00000065 05:53:47.703 SirPlant: 12778] *** ALIGNMENT SUCCEED *** rep[7504,8637]->src[13107,14240]
00000066 05:53:47.703 SirPlant: 12778] StartFailCor=0.293 StartCor=0.975 Sync1= 0.975 Sync2= 0.971 MinCor=0.881 EndCor=0.981 EndFailCor=0.128
00000067 05:53:47.734 SirPlant: 12778] Writing Intervening src[10203,13106] Len=2904
00000068 05:53:47.734 SirPlant: 15682] *** Writing seemingly EXTRA range rep[6602,7503] Len=902
00000069 05:53:47.734 SirPlant: 16584] Writing matching REPLACEMENT range rep[7504,8637] -> src[13107,14240] Len=1134
00000074 05:54:23.718 SirPlant: 17718] *** ALIGNMENT SUCCEED *** rep[9103,9217]->src[15293,15407]
00000075 05:54:23.718 SirPlant: 17718] StartFailCor=-0.123 StartCor=0.978 Sync1= 0.978 Sync2= 0.974 MinCor=0.953 EndCor=0.963 EndFailCor=0.134
00000076 05:54:23.734 SirPlant: 17718] Writing Intervening src[14241,15292] Len=1052
00000077 05:54:23.734 SirPlant: 18770] *** Writing seemingly EXTRA range rep[8638,9102] Len=465
00000078 05:54:23.734 SirPlant: 19235] Writing matching REPLACEMENT range rep[9103,9217] -> src[15293,15407] Len=115
00000079 05:54:38.484 SirPlant: 19350] *** ALIGNMENT SUCCEED *** rep[9218,9848]->src[18173,18803]
00000080 05:54:38.484 SirPlant: 19350] StartFailCor=-1.000 StartCor=0.961 Sync1= 0.961 Sync2= 0.960 MinCor=0.957 EndCor=0.964 EndFailCor=0.964
00000081 05:54:38.484 SirPlant: 19350] Writing Intervening src[15408,18172] Len=2765
00000082 05:54:38.484 SirPlant: 22115] Writing matching REPLACEMENT range rep[9218,9848] -> src[18173,18803] Len=631
00000083 05:54:38.484 SirPlant: 22746] Writing SOURCE END Range src[18804,19305] Len=502
00000084 05:54:38.484 SirPlant:
00000085 05:54:38.484 SirPlant: ######
00000086 05:54:38.484 SirPlant: # SrcLen= 19306 RepLen=9849 OutLen= 23248 RepMatched=5907 Replaced=5907 ExtrasInserted=3942 RepUnused=0
00000087 05:54:38.484 SirPlant: # Approx Frames Searched=3440675
00000088 05:54:38.484 SirPlant: # Local Stop Time = 2016-11-29 05:54:38.484 : Time Taken=264.926 Secs (4.415 mins)
00000089 05:54:38.484 SirPlant: ######

The "!!!" flagged line, shows where two unique ranges are correctly suspected of crossing over each other, ie one of the ranges should be enclosed in the other.

EDIT: Prune Command file

######
#
# Splice replacements
#
# Prune Command File
#
###
#
# Local Start Time = 2016-11-29 05:50:13.546
#
# Th=0.900 Th2=0.750 LumaTol=8.000
#
# MatchLen=60 SearchStep=15 Strict=True Project='Default'
#
# CW=0.33 BlkW=64 BlkH=64 OLapX=32 OLapY=32
#
# InsMode=2 (REPLACE & INSERT EXTRAS)
#
######
0 0, 98 # 0] Len=99
3 0, 809 # 99] -> src[99,908] Len=810 StartFailCor=-1.000 StartCor=0.939 Sync1=0.939 Sync2=0.951 MinCor=0.915 EndCor=0.970 EndFailCor=0.336
4 810, 961 # 909] Len=152 *** Seemingly EXTRA Replacement frames
3 962, 1127 # 1061] -> src[909,1074] Len=166 StartFailCor=-1.000 StartCor=0.960 Sync1=0.960 Sync2=0.960 MinCor=0.955 EndCor=0.967 EndFailCor=0.085
4 1128, 1457 # 1227] Len=330 *** Seemingly EXTRA Replacement frames
3 1458, 1609 # 1557] -> src[1075,1226] Len=152 StartFailCor=-1.000 StartCor=0.980 Sync1=0.980 Sync2=0.984 MinCor=0.946 EndCor=0.982 EndFailCor=0.238
0 1227, 4165 # 1709] Len=2939
3 1610, 2196 # 4648] -> src[4166,4752] Len=587 StartFailCor=-1.000 StartCor=0.989 Sync1=0.989 Sync2=0.989 MinCor=0.978 EndCor=0.981 EndFailCor=0.305
0 4753, 7145 # 5235] Len=2393
3 2197, 2733 # 7628] -> src[7146,7682] Len=537 StartFailCor=-1.000 StartCor=0.974 Sync1=0.974 Sync2=0.976 MinCor=0.952 EndCor=0.974 EndFailCor=-0.114
4 2734, 3569 # 8165] Len=836 *** Seemingly EXTRA Replacement frames
3 3570, 3880 # 9001] -> src[7683,7993] Len=311 StartFailCor=-1.000 StartCor=0.988 Sync1=0.988 Sync2=0.987 MinCor=0.954 EndCor=0.957 EndFailCor=-0.036
0 7994, 8724 # 9312] Len=731
4 3881, 5137 # 10043] Len=1257 *** Seemingly EXTRA Replacement frames
3 5138, 5390 # 11300] -> src[8725,8977] Len=253 StartFailCor=0.189 StartCor=0.930 Sync1=0.930 Sync2=0.930 MinCor=0.755 EndCor=0.758 EndFailCor=0.740
0 8978, 8991 # 11553] Len=14
3 5391, 6601 # 11567] -> src[8992,10202] Len=1211 StartFailCor=-1.000 StartCor=0.993 Sync1=0.993 Sync2=0.992 MinCor=0.977 EndCor=0.992 EndFailCor=0.332
0 10203, 13106 # 12778] Len=2904
4 6602, 7503 # 15682] Len=902 *** Seemingly EXTRA Replacement frames
3 7504, 8637 # 16584] -> src[13107,14240] Len=1134 StartFailCor=0.293 StartCor=0.975 Sync1=0.975 Sync2=0.971 MinCor=0.881 EndCor=0.981 EndFailCor=0.128
0 14241, 15292 # 17718] Len=1052
4 8638, 9102 # 18770] Len=465 *** Seemingly EXTRA Replacement frames
3 9103, 9217 # 19235] -> src[15293,15407] Len=115 StartFailCor=-0.123 StartCor=0.978 Sync1=0.978 Sync2=0.974 MinCor=0.953 EndCor=0.963 EndFailCor=0.134
0 15408, 18172 # 19350] Len=2765
3 9218, 9848 # 22115] -> src[18173,18803] Len=631 StartFailCor=-1.000 StartCor=0.961 Sync1=0.961 Sync2=0.960 MinCor=0.957 EndCor=0.964 EndFailCor=0.964
2 18804, 19305 # 22746] Len=502


And splicing script, showing what each section is

# SirPlant_Splice.avs

vSrc="Src.avi"
vRep="Rep.avi"

#
Project="Default"
Cmd=Project+"_Command.txt"
SHOW = True
SRC_AUDIO = False # Use only Source audio. ONLY if STRICT and not inserting EXTRA replacement ranges, else out of sync audio likely
STACK = False # Same for STACK, only if Strict and no EXTRA replacement ranges.

A=AVISource(vSrc) B=A C=A
D=AVISource(vRep) E=D F=D

A = (SHOW) ? A.AddBorders(0,0,0,16,$000000) : A
B = (SHOW) ? B.AddBorders(0,0,0,16,$404040) : B
C = (SHOW) ? C.AddBorders(0,0,0,16,$808080) : C
D = (SHOW) ? D.AddBorders(0,0,0,16,$00FF00) : D
E = (SHOW) ? E.AddBorders(0,0,0,16,$FF0000) : E
F = (SHOW) ? F.AddBorders(0,0,0,16,$0000FF) : F

SrcAudio=A.KillVideo

Nicks="""
InterveningSource=0
MatchingSource=1
EndRangeSource=2
MatchingReplacement=3
ExtraReplacement=4
EndRangeReplacement=5
"""


Prune(A,B,C,D,E,F,cmd=Cmd,NickName=Nicks,Show=SHOW)

(SRC_AUDIO) ? AudioDub(SrcAudio) : NOP

return (STACK) ? StackVertical(A,Last) : Last


By the way, for those of you thinking of testing out whether or not Bovril (straight out of the jar) is good if poured all over your keyboard, well it aint. (Makes for a surprisingly good glue)

EDIT: Updated zip (showing no downloads so hopefully no probs there), Changed OLapX and OLapY minimums to 0 (was 4 as per
wrong RT_FrameMovement docs [also updated]). + some tinkering with script.

EDIT: And the client script

# SirPlant_Client.avs

Import("SirPlant.avs")

srcFN = "Src.avi" # Source clip
repFN = "Rep.AVI" # Replace clip, Holds sequences that will replace those in source clip.
# Sequences in Replace clip should be in SAME order as in source clip.
# Clips must be same size, colorspace.
# When splicing sequences with Prune (later) everything must be similar, audio etc,
# also audio should both be normalized for similar voume levels, this detection stage ignores audio.
# In later script, Prune will replace with both video and audio from replace clip.
srcARR = SRCFN+".ARR" # Name of RT_QwikScan FingerPrint Array
srcNxtARR = SRCFN+".Next.ARR" # Name of RT_QwikScan Locator Next Array (Next == scan forward Array)
repARR = REPFN+".ARR" # Name of RT_QwikScan FingerPrint Array
repNxtARR = REPFN+".Next.ARR" # Name of RT_QwikScan Locator Next Array (Next == scan forward Array)
#
src = AVISource(srcFN)
rep = AVISource(repFN)
#
Th = 0.90
Th2 = 0.75
MatchLen = 60 # Minimum sequence length that must match (frames).
SEARCHSTEP = 15 # Minimum sequence length that must match (frames).
STRICT = True # True=strict 1:1 matching of frames, False=allow slight mismatch ie missing/extra frames in replacement clip.
LUMATOL = 8.0
VERBOSITY = 1
INSMODE = 0 # 0=Replace Only, 1=Insert Extras Only, 2=Replace and Insert Extras
BLKW = 64
BLKH = 64
OLAPX = BLKW/2 # Best locator = BlkW/2, Fastest = 0
OLAPY = BLKH/2 # Best locator = BlkH/2, Fastest = 0

X = 4 # Avoid edge noise.
Y = 4
W = -4
H =-96 # Avoid logos. Test clip also had Subtitles with were out by one frame in the rep clip, avoid those too.

### !!! MUST SET TRUE FIRST, THEN FALSE to RUN !!! ###
CREATE = False # Set True to Create QWikScan arrays for SirPlant. (View progress in DebugView)

(CREATE) ? RT_QwikScanCreate(Src,srcARR,"",srcNxtARR,debug=true,x=X,y=Y,w=W,h=H) : NOP # Only need create these once, Re-Create if change coords or clips.
(CREATE) ? RT_QwikScanCreate(rep,RepARR,"",repNxtARR,debug=true,x=X,y=Y,w=W,h=H) : NOP

### !!! USE DEBUGVIEW !!! ###
(!CREATE) ? SirPlant(src,rep,srcARR,srcNxtARR,repARR,repNxtARR,
\ Th=Th,Th2=Th2,MatchLen=MatchLen,SearchStep=SEARCHSTEP,LumaTol=LUMATOL,Strict=STRICT,InsMode=INSMODE,VerBosity=VERBOSITY,
\ Blkw=BLKW,BlkH=BLKH,OLapX=-OLAPX,OLapY=OLAPY
\ ) : NOP

return Messageclip("All Done")

[StainlessS]

RT_Stats v2.0 Beta 5, new version (see MediaFire in sig below this post).


v2.0 Beta 5. Added RT_DBaseReadCSV().


Added

RT_DBaseReadCSV(String DB, String CSV, String "Separator"=",")

DB, FileName of an RT_Stats DBase file.
CSV, FileName of a text CSV (Comma Separated Value) file.
Separator Default "," ie comma separator (first character only, 0, 13 and 10 illegal).

This function will extract CSV values in text file, and append them as records to an appropriately formatted DBase file.
Returns Int, the number of records added to the DBase.
(Maximum line length in CSV text file is 16*1024 characters).

Example:-
CSV.txt file containing text something like this
###
1 ,42.0 ,true , "Hello" # I am a comment
2 ,43.5 ,false, "Goodbye"
$FF,3.142,TRUE , "Coconuts" # So Am I
###
Script:-
DB = "My.DB"
CSV = "CSV.txt"
RT_DBaseAlloc(DB,0,"ifbs32") # Allocate Dbase with 0 pre-allocated records, 4 fields, 0)=Int, 1)=Float, 2)=bool, 3=String(maxlen=32).
Added=RT_DBaseReadCSV(DB,CSV)
Return MessageClip("Added "+String(Added)+" Records") # Shows "Added 3 Records".

[StainlessS]

RT_Stats v2.0 Beta 06, new version, see MediaFire (or SendSpace) in sig below this post.


v2.0 Beta 6. 28 Feb 2017.
RT_DBaseReadCSV(), relaxed Separator requirement if SPACE/TAB separated. Added StrDelimiter string Arg.


Main update

RT_DBaseReadCSV(String DB, String CSV, String "Separator"=",",String "StrDelimiter"="\"")

This function will extract CSV values in text file, and append them as records to an appropriately formatted DBase file.

DB, FileName of an RT_Stats DBase file.
CSV, FileName of a text CSV (Comma Separated Value) file.
Separator Default "," ie comma separator (first character only, Chr()'s 0, 13, 10, and 34 [Double Quote] illegal).
Default Separator "," (Comma) is optional and so long as CSV values are SPACE/TAB separated will not produce an error if separator is missing.
StrDelimiter Default = "\"" ie single character string Double Quote.
StrDelimiter (String Delimiter) can be multiple characters (empty string "" illegal, also cannot be contain the Separator character or
SPACE or TAB), Default is a Double Quote single character string.
CSV string values CANNOT contain StrDelimiter string, ie for default StrDelimiter, strings cannot contain a double quote character string.
So long as the StrDelimiter does not appear within any string, nearly any StrDelimiter can be used, eg "@@@" is a 3 character StrDelimiter
that you could use in place of double quote string delimiters, where the string values could then contain a double quote character, in
such a case you should enclosed CSV strings as in @@@Some text that contains a " double quote.@@@.
NOTE, StrDelimiter IS case sensitive, so if using alphabetic characters they have to match exactly.

Function returns Int, the number of records added to the DBase.
(Maximum line length in CSV text file is 16*1024 characters).

Example:-
CSV.txt file containing text something like this
###
1 ,42.0 ,true , "Hello" # I am a comment
2 ,43.5 ,false, "Goodbye"
$FF,3.142,TRUE , "Coconuts" # So Am I
###
Script:-
DB = "My.DB"
CSV = "CSV.txt"
RT_DBaseAlloc(DB,0,"ifbs32") # Allocate DBase with 0 pre-allocated records, 4 fields, 0)=Int, 1)=Float, 2)=bool, 3=String(maxlen=32).
Added=RT_DBaseReadCSV(DB,CSV)
Return MessageClip("Added "+String(Added)+" Records") # Shows "Added 3 Records".

[martin53]

By the way, for those of you thinking of testing out whether or not Bovril (straight out of the jar) is good if poured all over your keyboard, well it aint.
Dear StainlessS - though some time has passed since that mishap, my hearty compassion! gloating makes for the best laughs, you know that.

[StainlessS]

Dear StainlessS - though some time has passed since that mishap, my hearty compassion! gloating makes for the best laughs, you know that.

Yep, every few weeks I have to fold over a piece of paper to strengthen it, give it a lick and then try to unglue the E and 4 keys. otherwise I keep typing "e4" everywhere :(

A few weeks after it happened (and after posting about it), I found that somehow I had a pool of Bovril in the base of No 1 m/c (I always have the side covers off [and front facia too for some reason]), it had also glued a 1 TB 2.5" external hard drive (very securely) to the base of the m/c case,took about 10 mins to get it free with a screw driver and a little water. :eek:


EDIT: I was bending over the desk behind the monitor trying to reach something, with the jar in hand, bad idea.

EDIT: Just realized that you asked for something via PM at least 6 months ago, forgot sorry. I'll try to find time to dig it out of PM's and

see if I can assist (Same goes for JMac968, same exact thing).

EDIT:
In retrospect, methinks is maybe Jmac698, not 968, sorry, or not, just guessin').

[StainlessS]

RT_Stats v2.0 Beta 07, new version, see MediaFire (or SendSpace) in sig below this post.


v2.0 Beta 5. Added RT_DBaseReadCSV().
v2.0 Beta 6. 28 Feb 2017.
RT_DBaseReadCSV(), relaxed Separator requirement if SPACE/TAB separated. Added StrDelimiter string Arg.
v2.0 Beta 7. 02 Mar 2017.
RT_DBaseReadCSV(), Added StartField and EndField args. Max CSV line length increased to 64*1024 bytes.





RT_DBaseReadCSV(String DB, String CSV, String "Separator"=",",String "StrDelimiter"="\"",Int "StartField"=0,Int "EndField"=last_field)

This function will extract CSV values in text file, and append them as records to an appropriately formatted DBase file.

DB, FileName of an RT_Stats DBase file.
CSV, FileName of a text CSV (Comma Separated Value) file.
Separator Default "," ie comma separator (first character only, Chr()'s 0, 13, 10, and 34 [Double Quote] illegal).
Default Separator "," (Comma) is optional and so long as CSV values are SPACE/TAB separated will not produce an error if separator is missing.
StrDelimiter Default = "\"" ie single character string Double Quote.
StrDelimiter (String Delimiter) can be multiple characters (empty string "" illegal, also cannot contain the Separator character nor
SPACE or TAB), Default is a Double Quote single character string.
CSV string values CANNOT contain StrDelimiter string, ie for default StrDelimiter, strings cannot contain a double quote character string.
So long as the StrDelimiter does not appear within any string, nearly any StrDelimiter can be used, eg "@@@" is a 3 character StrDelimiter
that you could use in place of double quote string delimiters, where the string values could then contain a double quote character, in
such a case you should enclosed CSV strings as in @@@Some text that contains a " double quote.@@@.
NOTE, StrDelimiter IS case sensitive, so if using alphabetic characters they have to match exactly.

StartField Default = 0.
EndField Default = last field in DBase record (By default, all fields should be present in CSV file, else error).
Above StartField and EndField args allow setting of limited number of CONSECUTIVE fields within a record, the remaining
fields if any are set to NULL values, ie String all zero chars, bool = false, Float=0.0, int and bin=0.

Function returns Int, the number of records added to the DBase.
(Maximum line length in CSV text file is 64*1024 characters).

Example:-
CSV.txt file containing text something like this
###
1 ,42.0 ,true , "Hello" # I am a comment
2 ,43.5 ,false, "Goodbye"
$FF,3.142,TRUE , "Coconuts" # So Am I
###
Script:-
DB = "My.DB"
CSV = "CSV.txt"
RT_DBaseAlloc(DB,0,"ifbs32") # Allocate DBase with 0 pre-allocated records, 4 fields, 0)=Int, 1)=Float, 2)=bool, 3=String(maxlen=32).
Added=RT_DBaseReadCSV(DB,CSV)
Return MessageClip("Added "+String(Added)+" Records") # Shows "Added 3 Records".

[StainlessS]

RT_Stats v2.0 Beta 08, new version, see MediaFire (or SendSpace) in sig below this post.


v2.0 Beta 6. 28 Feb 2017.
RT_DBaseReadCSV(), relaxed Separator requirement if SPACE/TAB separated. Added StrDelimiter string Arg.
v2.0 Beta 7. 02 Mar 2017.
RT_DBaseReadCSV(), Added StartField and EndField args. Max CSV line length increased to 64*1024 bytes.
v2.0 Beta 8. 04 Mar 2017.
Added Functions:
RT_DBaseWriteCSV(), RT_DBaseFindSeq() and RT_DBaseRgbColorCount().


RT_DBaseWriteCSV

RT_DBaseWriteCSV(String DB, String CSV, String "Separator"=",",String "StrDelimiter"="\"",Int "StartField"=0,Int "EndField"=last_field
Int "Low"=0,Int "High"=Last_Record,Bool "Append"=False,String "Fmt"="")

This function will write a consecutinve number of fields from a DBase to CSV values in text file, for each DBase record between
records low and high inclusive.

DB, FileName of an RT_Stats DBase file.
CSV, FileName of an output text CSV (Comma Separated Value) file.
Separator Default "," ie comma separator (first character only, Chr()'s 0, 13, 10, and 34 [Double Quote] illegal).
Default Separator "," (Comma).

StrDelimiter Default = "\"" ie single character string Double Quote. String delimiters written around string fields when written to CSV.
StrDelimiter (String Delimiter) can be multiple characters (empty string "" illegal, also cannot contain the Separator
character nor SPACE or TAB), Default is a Double Quote single character string.
DBase string values CANNOT contain StrDelimiter string, ie for default StrDelimiter, strings cannot contain a double quote
character string. So long as the StrDelimiter does not appear within any DB string, nearly any StrDelimiter can be
used, eg "@@@" is a 3 character StrDelimiter that you could use in place of double quote string delimiters, where the
string values could then contain a double quote character, in such a case you could enclose CSV strings as in
@@@Some text that contains a " double quote.@@@.
NOTE, StrDelimiter IS case sensitive, so if using alphabetic characters they have to match exactly.

StartField Default = 0.
EndField Default = last field in DBase record (By default, all fields are written to CSV file).
Above StartField and EndField args allow writing of limited number of CONSECUTIVE fields within a record.

Append Default False. When True, append if CSV file already exists.

Fmt Default, "". The Fmt string provides a little bit of optional formatting for field types Int, Bin and Bool.
The length of Fmt string can be up to EndField-StartField+1, ie 1 character for each of the selected fields.
A single character can be set for each of the fields, first one affects StartField, and the last one affects
EndField (assuming all characters are set). A shorter Fmt string only affects those fields for the characters that
exist in the Fmt string.
A SPACE (' ') or '.' Fmt character uses default formatting behaviour (Proper Case for Bool, ie 1st charcter is upper
case, remainder are lower case).
Type Int, If ' ' or '.' then default eg "-1".
If 'x' then print fields as lower case Hex eg ffffffff.
If 'X' then print fields as upper case Hex eg FFFFFFFF.
If '$' then print fields as upper case Hex preceded by a '$' Dollar character eg eg $FFFFFFFF.
Type Bool. If ' ' or '.' then default eg "True".
if 'l' or 'L' then lower case eg 'true'.
if 'u' or 'U' then upper case eg 'TRUE'.
Type Bin, If ' ' or '.' then default eg "255".
If 'x' then print fields as lower case Hex eg ff.
If 'X' then print fields as upper case Hex eg FF.
If '$' then print fields as upper case Hex preceded by a '$' Dollar character eg eg $FF.
Types String, Float (or internal use type Double) will error if using anything other than '.' or ' '.


Function returns Int, the number of records added to the CSV file.

Example:-
CSVTXT="""
1,42.5,$FF,true, "D:\OUT\Test_000001.d2v" # Comment
2,43.5,$FE,false,"D:\OUT\Test_000002.d2v"
3,44.5,$FD,true, "D:\OUT\Test_000003.d2v"
4,45.5,$FC,false,"D:\OUT\Test_000004.d2v"
-1,45.5,$FC,false,"D:\OUT\Test_000004.d2v"
"""

DB = "My.DB"
CSVIN = "CSVIN.txt"
CSVOUT = "CSVOUT.txt" # Output CSV file.
RT_WriteFile(CSVIN,"%s",CSVTXT) # Create test Input CSV
RT_DBaseAlloc(DB,0,"ifnbs512") # Allocate DBase with 0 pre-allocated records, 5 fields, Int, Float, Bin, Bool and String[512].
Read=RT_DBaseReadCSV(DB,CSVIN)
Written=RT_DBaseWriteCSV(DB,CSVOUT,Append=false,Fmt="$.x..")
S=RT_String("Read = %d written=%d",read,Written) # Shows "Read=5 Written=5".
MessageClip(S)


RT_DBaseFindSeq (same functionality as DB_FindSeq script function here:- https://forum.doom9.org/showthread.php?p=1799116#post1799116 )

RT_DBaseFindSeq(String DB,Int S_Field,Int E_Field,Int Frame,Int "Low"=0,Int "High"=RT_DBaseRecords(DB)-1)

RT_DBaseFindSeq Returns clip (or record) number containing movie frame Frame, using Binary Search.

DB, DBase FileName,
S_Field, DB field that contains the first frame of the record clip/sequence (within entire movie).
E_Field, DB field that contains the Last frame of the record clip/sequence (within entire movie).
Low, Default 0. Lowest record number to scan (likely almost always 0).
High, Default RT_DBaseRecords(DB)-1. Hightest record number to scan (likely almost always as default).

Return of -1 = RECORD NOT FOUND.

eg If you have 4 clips in DBase, each of 10 frames then fields should contain data EXACTLY as below (must be ordered).
Record S_Field E_Field
0 0 9
1 10 19
2 20 29
3 30 39

RT_DBaseRgbColorCount

RT_DBaseRgbColorCount(String DB, clip c,Int "n"=current_frame)
RT_DBaseRgbColorCount, creates an RT_Stats DBase containing data on colors contained a video clip frame.
RGB24 and RGB32 only.
DB, Name of DBase.
c, Clip.
n, Default current_frame, frame to count RGB (will use current_frame in runtime environment).
Returns number of records in DBase, ie number of unique colors on frame n.
DBase Fields:-
ColorField = 0
CountField = 1

Example:-
DB="COLS.DB"
FRAME=0
ColorField = 0
CountField = 1
###
Avisource("OutAvi.Avi").ConvertToRGB24.Trim(FRAME,-1)
Cnt=RT_DBaseRgbColorCount(DB,Last,0) # Count colors of frame 0 (originally FRAME).
RT_DBaseQuickSort(DB,Field=CountField,Field2=ColorField) # Sort primarily by count, and then by color (ascending)
MostCommonColor=RT_DBaseGetField(DB,Cnt-1,ColorField) # Last one in DBase (last record)
MostCommonColorCount=RT_DBaseGetField(DB,Cnt-1,CountField)
RT_Subtitle("Unique Colors=%d\nMost Common Color=$%X (Count=%d)",Cnt,MostCommonColor,MostCommonColorCount)
Overlay(BlankClip(Width=64,Height=64,Color=MostCommonColor,Length=1).LetterBox(2,2,2,2,$FF00FF),x=(width/2)-34,y=(height/2)-34)
return last

[StainlessS]

RT_Stats v2.0 Beta 09, new version, see MediaFire (or SendSpace) in sig below this post.


v2.0 Beta 9. 04 Mar 2017.
Added args to RT_DBaseRgbColorCount(), x,y,w,h, Mask, MaskMin, MaskMax, msb.



RT_DBaseRgbColorCount(String DB, clip c,Int "n"=current_frame,Int "x"=0,Int "y"=0,Int "w"=0,int "h"=0
\ clip "Mask"=unused,Int "MaskMin"=128,int "MaskMax"=255,Int "msb"=8)
RT_DBaseRgbColorCount, creates an RT_Stats DBase containing data on colors contained a video clip frame.
RGB24 and RGB32 only.

DB, Name of DBase.
c, Clip.
n, Default current_frame, frame to count RGB (will use current_frame in runtime environment).
x,y,w,h, All default 0 (full frame). Area to test, as in crop.
Mask, Default unused. PLANAR only, If Mask set, and MaskMin <= MaskLumaPixel <= MaskMax then c pixel is counted, otherwise not.
MaskMin Default 128. See above.
MaskMax Default 255. See above.
msb Default 8. Range 1 -> 8 only.
Most Significan Bits of color primaries. If default, then possible return is up to about 16 million colors
(2 ^ (8 * 3)). If less than default 8, the number of possible colors is reduced so if eg msb=4 then
$FF and $F0 will be counted as the same, as they are both converted to $0F via shifting.
On writing to DBase after scan, would then be converted back up to $F0, again via shift with no rounding.


Returns number of records in DBase, ie number of unique colors on frame n in area x,y,w,h taking Mask args into account.
If using Mask, then good idea to check return value against 0, to see if it found any pixels that were both in the x,y,w,h area
and also complied with Maskmin, MaskMax requirement.

DBase Fields:-
ColorField = 0
CountField = 1

Example:-
DB="COLS.DB"
FRAME=0
ColorField = 0
CountField = 1
###
Avisource("OutAvi.Avi").ConvertToRGB24.Trim(FRAME,-1)
Cnt=RT_DBaseRgbColorCount(DB,Last,0) # Count colors of frame 0 (originally FRAME).
RT_DBaseQuickSort(DB,Field=CountField,Field2=ColorField) # Sort primarily by count, and then by color (ascending)
MostCommonColor=RT_DBaseGetField(DB,Cnt-1,ColorField) # Last one in DBase (last record)
MostCommonColorCount=RT_DBaseGetField(DB,Cnt-1,CountField)
RT_Subtitle("Unique Colors=%d\nMost Common Color=$%X (Count=%d)",Cnt,MostCommonColor,MostCommonColorCount)
Overlay(BlankClip(Width=64,Height=64,Color=MostCommonColor,Length=1).LetterBox(2,2,2,2,$FF00FF),x=(width/2)-34,y=(height/2)-34)
return last

[StainlessS]

Hi guys,

Due to PM, am releasing non (release) prepped version of SirPlant() [first posted here]:-
Find it here:- LINK REMOVED


Not sure but it may need latest RT_Stats.

Remember that I fixed at least one bug in version supplied in RT_Stats AVS folder (v2.0 beta 4).

Dont have time to do anything other than supply avs as it currently stands. (I've used it several times with VGood results).

[StainlessS]

Hi again,
just remembered that I did some kind of test addition to RT_Stats that SirPlant relies upon (one of the two test additions will likely be removed
when I decide which works best), so SirPlant as supplied in previous post may not work, so here new version RT (not prepped for release)
that I am putting up in StainlessS @ Mediafire below in sig, and also on SendSpace.

Hope it works ok :) EDIT: I use it constantly.

PS, the SecurityCamMotionDetect.avs (or whatever its called) has been updated a bit too.

Am removing link in prev post.

[doomwake]

Hi again,
just remembered that I did some kind of test addition to RT_Stats that SirPlant relies upon (one of the two test additions will likely be removed
when I decide which works best), so SirPlant as supplied in previous post may not work, so here new version RT (not prepped for release)
that I am putting up in StainlessS @ Mediafire below in sig, and also on SendSpace.

Hope it works ok :) EDIT: I use it constantly.

PS, the SecurityCamMotionDetect.avs (or whatever its called) has been updated a bit too.

Am removing link in prev post.

Sorry about the PM StainlessS I didn't mean to get on your bad side already. I didn't think you would mind the private messages because i'm only working with a part of RT-Stat and didn't want to take up post here in this thread but if you don't want the PM I will make a separate post about this and post the link when I have it.

Let me know if you would rather post the updates here but I think it would be better in its own thread if you would keep up with it. :):):)

[doomwake]

StainlessS, Mr wizard Please help advance your script a little more if you have a little free time. Its working out great! almost got all the frames besides a few and I'm wondering if you could add part of your FrameRep to get the rest that sirplant misses? If you could maybe add something that looks for frames that still have the logo and replace it with the best match for those frames? since you already have a logo script and SecurityCamExtractMotion script that locks onto a part of the frame and the framerep script. you could have it lock onto the logo and framerep with a frame from the replace clip that best matchs it? I would greatly appraite it and it should finish it and work great for tv rips. I cut out the logo for sirplant but Im wondering if you could at it to the last step SirPlant_Splice?

THank you,
Corey

[StainlessS]

@ Doomwake, I'm a little bit under the weather at the mo, will answser in good time.

Happy to see that your post has now been approved. (some kind of additional "security", I'm guessin.)

[danfgtn]

Since no one seems to want to approve my topic, can I ask a question here about avisynth? This is one of the topics with the most recent answers.

[johnmeyer]

Since no one seems to want to approve my topic, can I ask a question here about avisynth? This is one of the topics with the most recent answers.I think that is a bad idea. It is usually called "hijacking" a thread, which is considered bad form. Also, you won't get the answers you would get from starting your own thread, because people who might be interested in your particular problem may not be reading this thread (I only read about 5% of the posts in this forum).

I'm not sure why what you posted didn't "stick," but I suggest you try again.

[StainlessS]

danfgtn,
Yep best wait for your own thread to be approved, there have recently been other site issues where posts have not been approved:- https://forum.doom9.org/showthread.php?t=174663

Give a moderator a PM about it.

[danfgtn]

I understand, but I accidentally sent the topic twice already, if I send it again, I think it might be a problem. Thank you anyway.

[StainlessS]

Doomwake,

Have been struggling with how best to answer your post.
All of the plugins/scripts you mentioned, do not do what you presume that they do.
The de-logo script (S_ExLogo), only blurs out the area that you tell it to blur, it does no logo detection at all.
FrameRep, replaces frames back into same position that they were originally gotten from, using the same frames command file.
SecurityCamExtractMotion script is nothing whatever to do with what you want.

If the SirPlant script is not working, then (most likely) either args were not best set, or the clips were not in sync.
I would suggest that your only option is to edit manually (replacing the frames that were probably deemed 'Out Of Sync' by SirPlant.
(I have no idea how many, nor whether runs of frames or single frames were skipped).
SirPlant with Strict=False, already allows for every 2nd frame in a sequence to be up to 1 frame out of sync, if editing by hand you would
need to decide how to deal with out of sync frames yourself, and keep in mind that audio will likely end up out of sync if not careful.

Save result clip from SirPlant_Slice.avs (SRC_AUDIO=False, if Strict==False).
Save result clip from SirPlant_ShowNonReplaced.avs, (Frames from replace clip that were not inserted into src clip).

You need to save audio somehow (maybe just to a variable and then audioDub back again later), and simply use trim and splice in hand edit.

Some suggestions for worst case scenario settings


# SirPlant_Client.avs

Import("SirPlant.avs")

srcFN = "S2.avi" # Source clip
repFN = "S1.AVI" # Replace clip, Holds sequences that will replace those in source clip.
# Sequences in Replace clip should be in SAME order as in source clip.
# Clips must be same size, colorspace.
# When splicing sequences with Prune (later) everything must be similar, audio etc,
# also audio should both be normalized for similar voume levels, this detection stage ignores audio.
# In later script, Prune will replace with both video and audio from replace clip.
srcARR = SRCFN+".ARR" # Name of RT_QwikScan FingerPrint Array
srcNxtARR = SRCFN+".Next.ARR" # Name of RT_QwikScan Locator Next Array (Next == scan forward Array)
repARR = REPFN+".ARR" # Name of RT_QwikScan FingerPrint Array
repNxtARR = REPFN+".Next.ARR" # Name of RT_QwikScan Locator Next Array (Next == scan forward Array)
#
src = AVISource(srcFN)
rep = AVISource(repFN)
#
Th = 0.8 # Default 0.9(not close matching frames maybe about 0.8)
Th2 = 0.65 # Default 0.75(not close matching frames maybe about 0.65)
LUMATOL = 12.0 # Default 8.0(not close matching frames maybe up to about 16.0)
MatchLen = 32 # Default 60(Minimum sequence length that must match (frames). Default 60(not close matching frames maybe about 32)
SEARCHSTEP = 4 # Default MatchLen/4(not close matching frames maybe about 4) Frames to step after match failure.
STRICT = False # Default True=strict 1:1 matching of frames, False=allow slight mismatch ie missing/extra frames in replacement clip.
VERBOSITY = 5 # Debug output verbocity level, to DebugView.
INSMODE = 0 # Default 0=Replace Only, 1=Insert Extras Only, 2=Replace and Insert Extras
BLKW = 64 # default 64
BLKH = 64 # default BLKW
OLAPX = 0 # default BLKW/2
OLAPY = 0 # default BLKH/2

X = 4 # Avoid edge noise or logos. (All default 0)
Y = 4
W = -4
H = -4

### !!! MUST SET TRUE FIRST, THEN FALSE to RUN !!! ###
CREATE = False # Set True to Create QWikScan arrays for SirPlant. (View progress in DebugView)

(CREATE) ? RT_QwikScanCreate(Src,srcARR,"",srcNxtARR,debug=true,x=X,y=Y,w=W,h=H) : NOP # Only need create these once, Re-Create if change coords or clips.
(CREATE) ? RT_QwikScanCreate(rep,RepARR,"",repNxtARR,debug=true,x=X,y=Y,w=W,h=H) : NOP

### !!! USE DEBUGVIEW !!! ###
(!CREATE) ? SirPlant(src,rep,srcARR,srcNxtARR,repARR,repNxtARR,
\ Th=Th,Th2=Th2,MatchLen=MatchLen,SearchStep=SEARCHSTEP,LumaTol=LUMATOL,Strict=STRICT,InsMode=INSMODE,VerBosity=VERBOSITY,
\ Blkw=BLKW,BlkH=BLKH,OLapX=-OLAPX,OLapY=OLAPY
\ ) : NOP

return Messageclip("All Done")

You would need set X,Y,W,H coords for logo avoidance, or crop off during detect stage.



# SirPlant_Splice.avs

vSrc="S2.avi"
vRep="S1.avi"

#
Project="Default"
Cmd=Project+"_Command.txt"
SHOW = True
SRC_AUDIO = False # Use only Source audio. ONLY if STRICT and not inserting EXTRA replacement ranges, else out of sync audio likely
# Use SRC_AUDIO = True, Only if STRICT==TRUE at detector stage.

STACK = False # Same for STACK, only True if Strict and no EXTRA replacement ranges.

NORM=(!SHOW && !STACK && !SRC_AUDIO)


A=AVISource(vSrc)
D=AVISource(vRep)

A = (NORM) ? A.Normalize : A
D = (NORM) ? D.Normalize : D

B=A C=A
E=D F=D

A = (SHOW) ? A.AddBorders(0,0,0,16,$000000) : A
B = (SHOW) ? B.AddBorders(0,0,0,16,$404040) : B
C = (SHOW) ? C.AddBorders(0,0,0,16,$808080) : C
D = (SHOW) ? D.AddBorders(0,0,0,16,$00FF00) : D
E = (SHOW) ? E.AddBorders(0,0,0,16,$FF0000) : E
F = (SHOW) ? F.AddBorders(0,0,0,16,$0000FF) : F

SrcAudio=A.KillVideo

Nicks="""
InterveningSource=0
MatchingSource=1
EndRangeSource=2
MatchingReplacement=3
ExtraReplacement=4
EndRangeReplacement=5
"""


Prune(A,B,C,D,E,F,cmd=Cmd,NickName=Nicks,Show=SHOW)

(SRC_AUDIO) ? AudioDub(SrcAudio) : NOP

return (STACK) ? StackVertical(A,Last) : Last




# SirPlant_ShowNonReplaced.avs

repFN="s1.AVI"

Project="Default"

Cmd=Project+"_UnUsed.txt"

A=AVISource(repFN)

Nicks="""
ShortRangeLessThanMinReplace=0
SkippedExtras=1
"""
SHOW=True

Prune(A,A,cmd=Cmd,NickName=Nicks,Show=SHOW)


Good luck

EDIT: You could lower Th, Th2 and MatchLen even lower than suggested, but there is a small chance that the Moon could crash into the Earth tomorrow afternoon, if you do.

EDIT: Examining logs could tell you why frames were not replaced. (DebugView with Verbosity=5, is most comprehensive)

[StainlessS]

v2.0. beta 11, 10 Sept 2017. avs v2.58, v2.6, avs+v2.6 x64 versions, + source. See MediaFire and SendSpace below this post.


v2.0. beta 11, 10 Sept 2017.
Bugfix, RT_DBaseReadCSV write fail on zero length CSV string.


Or here:- http://www.mediafire.com/file/dhrrngn3g4iov1r/RT_Stats_25%2626_dll_v2.00Beta11_20170910.zip

[doomwake]

Hey StainlessS,
I was wondering how;s it going and if you have work on the scripts anymore since the last time we talked?
Also If you have some free time now to work on this with me to perfect it? I have time now if you do just let me know
Thanks,
Corey

Doomwake,

Have been struggling with how best to answer your post.
All of the plugins/scripts you mentioned, do not do what you presume that they do.
The de-logo script (S_ExLogo), only blurs out the area that you tell it to blur, it does no logo detection at all.
FrameRep, replaces frames back into same position that they were originally gotten from, using the same frames command file.
SecurityCamExtractMotion script is nothing whatever to do with what you want.

If the SirPlant script is not working, then (most likely) either args were not best set, or the clips were not in sync.
I would suggest that your only option is to edit manually (replacing the frames that were probably deemed 'Out Of Sync' by SirPlant.
(I have no idea how many, nor whether runs of frames or single frames were skipped).
SirPlant with Strict=False, already allows for every 2nd frame in a sequence to be up to 1 frame out of sync, if editing by hand you would
need to decide how to deal with out of sync frames yourself, and keep in mind that audio will likely end up out of sync if not careful.

Save result clip from SirPlant_Slice.avs (SRC_AUDIO=False, if Strict==False).
Save result clip from SirPlant_ShowNonReplaced.avs, (Frames from replace clip that were not inserted into src clip).

You need to save audio somehow (maybe just to a variable and then audioDub back again later), and simply use trim and splice in hand edit.

Some suggestions for worst case scenario settings

Good luck

EDIT: You could lower Th, Th2 and MatchLen even lower than suggested, but there is a small chance that the Moon could crash into the Earth tomorrow afternoon, if you do.

EDIT: Examining logs could tell you why frames were not replaced. (DebugView with Verbosity=5, is most comprehensive)

[StainlessS]

I have time now

Lucky you !

Got me a gigantic backlog of stuff. No, no further work on SirPlant, nothing more I can think of to improve matters, its all down to settings really,
if frames dont match, out of sync, then they dont match or are out of sync, not a lot I can do about that.
I'm trying to get back to RT_Stats so I can release final v2.0 and maybe bow out from Avisynth, at least spend less time on it, got me other
stuff I've been wanning to do for ages.

ssS

[StainlessS]

PlanetCrop() v1.03, update, runs under either AVS+ or GSCript.
Will be included in RT_Stats v2.0 Beta12

Function PlanetCrop(String "FileName",int "Baffle",Int "BlackRange",int "Grow_W",int "Grow_H",Float "FAR", \
Float "AutoLevelStrength",bool "ShowAutoLevel",bool "KeepMargin",bool "ShowMargin",bool "ShowBorder",bool "Debug",\
int "Matrix",bool "ShowDetectOnly") {
# ------------
VERS="PlanetCrop v1.03 - 02 Aug 2018" # Requires GScript, RT_Stats v1.10.
# ------------
# Given a series of time lapse planetoid images moving across frame (due to earth rotation), locates those images in frame, crops out
# and splices them all together into a video clip of required Frame Aspect Ratio, 1 FPS, with optional Auto Luma Levelling.
# First scans each image establishing maximum dimensions for whole sequence (after cropping out the planets). Optionally 'grows' those
# dimensions to maintain some of the area around the cropped planets. A 'canvas' size is then established to contain the now maximum
# dimensions and respecting the desired Frame Aspect Ratio, synthetic black borders are added where necessary.
# Borders are the synthetic areas added to maintain aspect ratio and can be hi-lited in red. Margins are areas that use the
# original image content instead of synthetic black borders (where possible), they replace synthetic borders, and can be hi-lited
# in green. Margins are the areas between the established crop coords and synthetic borders.
# --------------------------------------
# Args:-
# Filename: Default "*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF". Image Path and wildcard with mulitple pipe separated extensions.
# Baffle: Default 8. Number of scanlines that must break a threshold to be accepted as 'planet'. Avoids noise.
# Suggest about 4 -> 32. Must NOT be greater than diameter of planets (or anywhere near not a good idea).
# BlackRange: Default 32. Where background (sky) can vary in luminance by a significant amount. When creating detection clip, will
# use eqivalent to Levels(YPlaneMin+BlackRange-1,1.0,YPlaneMin+BlackRange,0,255,Coring=False) for detecting planet.
# Grow_W,Grow_H: Default 128. Size in pixels by which to 'grow' the accumulated maximum dimensions once all planets have been identified.
# FAR: Default 4.0/3.0. Frame Aspect Ratio.
# AutoLevelStrength: Default 0.0. Auto luma leveling, range 0.0 -> 1.0. 0.0 is OFF.
# ShowAutoLevel: Default False. If true then shows Auto leveling on right hande side of image ONLY.
# KeepMargin: Default True. When padding to maintain Frame Aspect Ratio, tries to keep image instead of adding synthetic borders.
# ShowMargin: Default False. If True, shows kept margins in Green.
# ShowBorder: Default False. When padding to maintain Frame Aspect Ratio, borders are shown in red.
# Debug: Default False. Show debug info via DebugView. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
# Matrix: Default=2(PC601), 3=PC709. Used in conversion of RGB to YUV-Y. (Dont use 0 or 1, TV levels)
# ShowDetectOnly: Default False. If True, then ONLY returns equivalent Detection clip that would be used, created using BlackRange.
# May assist in selecting BlackRange. NOTE, All images MUST be same size else error as no cropping done. Any non-black in the detect
# clip will be detected as possible planet (if Baffle consecutive scanlines present, then IS planet).
#

myName = "PlanetCrop: "
IsAvsPlus=(FindStr(UCase(versionString),"AVISYNTH+")!=0) HasGScript=RT_FunctionExist("GScript")
Assert(IsAvsPlus || HasGScript,RT_String("%sNeed either GScript or AVS+",myName))

DEBUG_YInRangeLocate=False
FileName = Default(FileName,"*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF")
Baffle=Default(Baffle,8)
BlackRange=Default(BlackRange,32) # YPlaneMin + BlackRange and above, is possible planet
Grow_W=Default(Grow_W,128)
Grow_H=Default(Grow_H,128)
FAR=Float(Default(FAR,4.0/3.0))
AutoLevelStrength=Float(Default(AutoLevelStrength,0.0))
ShowAutoLevel=Default(ShowAutoLevel,false)
KeepMargin=Default(KeepMargin,True)
ShowMargin=Default(ShowMargin,False)
ShowBorder=Default(ShowBorder,False)
DEBUG=Default(DEBUG,False)
Matrix=Default(Matrix,2) # PC601
ShowDetectOnly=Default(ShowDetectOnly,False) #
# Lesser changed stuff Hardcoded
BORDER = $FF0000 # Color for replaced borders.
MARGINBORDER= $00FF00 # Color for KEEP_MARGIN borders.
FPS = 1.0 # Whatever.
WMOD = 2 # For Canvas Width
HMOD = 2 # For Canvas Height
OUT_FILE = "PLANETCROP.List"
Assert(Baffle>0,"PlanetCrop: Baffle must be 1 or more")
Assert(BlackRange>=0,"PlanetCrop: BlackRange must be 1 or more")
Assert(Grow_W>=0,"PlanetCrop: Grow_W Cannot be -ve")
Assert(Grow_H>=0,"PlanetCrop: Grow_H Cannot be -ve")
Assert(FAR>0.0,"PlanetCrop: FAR must be greater than 0.0")
Assert(AutoLevelStrength>=0.0 && AutoLevelStrength <= 1.0,"PlanetCrop: AutoLevelStrength must be 0.0 -> 1.0")
Assert(Matrix>=2 && Matrix<=3,"PlanetCrop: Bad Matrix, suggest 2=PC601,3=PC709")
ShowAutoLevel=(AutoLevelStrength==0.0) ? false : ShowAutoLevel
PCStart=RT_Timer()
#
GS="""
#
if(DEBUG){ RT_Debug("PlanetCrop:",false) RT_Debug("PlanetCrop:",VERS,"- By StainlessS",false) RT_Debug("PlanetCrop:",false) }
picclip = 0 # Dummy for now
result=RT_WriteFileList(FileName,OUT_FILE) # Create a listing file of Pic files
Assert((result!=0), "No files found for "+FileName)
FILELIST=RT_ReadTxtFromFile(OUT_FILE) # Get list of files
NFILES=RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
if(DEBUG){RT_Debug("PlanetCrop:","Found files = " + String(NFILES),false) RT_Debug("PlanetCrop:",false)}
# Prescan to ascertain max wid/hit of all files, also Autocrop & Auto Levels
MAXH=0 MAXW=0 # init max wid and hit
MINLUM=256 MAXLUM= -1
CROPCOORDS="" # Clear results array (string)
AUTOLEVEL=(AutoLevelStrength>0.0) # Adjust using Levels.
if(ShowDetectOnly) {
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic file i
k=ImageReader(FN,end=0).ConvertToRGB32()
k.RT_YStats(0,x=4,y=4,w=-4,h=-4,threshold=0.2,matrix=Matrix,flgs=3,Prefix="PC_") # Get GlobVars PC_yMin & PC_yMax, avoid edges
k=k.Levels(PC_yMin+BlackRange-1,1.0,PC_yMin+BlackRange,0,255,Coring=false)
picclip = (IsClip(picclip)) ? picclip ++ k : k
}
} else {
For(i=0,NFILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic/vid file i
(DEBUG)?RT_Debug("PlanetCrop:",String(i+1)+"/"+String(NFILES)+") Getting File dimensions",FN,false):NOP
k=ImageReader(FN,end=0).ConvertToRGB32()
OrgW=k.width OrgH=k.height
# For QPC
k.RT_YStats(0,x=4,y=4,w=-4,h=-4,threshold=0.2,matrix=Matrix,flgs=3,Prefix="PC_") # Get GlobVars PC_yMin & PC_yMax, avoid edges
if(MINLUM>PC_yMin) {MINLUM=PC_yMin}
if(MAXLUM<PC_yMax) {MAXLUM=PC_yMax}
IRL=k.RT_YInRangeLocate(n=0,Baffle=Baffle,Matrix=Matrix,Prefix="PC_",debug=DEBUG&&DEBUG_YInRangeLocate,lo=PC_yMin+BlackRange-1,hi=255)
Assert(IRL==True,"PlanetCrop: Cannot Locate Planet")
PCS="PC_X="+String(PC_X)+" PC_Y="+String(PC_Y)+" PC_W="+String(PC_W)+" PC_H="+String(PC_H)
CROPCOORDS=RT_TxtAddStr(CROPCOORDS,PCS)
If(MAXW<PC_W) {MAXW=PC_W}
If(MAXH<PC_H) {MAXH=PC_H}
if (DEBUG) { # KEEP main debug messages together
RT_Debug ("PlanetCrop:","Orig Width =",String(orgW),"Orig Height =",String(OrgH),false)
if((PC_W != OrgW) || (PC_H != OrgH)) { # Was Caused by AUTOCROP
RT_Debug("PlanetCrop:","Crop Width =",String(PC_W),"Crop Height =",String(PC_H),false)
}
RT_Debug("PlanetCrop:","MAX Width So Far =",String(MAXW) ,"MAX Height So Far =",String(MAXH),false)
RT_Debug("PlanetCrop:","MIN LUMA So Far =",String(MINLUM)," MAX LUMA So Far =",String(MAXLUM),false)
RT_Debug("PlanetCrop:"," ----------------------------- ",false) # line separator
}
}
if(Grow_W>0 || Grow_H > 0) {
(DEBUG)?RT_Debug("PlanetCrop:","Enlarging Max Width,Height to enlarge canvas",false):NOP
MAXW=MAXW+Grow_W
MAXH=MAXH+Grow_H
}
maxFAR = Float(MAXW) / MAXH
if(FAR >= maxFAR) {
# Required Frame Aspect Ratio greater or same as maximum accumulated dimensions. Need to pad horizontal
canvasheight = Int((MAXH + (HMOD-1)) / HMOD) * HMOD # round UP to next multiple of HMOD
canvaswidth = Int((canvasheight * FAR + (WMOD-1)) / WMOD) * WMOD # round UP to next multiple of WMOD
} else { # Need to pad vertical
canvaswidth = Int((MAXW + (WMOD-1)) / WMOD) * WMOD
canvasheight = Int((canvaswidth / FAR + (HMOD-1)) / HMOD) * HMOD
}
if(DEBUG){
RT_Debug("PlanetCrop:","SETTING CanvasWidth =",String(canvaswidth)," CanvasHeight =",String(canvasheight),false)
RT_Debug("PlanetCrop:","Clip Global MINLUM =",String(MINLUM)," MAXLUM =",String(MAXLUM),false)
RT_Debug("PlanetCrop:"," ----------------------------- ",false) # line separator
}
CSMin = 0 CSMax = 255 Almin = CSMin AlMax=CSMax
LBORDER=BORDER RBORDER=BORDER # Init Default Show border colors
if (AUTOLEVEL || !ShowMargin || !Showborder) { # We need Minlum
if (AUTOLEVEL) { # AutoLevel based on clip global MINLUM & MAXLUM
ALMin = Int(CSMin - ((CSMin - MINLUM) * AutoLevelStrength) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - MAXLUM) * AutoLevelStrength)) # Round down
}
BACKGROUND=(MINLUM*256+MINLUM)*256+MINLUM # Min Luma as RGB color for non autolevel border
if(!Showborder) { # Show CSMin (0) where autoleveled else BACKGROUND (MINLUM)
RBORDER = (AUTOLEVEL) ? CSMin : BACKGROUND # Right half Border
LBORDER = (ShowAutoLevel) ? BACKGROUND : RBORDER # Left half Border
}
}
For(i=0,NFILES-1) { # Cropping and adding borders, Splicing to clip
FN=RT_TxtGetLine(FILELIST,i) # Filename of pic file i
if(DEBUG){RT_Debug("PlanetCrop:",string(i+1)+"/"+String(NFILES)+")","Processing File",FN,false)}
k=ImageReader(FN,end=0).ConvertToRGB32()
OrgW=k.width OrgH=k.height
Eval(RT_TxtGetLine(CROPCOORDS,i))
Left =(canvaswidth-PC_W) / 2 Right = canvaswidth-PC_W - Left # Borders needed to respect FAR
Top =(canvasheight-PC_H) / 2Bot = canvasheight-PC_H-Top
Left2=0 Right2=0 Top2=0 Bot2=0 # Init margin borders
if(KeepMargin) { # Get margins (available image usable instead of artificial border [maintain FAR])
Left2 = ((PC_X<Left) ? PC_X : Left) # margin limited by how much image is available
Left=Left-Left2 # Reduce artificial border by margin
Right2 = (((OrgW-PC_X-PC_W)<Right) ? (OrgW-PC_X-PC_W) : Right) Right=Right-Right2
Top2 = ((PC_Y<Top) ? PC_Y : Top) Top=Top-Top2
Bot2 = (((OrgH-PC_Y-PC_H)<Bot) ? (OrgH-PC_Y-PC_H) : Bot) Bot=Bot-Bot2
if(!ShowMargin) { # Reduce cropping to keep original image in margin area, ELSE AddBorders show green margins.
PC_X=PC_X-Left2 PC_W=PC_W+Left2+Right2 PC_Y=PC_Y-Top2 PC_H=PC_H+Top2+Bot2
Left2=0 Right2=0 Top2=0 Bot2=0 # Not using Margin Addborders
}
}
(DEBUG)? RT_Debug("PlanetCrop:","Crop("+String(PC_X)+","+String(PC_Y)+","+String(PC_W)+","+String(PC_H)+")",false) : NOP
k=k.Crop(PC_X,PC_Y,PC_W,PC_H)
tmpw=(CanvasWidth/2) - Left - Left2 # Canvas centered rather than cropped planet centered
LeftClip = k.Crop(0,0,tmpw,-0)
RightClip= k.Crop(tmpw,0,-0,-0)
if (AUTOLEVEL) {
(DEBUG)?RT_Debug("PlanetCrop:","Levels("+String(ALMin)+",1.0,"+String(AlMax)+","+String(CSMin)+","+String(CSMax)+",Coring=False)",false):NOP
RightClip = RightClip.Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DONT EVER USE Coring
if(!ShowAutoLevel) {
LeftClip = LeftClip.Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DONT EVER USE Coring
}
}
if(KeepMargin && ShowMargin) {
(DEBUG)?RT_Debug("PlanetCrop:","Margin AddBorders("+String(Left2)+","+String(Top2)+","+String(Right2)+","+String(Bot2)+")",false):NOP
LeftClip = LeftClip.Addborders (Left2,Top2,0,Bot2,MARGINBORDER)
RightClip = RightClip.Addborders(0,Top2,Right2,Bot2,MARGINBORDER)
}
(DEBUG) ? RT_Debug("PlanetCrop:","AddBorders("+String(Left)+","+String(Top)+","+String(Right)+","+String(Bot)+")",false) : NOP
LeftClip = LeftClip.Addborders (Left,Top,0,Bot,LBORDER)
RightClip = RightClip.Addborders(0,Top,Right,Bot,RBORDER)
k=StackHorizontal(LeftClip,RightClip)
picclip = (IsClip(picclip)) ? picclip ++ k : k # Append to clip so far
if(DEBUG){
RT_Debug("PlanetCrop:",false)
RT_Debug("PlanetCrop:"," ----------------------------- ",false)
}
}
}
PCEnd=RT_Timer()
if(DEBUG) {
RT_Debug("PlanetCrop:","Deleting",OUT_FILE,false)
RT_Debug("PlanetCrop:","Time Taken="+String(PCEnd-PCStart,"%6.2f")+"Secs",false)
}
RT_FileDelete(OUT_FILE)
"""

HasGScript ? GScript(GS) : Eval(GS) # Use GSCript if installed (loaded plugs override builtin)

return picclip.ConvertToRGB24().assumefps(fps) # Addborders done (bug in AddBorders rgb24)
}

[jmac698]

Thanks, I have used this script a lot. I started modifying it for 16 bit support. Maybe this can be done now with avs+ since it added high bit depth. Will it just work? Does your rt stats support avs+ convention for high bit depth?

[StainlessS]

Does your rt stats support avs+ convention for high bit depth?

Sorry, no, I dont have the first idea how that stuff works, never touched it.

Only change was to enable use with either GScript OR AVS+ with/without GScript.

Post your mods now that you have permanently taken over development (and thanx so very much too) :)

[StainlessS]

Small script function to enclose a string in double quotes (if you dont know what you might want it for, then you probably dont need it).
If the string itself contains double quotes then encapsulates in set of tripple double quotes, otherwise a single pair of double quotes.

EnQuot() for RT_Stats, EnQuot2() for avs v2.60 only (req FillStr v2,.60)


BlankClip(Width=320,height=64,color=-$FF000000)

S=""""We "have' a' " QU"ote or ' two"""

#S=""

Function EnQuot(string S) { Q=RT_StrPad("", S.RT_FindStr(Chr(34))==0 ? 1 : 3 ,Chr(34)) Return Q+S+Q } # Req RT_Stats v1.43+
Function EnQuot2(string S) { Q=FillStr(S.FindStr(Chr(34))==0 ? 1 : 3 ,Chr(34)) Return Q+S+Q } # Requires Avs v2.60 for FillStr

S=S.Enquot # RT
#S=S.Enquot2 # Avs v2.6

Subtitle(S)

Result without EnQuot.
https://i.postimg.cc/C1CXvtBL/En-Quot-Before.jpg (https://postimages.org/)

Result with EnQuot.
https://i.postimg.cc/KvB8DcQn/EnQuot.jpg (https://postimages.org/)

[GillesH]

Hello StainlessS
Until Pinterf finds a solution for the RunTime functions in AVS+, would there be a possibility to use the RT_Stats to do this :
In a clip, simply detect the first duplicate frame and enter its number in a text file.
For example: I have a series of normal and different frames and towards the end, I have a sequence of duplicate frames identical to the last normal image.

Then, I simply want to delete this end of identical frames by a TRIM. But we must identify the first identical frame.
Previously, I did it with FrameEvaluate and YDifferenceToNext.

Can RT_Stats do the same ? Please tell me the procedure.

[StainlessS]

Did you try use eg GScriptClip() instead of ScriptClip when Grunt Installed with MT. ?
Also, FrameEvaluate use GFrameEvaluate() etc.

Or you could do something like this

Colorbars
Function Fn1(clip c,int n) {
c # added for the added x=AverageLuma
current_frame=n
EX =VarExist("current_frame")
# x=AverageLuma # Now works OK in MT
return c.Subtitle("current_Frame Exist = " + string(EX))
}

SSS="""Fn1(current_frame)"""

Last.Scriptclip(SSS)

return last


EDIT: As above, eg builtin AverageLuma or whatever will work OK when you set current_frame yourself.

EDIT: And why oh why do you think that you need MT for detecting duplicate frames ???
DONT USE IT (you probably would end up with dupe frame list with shuffled frame numbers).

Duplicity2 is very good at detecting duplicates, writes frames file[Much better than any YDifferenceFromNext thing], and if required blends/interpolates dupes or replaces with exact dupes.
To remove duplicates with frames file, then can use FrameSel script SelectRanges.avs RejectRanges() Function, also requires Prune if trimming out audio.
EDIT: For RejectRanges(), would not matter if frame numbers shuffled in frames file, FrameSel() would order them first anyway.

RejectRanges()

Function RejectRanges(clip c,String "SCmd",String "Cmd",Bool "TrimAudio",Float "FadeMS") {
# RejectRanges() by StainlessS. Required:- FrameSel, Prune, RT_Stats
# Wrapper to delete frames/ranges along with audio, can supply frames/ranges in SCmd string And/Or Cmd file.
# The wrapper makes for easier usage of Prune() which supports up to 256 input clips, but requires a clip index,
# eg '3, 100,200' would specify clip 3, range 100 to 200. The wrapper does away with the necessity for the clip index as we
# are only using a single clip here. Prune also does not have a 'reject' arg to delete specified frames rather than select them,
# this wrapper also converts a list of frames to delete into a list of frames to select so that we can use Prune and its audio
# capability.
#
# SCmd: Frames/Ranges specified in String (Frames/Ranges either Chr(10) or ';' separated, infix ',' specifies range, eg 'start,end').
# Cmd: Frames/Ranges specified in file (one frame/range per line, comments also allowed, see FrameSel for Further info).
# TrimAudio:
# True(default), deletes audio belonging to deleted frames
# False, returns original audio, probably out of sync.
# FadeMS: (default 1.0 millisec). Linear Audio Fade duration at splices when TrimAudio==true, 0 = dont fade (might result in audio 'clicks/cracks').
c
TrimAudio=Default(TrimAudio,True) # default true trims audio, false returns original audio (audiodubbed, as Framesel returns no audio)
FadeMS=Float(Default(FadeMS,1.0)) # 1 millisecond linear fadeout/fadein at splices
PruneCmd = (TrimAudio) ? "~Prune_"+RT_LocalTimeString+".txt" : ""
(!TrimAudio)
\ ? FrameSel(scmd=SCmd,cmd=Cmd,reject=true)
\ : FrameSel_CmdReWrite(PruneCmd,scmd=SCmd,cmd=Cmd,reject=true,Prune=True,range=true)
(TrimAudio) ? Prune(Cmd=PruneCmd,FadeIn=True,FadeSplice=True,FadeOut=True,Fade=FadeMS) : NOP
# If TrimAudio==true then delete Prune temp file, Else restore original Audio to the now audio-less clip
(TrimAudio)
\ ? RT_FileDelete(PruneCmd)
\ : (c.HasAudio) ? AudioDub(c) : NOP
Return Last
}

[GillesH]

Thanks for these informations. I will continue testing from your information, and in particular Duplicity2 ...

I use GetDups in a script and, sometimes, there are duplicate frames at the end of the clip, after treatment.
It is necessary then, to delete these images by a TRIM.
The use of the MT is only to save time because the script can include other filters.

[StainlessS]

I dont use GetDups, but if dupes at end of clip, I assume that it runs real time without frames file, and as clip length CANNOT change in avisynth, produces dupes at the end, for you to chop off.
I personally have never use MT at all.

You can process multiple sources at same time using two instances of Avisynth, or even 4. [EDIT: or even more]

FFMS_MakeMultiPartScripts.avsi
https://forum.doom9.org/showthread.php?p=1873783#post1873783

# FFMS_MakeMultiPartScripts.avsi

Function FFMS_MakeMultiPartScript(String FnList,String "OutPath",Int "Parts") {
/*
FFMS_MakeMultiPartScripts v0.04 ::: Requires RT_Stats, Avs+ OR GScript.

FFMS_MakeMultiPartScript(String FnList,String "OutPath"="",Int "Parts"=4)
FnList, No default. Multi-line string of avs filenames(1 or more).
Source scripts to split must use FFMS2 source filter.
Source scripts clip length must be at least (20 * Parts) frames else error abort.
This function sets a Global FFMS_MakeMultiPartScript_Glb variable, can be detected in source
avs scripts so as to determine if called by this function (for whatever reason).
OutPath, Default "", Same Directory as source file directory. Path (must exist) where output scripts written.
Where OutPath = default "", and multiple input files with different directories given in FnList, OutPath will use input file directory
for each input file (even when different directories, will not be different paths where fileSelector client script used).
Use "." for current directory.
Parts, Default 4. Number of parts that source script is split into.
*/

For process same script in multiple parts, WITHOUT MT.
Above function creates multiple scripts (default 4) that import original script and chop in pieces at I frames into eg 4 parts.
You use separate invocation of avisynth on each result script, if output to encode, then use IsStitchAble (or whatever it is called) so result clips can be spliced together.

[GillesH]

Yes, still a great idea to test this multi instances. Thank you.

[StainlessS]

Below StringRepeater() script function does not use RT_Stats, but this a good as any place to put it, perhaps of use.


Function StringRepeater(String BaseS, String RepS, int n) { # https://forum.doom9.org/showthread.php?p=1878193#post1878193
# Add n instances of repeat string RepS, to base string BaseS.
return (n>=1) ? StringRepeater(BaseS+RepS,RepS,n-1):BaseS
}


Demo

# DEMO USE # For AVS+, GScript stuff commented out
# GScript("""
SSS=""
for(i=0,8) {
S=StringRepeater("MyClip",".Blur(1.58,1.58)",i)
SSS=RT_String("%s%d] %s\\n",SSS,i,S)
}

Colorbars(width=1024,Height=240).Subtitle(SSS,lsp=0)
# """) # End Of GScript

https://i.postimg.cc/cJs1XvNT/String-Repeater.jpg (https://postimages.org/)

EDIT: For above demo, could then do eg Eval(S) to to use one of the result strings.

eg

MyClip=Colorbars
Eval(StringRepeater("MyClip",".Blur(1.58,1.58)",10)) # for 10 iterations of blur(1.58,1.58), where 1.58 is MAX possible blur.


EDIT: Or Sharpen 10 times at max

MyClip=Colorbars
Return Eval(StringRepeater("MyClip",".Sharpen(1.0,1.0)",10)) # for 10 instances of Sharpen at max possible.

Sharpen @ Max x 10.
https://i.postimg.cc/XY7Bvj2Z/Sharpen10.jpg (https://postimages.org/)

EDIT: Sharpen @ Max x 50.
https://i.postimg.cc/2yVXmRPg/Sharpen50.jpg (https://postimages.org/)
EDIT: Sharpen @ Max x 250.
https://i.postimg.cc/vH9LFZ45/Sharpen250.jpg (https://postimages.org/)

EDIT: Have not tried to max it out to find where stack overflow occurs, but for some other function many years ago a recursive
function using GScript maxed out at about 450 recurses, and error abort (probably using avs v2.58).

[StainlessS]

RT_Stats has function RT_FunctionExist() to detect builtin functions, dll functions, and script functions.

Here a routine which seems to work ok from v2.58+ without any external dll functions.

FuncNameExists

Function FuncNameExists(String Fn){Try{Eval(Fn+"()")B=True}catch(e){Assert(e.FindStr("syntax")==0,"FuncNameExists: Error in Function Name '"+Fn+"'")B=(e.FindStr("no function named")==0)}Return B}


test client

Function FuncNameExists(String Fn){Try{Eval(Fn+"()")B=True}catch(e){Assert(e.FindStr("syntax")==0,"FuncNameExists: Error in Function Name '"+Fn+"'")B=(e.FindStr("no function named")==0)}Return B}

### BELOW TESTS FOR Function FuncNameExists()

Function Sfn() { return 0 }
Function Sfn2(int i) { return 0 }
Function Sfn3(int "i") { return 0 }

Function BN(bool B) { Return (B) ? "Succeeds" : "Fails" }

#FuncNameExists("123") # Will fail Assert, "Error in Function Name'123'"

S1=FuncNameExists("SFN").BN # Succeeds (exists)
S2=FuncNameExists("SFN2").BN # Fails due to missing compulsory arg, converted to Exists.
S3=FuncNameExists("SFN3").BN # Succeeds (exists)
S4=FuncNameExists("BlankClip").BN # Succeeds (exists)
S5=FuncNameExists("FuncNameExists").BN # Succeeds (self exists)
S6=FuncNameExists("zzz").BN # Fails, does not exist due to "there is no function named..."
EOL="\n"

return BlankClip.Subtitle(S1+EOL+S2+EOL+S3+EOL+S4+EOL+S5+EOL+S6,lsp=0)

Minor EDIT

Not tested to destruction, but seems OK.

I guess here as good a place to post as any.

EDIT: If you discover problem function names. then post such here, thanx.

[real.finder]

Hello StainlessS

I was thinking since there are RT_PluginParam, is it possible to add RT_AddFunctions that load some shadow Functions from some file into avs and when some Function used by user then RT_AddFunctions load the actual dll for that Function

and that the Functions file made by either RT_AddFunctions with some parameter or another Function

the point of this is to not load many plugins since avs+ don't unload the plugins that in autoload https://github.com/pinterf/AviSynthPlus/issues/11#issuecomment-463595150 so I am plan for move some of them in sub folder

and also since I made my plugins folder work with even 2.5 in case I need it, it's will also be useful when I use 2.6 since I did already put the 2.6/avs+ plugins in sub folder and make them over load the one in the main folder when avs 2.6/avs+ is used and with this in avs 2.6 it will be used some of the 50 plugins limit already! so the idea of RT_AddFunctions will help a lot in any case

[StainlessS]

RF,
I've been struggling with the problem of how to answer your question.
RT_PluginParam is irrelevant and does not work properley anyway. RT_PluginParam can only return a single parameter list,
here eg four alternative parameter lists.

AssumeFPS(clip clip, float fps [, bool sync_audio])
AssumeFPS(clip clip, int numerator [, int denominator, bool sync_audio])
AssumeFPS(clip clip1, clip clip2 [, bool sync_audio])
AssumeFPS(clip clip1, string preset [, bool sync_audio])

RT_PluginParam can only return one of the above lists, although in a format like this [EDIT: the 3rd in above list, presume this is not the order added in source code]

AssumeFPS "cc[sync_audio]b"

There is no mechanism that I am aware of to extract multiple parameter lists, probably only the first one added by the avisynth.dll or an external plugin dll AddFunction() call.

AddFunction(), can only be used in the dll (avisynth or external) that contains the executable code, indeed it has to pass a pointer to the code that creates a filter/function.

The only functions that can possibly work are the LoadPlugin/LoadCPlugin style functions, these are part of avisynth.dll.
IScriptEnvironment2 has a LoadPlugin entry, so that would work but the problem is how to magically know when plugin needs loaded, I have no idea how to do that,
and would not really like the idea of simply scanning the current script for user supplied list of function names, both with and without dll filename extended name, eg
FredFunc1() and Fred_FredFunc1(). Additionally would not have any idea how to cope with script in avsi files.
Overall I dont like the idea nor do I think I could do it properley, I just dont know how.

AvsInit.avsi, has a bunch of functions for loading a single script or dll file, or for loading a complete directory of avs/avsi/dll files.

Here some funcs for loading complete directory of files.

Directory of files load/Import functions. Return total number of scripts imported,
or dll's loaded,
AvsInit_DImport(Dir) Import[GImport() if GScript Present] All avs/avsi scripts in folder Dir, sending debug
stuff (success/fail) to DebugView window.
AvsInit_DLoad(Dir) Load all CPP/C plugin dll's in folder Dir, sending debug stuff (success/fail) to
DebugView window.

Both DImport and DLoad, funcs may be of use in user functions to satisfy some script
dependency, where a complex script requires multiple dll's or scripts loaded.

The Import style functions use GImport() when GScript is available, otherwise Import(). Any scripts imported should have any
GScript style wrappers removed, eg

>>>> CODE >>>>
GScript(""" # <<<<<<<<<<========= REMOVE THIS LINE
...
Some gscript stuff, if/else/for/next while etc
...
""") # <<<<<<<<<<========= REMOVE THIS LINE
<<<< END CODE <<<<

With GScript wrappers removed and imported via this library, then should be able to use same scripts functions in any version of avs
standard(with Gscript plugin) or avs+, whether it be avs v2.58, avs v2.60/2.61 Standard, avs+ x86 or x64.

[real.finder]

but the problem is how to magically know when plugin needs loaded, I have no idea how to do that,
and would not really like the idea of simply scanning the current script for user supplied list of function names, both with and without dll filename extended name, eg
FredFunc1() and Fred_FredFunc1(). Additionally would not have any idea how to cope with script in avsi files.

I already have something like this

function bassAudioSource(String "Source", bool "normalize") {
LoadPlugin(AviSynthPluginsDir + "bass/BassAudio.dll")
bassAudioSource(Source, normalize)
}

with this it's only use the script wrapper in 1st time since the LoadPlugin is over load the script one

I think something similar can be made with c++ since there are some plugins that can't be simulated by script parameters

[StainlessS]

I think something similar can be made with c++ since there are some plugins that can't be simulated by script parameters

What plugins ?

EDIT: And I forgot to mention that the single parameter list that can be extracted, can only be done when the plugin has already been loaded.

[StainlessS]

I was a bit curious to see what happens when LoadPlugin called, and the dll is already loaded.

C++ source,

#define BUG // Allow DPRINTF() style stuff, comment out removes DPRINTF()'s from binary. (requires DebugView)

#include "Compiler.h"
#include <windows.h>
#include <stdio.h> // OutputDebugString(), Needs DebugView utility, http://technet.microsoft.com/en-gb/sysinternals/bb545027
#include "Avisynth.h"

#if defined(BUG)
// Use as in:- dprintf("Hello %s",__DATE__); // Produces eg "Hello Oct 23 2017".
int __cdecl dprintf(char* fmt, ...) {
char printString[2048]="TestPlug: ";
char *p=printString;
for(;*p++;);
--p; // @ null term
va_list argp;
va_start(argp, fmt);
vsprintf(p, fmt, argp);
va_end(argp);
for(;*p++;);
--p; // @ null term
if(printString == p || p[-1] != '\n') {
p[0]='\n'; // append n/l if not there already
p[1]='\0';
}
OutputDebugString(printString);
return int(p-printString); // strlen printString
}
#endif

AVSValue __cdecl TestPlug(AVSValue args, void* user_data, IScriptEnvironment* env) {
DPRINTF("Returning 42\n")
return 42;
}

const AVS_Linkage *AVS_linkage = 0;

extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit3(IScriptEnvironment* env, const AVS_Linkage* const vectors) {
AVS_linkage = vectors;
DPRINTF("AvisynthPluginInit3 Calling env->AddFunction\n")
env->AddFunction("TestPlug","",TestPlug,0);
DPRINTF("AvisynthPluginInit3 returning to Avisynth\n")
return 0;
}


Plug.avs

Function Testing(String s) {
RT_DebugF("Testing: '%s'",s)
LoadPlugin(".\TestPlug_x86.dll")
Return TestPlug()
}


BlankClip

Testing("Call 1")
Testing("Call 2")
Testing("Call 3")

return last


Result in DebugView

00000015 1.12639999 RT_DebugF: Testing: 'Call 1'
00000016 1.12908685 TestPlug: AvisynthPluginInit3 Calling env->AddFunction
00000017 1.12921154 TestPlug: AvisynthPluginInit3 returning to Avisynth
00000018 1.12928331 TestPlug: Returning 42
00000019 1.12940717 RT_DebugF: Testing: 'Call 2'
00000020 1.12948728 TestPlug: Returning 42
00000021 1.12958181 RT_DebugF: Testing: 'Call 3'
00000022 1.12965786 TestPlug: Returning 42


So LoadPlugin seems not to load the dll more than once (or at least does not call AvisynthPluginInit3 for the dll more than once).

Tested only for Avs+ v3.5.2

[real.finder]

What plugins ?

don't remember them now, and I ended up only use it with functions that has string input, but not DirectShowSource since it can has multi input!
https://i.postimg.cc/LXHpyfbP/Untitled.png (https://postimages.org/)

same for any plugin that has something like this (like LGhost)


EDIT: And I forgot to mention that the single parameter list that can be extracted, can only be done when the plugin has already been loaded.

that not a problem since the idea is scan the plugins one time and save the output in some avsplugins.ext that can also be modified later in any text editor

[real.finder]

I was a bit curious to see what happens when LoadPlugin called, and the dll is already loaded.

C++ source,

#define BUG // Allow DPRINTF() style stuff, comment out removes DPRINTF()'s from binary. (requires DebugView)

#include "Compiler.h"
#include <windows.h>
#include <stdio.h> // OutputDebugString(), Needs DebugView utility, http://technet.microsoft.com/en-gb/sysinternals/bb545027
#include "Avisynth.h"

#if defined(BUG)
// Use as in:- dprintf("Hello %s",__DATE__); // Produces eg "Hello Oct 23 2017".
int __cdecl dprintf(char* fmt, ...) {
char printString[2048]="TestPlug: ";
char *p=printString;
for(;*p++;);
--p; // @ null term
va_list argp;
va_start(argp, fmt);
vsprintf(p, fmt, argp);
va_end(argp);
for(;*p++;);
--p; // @ null term
if(printString == p || p[-1] != '\n') {
p[0]='\n'; // append n/l if not there already
p[1]='\0';
}
OutputDebugString(printString);
return int(p-printString); // strlen printString
}
#endif

AVSValue __cdecl TestPlug(AVSValue args, void* user_data, IScriptEnvironment* env) {
DPRINTF("Returning 42\n")
return 42;
}

const AVS_Linkage *AVS_linkage = 0;

extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit3(IScriptEnvironment* env, const AVS_Linkage* const vectors) {
AVS_linkage = vectors;
DPRINTF("AvisynthPluginInit3 Calling env->AddFunction\n")
env->AddFunction("TestPlug","",TestPlug,0);
DPRINTF("AvisynthPluginInit3 returning to Avisynth\n")
return 0;
}


Plug.avs

Function Testing(String s) {
RT_DebugF("Testing: '%s'",s)
LoadPlugin(".\TestPlug_x86.dll")
Return TestPlug()
}


BlankClip

Testing("Call 1")
Testing("Call 2")
Testing("Call 3")

return last


Result in DebugView

00000015 1.12639999 RT_DebugF: Testing: 'Call 1'
00000016 1.12908685 TestPlug: AvisynthPluginInit3 Calling env->AddFunction
00000017 1.12921154 TestPlug: AvisynthPluginInit3 returning to Avisynth
00000018 1.12928331 TestPlug: Returning 42
00000019 1.12940717 RT_DebugF: Testing: 'Call 2'
00000020 1.12948728 TestPlug: Returning 42
00000021 1.12958181 RT_DebugF: Testing: 'Call 3'
00000022 1.12965786 TestPlug: Returning 42


So LoadPlugin seems not to load the dll more than once (or at least does not call AvisynthPluginInit3 for the dll more than once).

Tested only for Avs+ v3.5.0

that because they are not same Function name

edit: seems I get it wrong! so it only load the plugin one time even if user load it many time?

[StainlessS]

Well as I said, only one can be extracted anyway, I have asked if could be expanded, or if some way already implemented but never got any reply.
Also, I would not fancy trying to re-implement avisynth parameter parsing, and having to deal with Explicit/Implicit Last, and trying against
all possible prototypes (if we could find them all).

EDIT:

edit: seems I get it wrong! so it only load the plugin one time even if user load it many time?
Yes. [EDIT: Tested only for Avs+ v3.5.2]

[real.finder]

Well as I said, only one can be extracted anyway, I have asked if could be expanded, or if some way already implemented but never got any reply.
Also, I would not fancy trying to re-implement avisynth parameter parsing, and having to deal with Explicit/Implicit Last, and trying against
all possible prototypes (if we could find them all).


well at least there should some way that work with the all plugins in autoload folder same as avspmod did, in the end that list with all the Functions will help, then the only thing left is the handwork in that avsplugins.ext and set load the plugin manually from subfolder (after move the plugins to it) for the Functions, then the new RT_AddFunctions load that avsplugins.ext (by put it in some .avsi in autoload folder) and did the rest when user call some Function

[StainlessS]

end that list with all the Functions will help

AvsPMod list looks to be hand edited, and some of the prototypes are not the single prototype that is available, and so looks to be hand added,
and no muliple prototypes are in that list (that I noticed).

I just do not see this idea of yours being practical, and sorry but I am not about to get involved in it.

[real.finder]

AvsPMod list looks to be hand edited, and some of the prototypes are not the single prototype that is available, and so looks to be hand added,
and no muliple prototypes are in that list (that I noticed).

some are hand edited, some are not, there are many new things in plugins just added when I update the plugins


I just do not see this idea of yours being practical, and sorry but I am not about to get involved in it.

ok, thank you anyway

[gispos]

AvsPMod list looks to be hand edited, and some of the prototypes are not the single prototype that is available, and so looks to be hand added,
and no muliple prototypes are in that list (that I noticed).

I just do not see this idea of yours being practical, and sorry but I am not about to get involved in it.

some are hand edited, some are not, there are many new things in plugins just added when I update the plugins



ok, thank you anyway
I didn't look closely at the code, but had scanned it a while ago. if I remember correctly::

The functions and associated parameters are normally read from the core filters, loaded plugins and user functions.
However, there is the database in which parameter values for some functions are predefined.
So if this function name is found in the database, or if a self-defined change is found in the options, this is then displayed as the calltip.

Is also a functions parameters changed or added and this functions name is in the database, the new parameters are not read.
Theoretically: if you delete the database and the options all new parameters should be found. However, there are no predefined values.

I hope that I write clearly.

Edit: Or someone has to maintain the database:)

[StainlessS]

Gispos, am I right in thinking that you only use a single parameter list version of any filter/function,
eg here are 4 alternative parameter lists for assumeFPS (as posted a little bit earlier)

AssumeFPS(clip clip, float fps [, bool sync_audio])
AssumeFPS(clip clip, int numerator [, int denominator, bool sync_audio])
AssumeFPS(clip clip1, clip clip2 [, bool sync_audio])
AssumeFPS(clip clip1, string preset [, bool sync_audio])


EDIT: They actually look like this when extracted:-

AssumeFPS "cc[sync_audio]b"


Also, have you ever managed to extract more than a single parameter list for any function (either from avisynth.dll or any plugin dll) ?

Here is code I use to extract parameter list using the 'secret' variable name parts "$Plugin!" and "!Param$".


AVSValue __cdecl RT_PluginParam(AVSValue args, void* user_data, IScriptEnvironment* env) {
char * myName="RT_PluginParam: ";
const char *s=args[0].AsString();
int len = int(strlen(s));
char *bf = new char[len + 16];
if(bf==NULL) env->ThrowError("%sCannot allocate memory",myName);
strcpy(bf,"$Plugin!");
strcat(bf,s);
strcat(bf,"!Param$");
char *ps;
try {
AVSValue var = env->GetVar(bf);
if(!var.IsString()) {
delete [] bf;
env->ThrowError("%s Var '$Plugin!%s!Param$' is not a string",myName,s);
}
delete [] bf;
ps = (char*)var.AsString();
} catch (IScriptEnvironment::NotFound) {
ps="";
}
return env->SaveString(ps);
}

EDIT: Crap, if try fails, then we dont delete [] bf as we should, ie bug.

[gispos]

Gispos, am I right in thinking that you only use a single parameter list version of any filter/function,
eg here are 4 alternative parameter lists for assumeFPS (as posted a little bit earlier)

AssumeFPS(clip clip, float fps [, bool sync_audio])
AssumeFPS(clip clip, int numerator [, int denominator, bool sync_audio])
AssumeFPS(clip clip1, clip clip2 [, bool sync_audio])
AssumeFPS(clip clip1, string preset [, bool sync_audio])


EDIT: They actually look like this when extracted:-

AssumeFPS "cc[sync_audio]b"


Also, have you ever managed to extract more than a single parameter list for any function (either from avisynth.dll or any plugin dll ?

Here is code I use to extract parameter list using the 'secret' variable name parts "$Plugin!" and "!Param$".


AVSValue __cdecl RT_PluginParam(AVSValue args, void* user_data, IScriptEnvironment* env) {
char * myName="RT_PluginParam: ";
const char *s=args[0].AsString();
int len = int(strlen(s));
char *bf = new char[len + 16];
if(bf==NULL) env->ThrowError("%sCannot allocate memory",myName);
strcpy(bf,"$Plugin!");
strcat(bf,s);
strcat(bf,"!Param$");
char *ps;
try {
AVSValue var = env->GetVar(bf);
if(!var.IsString()) {
delete [] bf;
env->ThrowError("%s Var '$Plugin!%s!Param$' is not a string",myName,s);
}
delete [] bf;
ps = (char*)var.AsString();
} catch (IScriptEnvironment::NotFound) {
ps="";
}
return env->SaveString(ps);
}

I think that this is difficult or even impossible to present several variants of a function as a call tip. I should have a look at that. If I feel like it.

[StainlessS]

I just meant have you ever extracted parameter variants at all, whether for call tip or other use dont matter.
It must be possible, I guess I have to dissect the avs source again.
Probably just searching for "$Plugin!" and "!Param$" would find out where the relevant code is hiding.

I see this in your filterdb.dat,

AssumeFPS(
clip,
string fps= ("ntsc_film"/ "ntsc_video"/ "ntsc_double"/ "ntsc_quad"/ "ntsc_round_film"/ "ntsc_round_video"/ "ntsc_round_double"/ "ntsc_round_quad"/ "film"/ "pal_film"/ "pal_video"/ "pal_double"/ "pal_quad"),
[int "denominator"],
bool "sync_audio"=false
)


Anyway, if I find out how to extract alternative lists, I'll try to remember to let you know bout it.

[gispos]

Just tested:
two functions with the same name and different parameters. The last function always appears as a call tip.
The entry that has the same function name is always overwritten. Whether I can change that...,
the function in the script must also be found, read and then the parameters for this function must be found in the list. It probably won't be that easy.

Edit: I've thought enough for today. bye

[StainlessS]

By the way gis, I noticed a bug in prev posted C source, forgot to delete [] bf when try fails.

EDIT:

AVSValue __cdecl RT_PluginParam(AVSValue args, void* user_data, IScriptEnvironment* env) {
char * myName="RT_PluginParam: ";
const char *s=args[0].AsString();
int len = int(strlen(s));
char *bf = new char[len + 16];
if(bf==NULL) env->ThrowError("%sCannot allocate memory",myName);
strcpy(bf,"$Plugin!");
strcat(bf,s);
strcat(bf,"!Param$");
char *ps;
try {
AVSValue var = env->GetVar(bf);
if(!var.IsString()) {
delete [] bf;
env->ThrowError("%s Var '$Plugin!%s!Param$' is not a string",myName,s);
}
delete [] bf; // Remove
ps = (char*)var.AsString();
} catch (IScriptEnvironment::NotFound) {
// delete Missing here
ps="";
}
delete [] bf; // Add
return env->SaveString(ps);
}

Maybe move the red delete to where the blue delete is.

[gispos]

I had to look. You make me crazy.:)
self.installed_avsi_filternames = set ()
In delphi, the same name can only exist once in a set, and in Python too (I think). Then a lot of code would have to be changed. I don't know if I can do it.

But now is really enough for today.:)

[real.finder]

maybe this will help https://github.com/jeeb/avisynth/blob/00889df587c36fad1c6ba2c3df6baedf20e61ec6/src/core/avisynth.cpp#L1160

[StainlessS]

Yes thanks RF, I have seen that lot before, still not sure if its possible to extract multiple parameter type variations.
I've spent some time looking at it again, but my brain aint working at all, (constantly feeling exhausted, even when I've just got out of bed),
I will though try again after a good dose of beer and a good nights sleep.

[real.finder]

I already have something like this

function bassAudioSource(String "Source", bool "normalize") {
LoadPlugin(AviSynthPluginsDir + "bass/BassAudio.dll")
bassAudioSource(Source, normalize)
}

with this it's only use the script wrapper in 1st time since the LoadPlugin is over load the script one

I think something similar can be made with c++ since there are some plugins that can't be simulated by script parameters


What plugins ?

don't remember them now, and I ended up only use it with functions that has string input, but not DirectShowSource since it can has multi input!
https://i.postimg.cc/LXHpyfbP/Untitled.png (https://postimages.org/)

same for any plugin that has something like this (like LGhost)

ok, I redid some test, seems it work even with clip input

function AddGrainC(clip, float "var", float "uvar", float "hcorr", float "vcorr", int "seed", bool "constant", bool "sse2") {
LoadPlugin(AviSynthPlugins64Dir + "25/AddGrainC.dll")
clip.AddGrainC (var, uvar, hcorr, vcorr, seed, constant, sse2)
}

function AddGrain (clip, float "var", float "hcorr", float "vcorr", float "uvar", int "seed", bool "constant", bool "sse2") {
LoadPlugin(AviSynthPlugins64Dir + "25/AddGrainC.dll")
clip.AddGrain (var, uvar, hcorr, vcorr, seed, constant, sse2)
}

edit: AddGrain (clip clip,
also work

[StainlessS]

Yes, I thought the awkward one you were having trouble with was the multiple filenames thing, DirectShowSource(string filename, [...], bool "seek", ... )
Provided no array arg (ie multiple of same type), and all compulsory args provided, then can just pass on the args to the dll function,
also you do not have to do anything for optional non provided defaults, ie dont use eg Default(Something,true), just pass on Something and let dll plugin handle it.

[real.finder]

can RT_WriteFileList write in RAM (To avoid any temporary file) or there are another functions can do that?

I did steal some code from you and make insert_types (base on VS Function with same name, there are some friend need it for some After Effects things)

Function insert_types(clip clp, string "dir", string "container", string "sourcecall") {

Assert(Defined(dir), "insert_types: you must set the dir")

WildCard=Default(container,"*.AVI")
sourcecall=Default(sourcecall,"ffms2")

clp
TMPFile="~AVIVid.TMP"
scstr= """
eS=RightStr(dir,1)
Path=(eS!="\" && eS!="/") ? dir+"\" : dir # Ensure End Slash
TestPath=LeftStr(Path,Strlen(Path)-1) # Test PATH Exist without End Slash, else Error
Assert(Exist(TestPath),"insert_types: Path Does not exist ("+Path+")")
SearchString = Path + WildCard
Got = RT_WriteFileList(SearchString,TMPFile)
while(Got > 0) {
SS=RT_ReadTxtFromFile(TMPFile) # Read multiline list of filenames
got=Got-1
S=RT_TxtGetLine(SS,Got)
fi=FindStr(s,"_")
fr=s.MidStr(fi+1).Value.Int
last.insertsign(eval(sourcecall+string("(s)")),fr)
}
RT_FileDelete(TMPFile)
"""
IsAvsPlus ? eval(scstr) : GSCript(scstr)
return last
}

and it's also in my mod of insertsign.avsi

it work if the dir has files like

Sign_2333.avi
Sign_3343.avi
Sign_79343.avi

and it automatically set start frame base on the number in the end of the file name

[StainlessS]

Sorry, no RT_WriteFileList equivalent write to RAM, users are just not that trustworthy [and scripts can crash]. [Although you can read file result into a multi-line string]
You were not really using the function correctly, was doing way too much work.

Have taken liberty of doing a bit of a mod, should work just fine
[I started mod reading complete file into text string, but then realized that your script was not working well and that is why you asked about RAM write, so just fixed what you had already done].


Function insert_types(clip clp, string dir, string "container", string "sourcecall") {
Assert(dir!="", "insert_types: you must set the dir") # We made Dir Non Optional
WildCard=Default(container,"*.AVI")
sourcecall=Default(sourcecall,"ffms2")
clp
eS=RightStr(dir,1)
Path=(eS!="\" && eS!="/") ? dir+"\" : dir # Ensure End Slash
TestPath=LeftStr(Path,Strlen(Path)-1) # Test PATH Exist without End Slash, else Error
Assert(Exist(TestPath),"insert_types: Path Does not exist ("+Path+")")
SearchString = Path + WildCard
TMPFile=RT_GetSystemEnv("TEMP") + "\~AVIVid_" + RT_LocalTimeString(file=True) + ".TMP" # Unique name in User TEMP directory
LineCount = RT_WriteFileList(SearchString,TMPFile)
scstr= """
for(Line = 0, LineCount - 1) { # Line = zero relative line number
s=TMPFile.RT_ReadTxtFromFile(Lines=1,Start=Line).RT_TxtGetLine(Line=0) # Read single Line from file, convert to single line string [not multi-line ie strip NewLine Chr(10)]
fi=FindStr(s,"_")
fr=s.MidStr(fi+1).Value.Int
last.insertsign(eval(sourcecall+string("(s)")),fr)
}
RT_FileDelete(TMPFile) # Dump file when done with it
"""
IsAvsPlus ? eval(scstr) : GSCript(scstr) # External dll overrides builtin plugs, perhaps user intended GScript to be used if is in Plugins dir.
return last
}


Not tested AT ALL, give it a test, I'm sure that you will will have no major problems with it.

EDIT: I've done it in ascending order, if you need descending then use for(Line=LineCount-1,0,-1)

[real.finder]

Sorry, no RT_WriteFileList equivalent write to RAM, users are just not that trustworthy [and scripts can crash]. [Although you can read file result into a multi-line string]
You were not really using the function correctly, was doing way too much work.

Have taken liberty of doing a bit of a mod, should work just fine
[I started mod reading complete file into text string, but then realized that your script was not working well and that is why you asked about RAM write, so just fixed what you had already done].


Function insert_types(clip clp, string dir, string "container", string "sourcecall") {
Assert(dir!="", "insert_types: you must set the dir") # We made Dir Non Optional
WildCard=Default(container,"*.AVI")
sourcecall=Default(sourcecall,"ffms2")
clp
eS=RightStr(dir,1)
Path=(eS!="\" && eS!="/") ? dir+"\" : dir # Ensure End Slash
TestPath=LeftStr(Path,Strlen(Path)-1) # Test PATH Exist without End Slash, else Error
Assert(Exist(TestPath),"insert_types: Path Does not exist ("+Path+")")
SearchString = Path + WildCard
TMPFile=RT_GetSystemEnv("TEMP") + "\~AVIVid_" + RT_LocalTimeString(file=True) + ".TMP" # Unique name in User TEMP directory
LineCount = RT_WriteFileList(SearchString,TMPFile)
scstr= """
for(Line = 0, LineCount - 1) { # Line = zero relative line number
s=TMPFile.RT_ReadTxtFromFile(Lines=1,Start=Line).RT_TxtGetLine(Line=0) # Read single Line from file, convert to single line string [not multi-line ie strip NewLine Chr(10)]
fi=FindStr(s,"_")
fr=s.MidStr(fi+1).Value.Int
last.insertsign(eval(sourcecall+string("(s)")),fr)
}
RT_FileDelete(TMPFile) # Dump file when done with it
"""
IsAvsPlus ? eval(scstr) : GSCript(scstr) # External dll overrides builtin plugs, perhaps user intended GScript to be used if is in Plugins dir.
return last
}


Not tested AT ALL, give it a test, I'm sure that you will will have no major problems with it.

EDIT: I've done it in ascending order, if you need descending then use for(Line=LineCount-1,0,-1)

it's was work but I think your edit make it better

it was put the temp file near the encoding script, your edit seems put it the windows temp folder which is better, but still the .py seems don't make any temporary file

def insert_types(out, dir):
for root, dirs, files in walk(dir):
for f in files:
match = search(r"(\d*)\.avi$", f)
if match:
startframe = match.group(1)
out = fvf.InsertSign(out, join(root, f), int(startframe), matrix="709")
return out

[StainlessS]

Yeh the VS Python thingy looks like it has some kind of operating system handle for walking the directory, so does not create a text file,
proper programming language can do things like that whereas we have lesser control, especially for cleaning up open handles on crash and stuff like that.

it's was work but I think your edit make it better
yeh but, for every single entry it read the whole file list [EDIT: Into a multi-line string] and selected the one it wanted from the complete list read, every time.
The fix just reads a single line filename from the file list when required.

[real.finder]

Yes, I thought the awkward one you were having trouble with was the multiple filenames thing, DirectShowSource(string filename, [...], bool "seek", ... )
Provided no array arg (ie multiple of same type), and all compulsory args provided, then can just pass on the args to the dll function,
also you do not have to do anything for optional non provided defaults, ie dont use eg Default(Something,true), just pass on Something and let dll plugin handle it.

I find that Average also has [, ...]

Average(var [, ...])

I don't think this can be done in the script method
https://github.com/pinterf/Average/blob/cdf01b60de071d5e4e1dd1ab71f62248d1ea4759/Average/average.cpp#L544

[StainlessS]

Nope cant be done unless arrays implemented in script.
[No idea if arrays version of 3.6 would work, I have been sticking to 3.52 noarray version].
Assuming that arrays as args to script functions do work, then try to just pass the array to the dll. [see what happens]

EDIT: Actually I'm using 3.5 r3106.

[poisondeathray]

Can you load a random image sequence (not sequentially numbered; something like dog.jpg, cat.jpg, house.jpg...) , and have them subtitled (text overlay) with the filename (dog, cat, house) but without the "jpg" extension) ?

ImageSequence (CoronaSequence) can load wildcards "*.jpg", and has the textmode option that does this, but includes the extension


Is it possible with RT_Stats RT_Subtitle or another method ?

[StainlessS]

I dont know ImageSequence (CoronaSequence).
But, there is this,

RT_WriteFileList(String,String,bool "append"=false)
Non-clip function.
This function, takes as the first string, a filename template with wildcards, this template can be eg
"Fred.jpg" or "Fr?d.jp*g". where the '?' character matches any single character and the '*' wildcard matches zero or more characters.
v1.03, Can take multiple pipe ('|') separated Wilcards in the extension ONLY, as in "D:\Pics\*.BMP|JP*G|PNG"
The second string is the output filename which will be created and contain the files matching the input file template.
The function will return the number of filenames written into the output file. Directories are ignored altogether
and not written to the output file. A return of -1 indicates a problem opening/writing the output file.
v1.06, Append, if true appends to existing file. NOTE, the return value does NOT include any already existing
filenames if append is true (use RT_FileQueryLines to inquire total number of filenames when append = true).


There is a scipt in RT_Stats AVS\IMAGE\ directory which could be used, or hacked, ImageSplicer.avs,
I will not post it here as it is of fair size, and you will have it yourself already if you have RT_.

This is an earlier version of similar type, bit shorter, maybe easier to hack.

PicVidShow.avs

# EDIT To Suit # https://forum.doom9.org/showthread.php?p=1604043#post1604043

VID = True # True == Video : False == Pics, see IN_FILES directories/wildcards

IN_FILES = (VID) ? "D:\VID\*.AVI|MP*G" : "D:\Pics\*.BMP|JP*G|PNG" # Added '|' Pipe separator for multiple Wildcard in Extensions ONLY.
AUTOCROP = True # Crop off black borders.
SHOWBORDER = True # True Show replaced border in Red, else Black. (Test AUTOCROP)
AUTOLEVEL_STRENGTH = 0.666 # AutoLevel Strength 0.0 -> 1.0
SHOWAUTOLEVEL = True # Right Hand Side Only AutoLeveled
#
FAR = 4.0/3.0 # Required Output Frame Aspect Ratio
SzLimit = 1024
DEBUG = False # Debug
# ---------------------
BORDER = (SHOWBORDER) ? $FF0000: (VID) ? $101010 : $000000 # Color for replaced borders.
SUBS = True # Subtitles on clips/images
DEBUG_QBC = True # Debug QueryBorderCrop
DEBUG_QLMM = True # Debug QueryLumaMinMax
FPS = (VID) ? 30000.0/1001.0 : 1.0 # Whatever.
OUT_FILE = "FILE.List" # Dont need to change really, just a name.
#--------------------------------------------------------------------------------
GScript("""
picclip = 0 # dummy for now
result=RT_WriteFileList(IN_FILES,OUT_FILE) # Create a listing file of Pic files
Assert((result!=0), "No files found")
FILELIST=RT_ReadTxtFromFile(OUT_FILE) # Get list of files
FILES=RT_TxtQueryLines(FILELIST) # Query Number of lines in String ie number of files.
if(DEBUG){RT_Debug("Found files = " + String(FILES))}
# Prescan to ascertain max wid/hit of all files, also Autocrop & Auto Levels
MAXH=0 MAXW=0 # init max wid and hit
CROPCOORDS=""
AUTOLEVELS=""
AUTOLEVEL=(AUTOLEVEL_STRENGTH>0.0) # Adjust using Levels.
For(i=0,FILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of avi file i
if(DEBUG){RT_Debug(String(i)+") Getting File dimensions",FN)}
if(VID) {
k=DirectshowSource(FN)
k=k.ConvertToYUY2().Trim(0,0)
} else {
k=ImageReader(FN,end=0).ConvertToRGB32() # With Bmap, some come up as RGB24 bit some RGB32.
}
OrgH=k.height OrgW=k.width
if(DEBUG){RT_Debug ("Orig Width =",String(orgW),"Orig Height =",String(OrgH))}
if (AUTOCROP) {
QBC = k.QueryBorderCrop(samples=24,laced=VID,debug=(DEBUG&&DEBUG_QLMM))
QBC = RT_TxtGetLine(QBC,2) # CropMore
CROPCOORDS=RT_TxtAddStr(CROPCOORDS,QBC) # Avisynth Bugfix, same as CROPCOORDS = CROPCOORDS + QBC + Chr(10)
Eval(QBC)
k=k.Crop(QBCropXM,QBCropYM,QBCropWM,QBCropHM)
}
if (AUTOLEVEL) {
QLMM = k.QueryLumaMinMax(debug=(DEBUG&&DEBUG_QLMM))
Eval(QLMM)
AUTOLEVELS=RT_TxtAddStr(AUTOLEVELS,QLMM) # Avisynth Bugfix, same as AUTOLEVELS = AUTOLEVELS + QLMM + Chr(10)
}
H=k.height W=k.width
If(MAXH<H) {MAXH=H}
If(MAXW<W) {MAXW=W}
if(DEBUG){RT_Debug("MAX Width So Far =",String(MAXW),"MAX Height So Far =",String(MAXH))}
if(DEBUG){RT_Debug(" ----------------------------- ")} # line separator
}
maxFAR = MAXW / MAXH

if(FAR >= maxFAR) {
canvasheight = (MAXH > SzLimit) ? SzLimit : MAXH
canvasheight = ((canvasheight + 1) / 2) * 2
canvaswidth = Int((canvasheight * FAR + 2) / 4) * 4
} else {
canvaswidth = (MAXW > SzLimit) ? SzLimit : MAXW
canvaswidth = ((canvaswidth + 2) / 4) * 4
canvasheight = Int((canvaswidth * FAR + 1) / 2) * 2
}

if(DEBUG){RT_Debug("SETTING CanvasWidth =",String(canvaswidth)," CanvasWeight =",String(canvasheight))}
if(DEBUG){RT_Debug(" ----------------------------- ")} # line separator
For(i=0,FILES-1) {
FN=RT_TxtGetLine(FILELIST,i) # Filename of avi file i
if(DEBUG){RT_Debug(string(i)+ ")","Processing File",FN)}
if(VID) {
k=DirectshowSource(FN).ConvertToYUY2().Trim(0,0)
} else {
k=ImageReader(FN,end=0).ConvertToRGB32() # With Bmap, RGB24 problems with addborders
}
OrgH=k.height OrgW=k.width
if (AUTOLEVEL) {
# We do AUTOLEVELS before crop, because of SHOWAUTOLEVEL
QAT = RT_TxtGetLine(AUTOLEVELS,i)
Eval(QAT)
if(k.IsYUV()) {
CSMin = 16
CSMax = 235
} else {
CSMin = 0
CSMax = 255
}
TMP_K=k
ALMin = Int(CSMin - ((CSMin - QLMMMin) * AUTOLEVEL_STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * AUTOLEVEL_STRENGTH)) # Round down
k = k.Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DONT USE Coring
if(SHOWAUTOLEVEL) {
tmpw=int((k.width/4)*2)
Left = TMP_K.Crop(0,0,tmpw,-0)
Right= k.Crop(tmpw,0,-0,-0)
k=StackHorizontal(Left,Right)
}
}

if (AUTOCROP) {
QBC = RT_TxtGetLine(CROPCOORDS,i)
Eval(QBC)
k=k.Crop(QBCropXM,QBCropYM,QBCropWM,QBCropHM)
}

iWidth=k.Width() iHeight=k.Height()

cFAR = float(canvaswidth) / canvasheight
iFar = float(iWidth) / iHeight
Left=0 Top=0 Right=0 Bot=0

if(iFAR >= cFAR) {
oWid = canvaswidth
oHit = (oWid*iHeight/iWidth/4)*4
Top = (canvasheight-oHit) / 2
Bot = canvasheight - oHit - Top
} else {
oHit = canvasheight
oWid = (oHit*iWidth/iHeight/4)*4
Left = (canvaswidth-oWid) / 2
Right = canvaswidth - oWid - Left
}

RT_Debug("Resize","oWid="+String(oWid),"oHit="+String(oHit),"Left="+String(Left),"Top="+String(Top),"Rgt="+String(Right),"Bot="+String(Bot))
k = k.Spline36Resize(oWid,oHit).addborders(Left,Top,Right,Bot,BORDER).assumefps(fps)

if(SUBS) {
k=k.subtitle(String(i) + "/"+String(FILES-1) +" " + FN)
k=k.subtitle("Orig width=" + string(orgW) +" " + "Orig height=" + string(orgH) +" FAR="+String(Float(orgW) /orgH, "%.2f"),20,20)
if (AUTOCROP) {k=k.subtitle("Crop width=" + string(iWidth) +" " + "Crop height=" + string(iHeight) +" FAR="+String(Float(iWidth) /iHeight, "%.2f"),20,40)
} else {k=k.subtitle("NO AUTOCROP",20,40)}
k=k.subtitle("New width=" + string(oWid) +" " + "New height=" + string(oHit)+" FAR="+String(Float(oWid)/oHit,"%.2f"),20,60)
k=k.subtitle("Out width=" + string(k.Width) +" " + "Out height=" + string(k.Height)+" FAR="+String(Float(k.Width)/k.Height,"%.2f"),20,80)
if(AUTOLEVEL) {
k=k.subtitle("Auto Levels(" + String(ALMin) + ",1.0," + String(ALMAx) + "," + String(CSMin) + "," + String(CSMax) + ")",20,100)
k=k.subtitle("QLMMMin = " + String(QLMMMin) + " QLMMMax = " +String(QLMMMAx),20,120)
k=k.subtitle("AUTOLEVEL @ " + String(AUTOLEVEL_STRENGTH*100.0,"%.0f") + "%",20,140)
} else {k=k.subtitle("NO AUTOLEVEL",20,100)}
}
picclip = (IsClip(picclip)) ? picclip ++ k : k # Append to clip so far
if(!VID){k=k.ConvertToRGB24()}
if(DEBUG){RT_Debug(" ")} # line separator
if(DEBUG){RT_Debug(" ----------------------------- ")} # line separator
}
""")
PicClip


EDIT: With the filename, you can use below for just the filename node without path or extension.


RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the
parts joined together.
'Get' (default 15, 1 -> 15), If set, Bit 0=DRIVE, 1=Dir, 2=Name, 4=Extension.
Add 1 for Drive (bit 0), add 2 for Dir (bit 1), add 4 for Name (bit 2), add 8 for Extension (bit 3).
Some combinations do not make sense, eg Drive + Extension (1+8=9). Below sensible options.
1 = Drive (includes trailing ':')
2 = Dir (includes trailing '\')
3 = Drive + Dir
4 = Name
6 = Dir + Name
7 = Drive + Dir + Name
8 = Extension (includes leading '.')
12 = Name + Extension
14 = Dir + Name + Extension
15 = Drive + Dir + Name + Extension
Assuming a current working directory of eg "D:\avs\avi\", 'filename'="test.avi" and 'get'=15, returns "D:\avs\avi\test.avi",
so given a relative filename and default 'get'=15, is equivalent to RT_GetFullPathName(filename).


EDIT: If you have difficulty, post again tomorrow. [I've gorra get some sleep now].

select images from list in random order, might need plugin for that.

[poisondeathray]

Thanks, I can load with ImageSplicer.avs , but not sure how to hack the subs=true part of the script to show what I'm guessing is RT_FilenameSplit get=4 (for name only) for the current filename of image into the subtitle overlay

[StainlessS]

I'll come back again after I hack additional functionality into the Shuffle plugin:- https://forum.doom9.org/showthread.php?t=176193

Separate filter [Maybe ShuffleT(), ie Shuffle Temporal], which shuffles frames based on some secret key/seed. should be OK in HBD I think.

Clip encryption can of course also use this to great effect either before or after spatial shuffle.

[StainlessS]

if(SUBS) {
k=k.subtitle(String(i) + "/"+String(FILES-1) +" " + FN)


Maybe something like




if(SUBS) {
FN_NAME_ONLY = RT_FilenameSplit(FN,get=4) # Name node only, no path, no extension
k=k.subtitle(String(i) + "/"+String(FILES-1) +" " + FN_NAME_ONLY)


Nearly done with randomize clip frames order thing.

[poisondeathray]

Thanks it works. The only thing needed in this case was the filename, so just FN_NAME ONLY was sufficient - and I commented out the other parts for the subs/text like frame number , aspect ratio, etc...


if(SUBS) {
FN_NAME_ONLY = RT_FilenameSplit(FN,get=4) # Name node only, no path, no extension
k=k.subtitle(FN_NAME_ONLY)
}


I knew it could be done with RT_Stats, I just had a "feeling" it would be covered . There so much stuff in RT_Stats that I haven't even scratched the surface

Thanks again

[StainlessS]

About to post in Shuffle thread, New ShuffleT(clip c, Int "Seed"=0 , Bool "Inverse"=False ) # Shuffle Temporal, ie shuffle frame order. All valid colorspaces.

[real.finder]

hi StainlessS

I am trying to make function that convert avs trim to trimIn for tfm and still apply the normal trim in the same time, so I come with this

function intrim(clip i, string "trimline", string "output", string "type") {
Assert(defined(trimline), "'intrim' trimline is needed")
i
lastframe = framecount() - 1
TrimFile=defined(output) ? output : "TrimFile.TXT"
intrimloop(trimline, TrimFile, type, lastframe, true)
eval(trimline)
}

function intrimloop(clip i, string "trimline", string "output", string "type", int "lastframe", bool "firstcall") {
firstcall = default(firstcall, false)
ofindtrim = FindStr(trimline, "trim")
sishtrim = ofindtrim != 0
trimstart = FindStr(trimline, "(")
findtrimstart = trimstart + 1
trimend = FindStr(trimline, ")")
findtrimend = trimend - 1
taketrim = MidStr(trimline, findtrimstart, findtrimend)
trims = FindStr(taketrim, ",")
strim = MidStr(taketrim,1,trims-1).Value.Int
etrim = MidStr(taketrim,trims+1).Value.Int
etrim = etrim == 0 ? lastframe : etrim
ntrimr = rightStr(trimline,StrLen(trimline)-trimend)
ofindntrim = FindStr(ntrimr, "trim")
sishntrim = ofindtrim != 0
ntrimstart = FindStr(ntrimr, "(")
findntrimstart = ntrimstart + 1
ntrims = FindStr(ntrimr, ",")
sntrim = MidStr(ntrimr,1,ntrims-1).Value.Int
istrim = strim !=0 && firstcall ? 0 : etrim + 1
ietrim = firstcall && istrim == 0 ? strim-1 : sishntrim ? sntrim - 1 : lastframe
writeline = !sishntrim ? etrim < lastframe : strim !=0 && sishtrim
writeline ? RT_WriteFile(output, \
"%d,%d\n", istrim, ietrim) : nop()
sishntrim ? intrimloop(i, firstcall && istrim == 0 ? trimline : rightStr(trimline,StrLen(trimline)-trimend), output, type, lastframe) : i
}

but with

ColorBars()
intrim("Trim(2, 198)++Trim(243, 2410)++Trim(20768, 39916)++Trim(42635, 43513)")

it only did
0,1


it should add more lines or did I missed something?

[StainlessS]

RF, can you post exactly what you expect out of it (I aint ever used that intrim with tfm and am falling asleep as I write this).

EDIT: Like this ? [ranges on separate lines]

0,1
199,242
2411,20767
39917,42634
43514,107891

Last frame number gotten from Framecount of Colorbars

[real.finder]

RF, can you post exactly what you expect out of it (I aint ever used that intrim with tfm and am falling asleep as I write this).
for ColorBars()
intrim("Trim(2, 198)++Trim(243, 2410)++Trim(20768, 39916)++Trim(42635, 43513)")
0,1
199,242
2411,20767
39917,42634
43514,107891


EDIT: Like this ?

0,1
199,242
2411,20767
39917,42634
43514,107891

Last frame number gotten from Framecount of Colorbars

yes

[StainlessS]

Would something like this be preferable ? [ FrameSel:- https://forum.doom9.org/showthread.php?t=167971 ]


ColorBars().ShowFrameNumber

ofn=".\Outfile.txt"

SCMD="""
2,198
243,2410
20768,39916
42635,43513
"""

FrameSel_CmdReWrite(Last,ofn,scmd=SCmd,Reject=True,Range=True,Space=False,Prune=False)

Last.FrameSel(scmd=SCMD)


Results in Outfile.txt

0,1
199,242
2411,20767
39917,42634
43514,107891


EDIT: Framesel does not deal with audio, however can use Prune() via SelectRanges() and RejectRanges():- https://forum.doom9.org/showthread.php?p=1683559#post1683559

EDIT: Or like this using Prune

ColorBars().ShowFrameNumber

outfn = ".\Outfile.txt"
Prunefn= ".\Prunefile.txt"

SCMD="""
2,198
243,2410
20768,39916
42635,43513
"""

FrameSel_CmdReWrite(Last,outfn, scmd=SCmd,Reject=True,Range=True,Space=False,Prune=False)
FrameSel_CmdReWrite(Last,Prunefn,scmd=SCmd,Reject=false,Range=True,Space=False,Prune=true) # Fixed in Blue

Last.Prune(cmd=Prunefn)


outfile.txt

0,1
199,242
2411,20767
39917,42634
43514,107891


Prunefile.txt EDIT: Oops was wrong as for fix BLUE above

0,2,198
0,243,2410
0,20768,39916
0,42635,43513

[real.finder]

Would something like this be preferable ? [ FrameSel:- https://forum.doom9.org/showthread.php?t=167971 ]


ColorBars().ShowFrameNumber

ofn=".\Outfile.txt"

SCMD="""
2,198
243,2410
20768,39916
42635,43513
"""

FrameSel_CmdReWrite(Last,ofn,scmd=SCmd,Reject=True,Range=True,Space=False,Prune=False)

Last.FrameSel(scmd=SCMD)


Results in Outfile.txt

0,1
199,242
2411,20767
39917,42634
43514,107891


I will try it then

[StainlessS]

The SelectRanges() version [but does pretty much same as the Prune version in prev post script]

ColorBars().ShowFrameNumber

outfn = ".\Outfile.txt"

SCMD="""
2,198
243,2410
20768,39916
42635,43513
"""

FrameSel_CmdReWrite(Last,outfn, scmd=SCmd,Reject=True,Range=True,Space=False,Prune=False) # Trim excluded

SelectRanges(scmd=SCMD) # Trim Included (SelectRanges Script in FrameSel AVS dir (also requires Prune if audio))


EDIT: Also NOTE, in your original script, RT_WriteFile would require "Append=true" to work properley [when bugs fixed].

EDIT: Maybe best like this

ColorBars().ShowFrameNumber

outfn = ".\Outfile.txt"

PruneTMP=RT_GetSystemEnv("TEMP") + "\~PruneTMP_" + RT_LocalTimeString(file=True) + ".TMP" # Unique name in User TEMP directory

SCMD="""
2,198
243,2410
20768,39916
42635,43513
"""

FrameSel_CmdReWrite(Last,outfn, scmd=SCmd,Reject=True,Range=True,Space=False,Prune=False) # Trim Excluded ranges
FrameSel_CmdReWrite(Last,PruneTMP,scmd=SCmd,Reject=False,Range=True,Space=False,Prune=true) # Trim Included ranges in Prune Format # Fixed in Blue

Prune(cmd=PruneTMP) # Trim

RT_FileDelete(PruneTMP) # Del temp prune command file

Return Last

and again result outfile of trim excluded ranges

0,1
199,242
2411,20767
39917,42634
43514,107891

[real.finder]

EDIT: Also NOTE, in your original script, RT_WriteFile would require "Append=true" to work properley [when bugs fixed].


thanks, it's work now :)


function intrim(clip i, string "trimline", string "output", string "type") {
Assert(defined(trimline), "'intrim' trimline is needed")
i
lastframe = framecount() - 1
TrimFile=defined(output) ? output : "TrimFile.TXT"
intrimloop(LCase(trimline), TrimFile, type, lastframe, true)
eval(trimline)
}

function intrimloop(clip i, string "trimline", string "output", string "type", int "lastframe", bool "firstcall") {
firstcall = default(firstcall, false)
ofindtrim = FindStr(trimline, "trim")
sishtrim = ofindtrim != 0
trimstart = FindStr(trimline, "(")
findtrimstart = trimstart + 1
trimend = FindStr(trimline, ")")
findtrimend = trimend - 1
taketrim = MidStr(trimline, findtrimstart, findtrimend)
trims = FindStr(taketrim, ",")
strim = MidStr(taketrim,1,trims-1).Value.Int
etrim = MidStr(taketrim,trims+1).Value.Int
etrim = etrim == 0 ? lastframe : etrim
ntrimr = rightStr(trimline,StrLen(trimline)-trimend)
ofindntrim = FindStr(ntrimr, "trim")
sishntrim = ofindntrim != 0
ntrimstart = sishntrim ? FindStr(ntrimr, "(") : nop()
findntrimstart = !sishntrim ? nop() : ntrimstart + 1
ntrims = !sishntrim ? nop() : FindStr(ntrimr, ",")
sntrim = !sishntrim ? nop() : MidStr(ntrimr,findntrimstart,ntrims-1).Value.Int
istrim = strim !=0 && firstcall ? 0 : etrim + 1
ietrim = firstcall && istrim == 0 ? strim-1 : sishntrim ? sntrim - 1 : lastframe
writeline = sishtrim ? sishntrim ? strim != 0 : etrim < lastframe : false
writeline ? RT_WriteFile(output, \
"%d,%d\n", istrim, ietrim, append=true) : nop()
sishntrim ? intrimloop(i, firstcall && istrim == 0 ? trimline : rightStr(trimline,StrLen(trimline)-trimend), output, type, lastframe) : i
}


now I need to make it better, and made another function to abridgement trims line like this trim(2, 198)++trim(199, 242)++trim(243, 2410)++Trim(2411, 20767)++Trim(20768, 39916)++Trim(39917, 42634)++Trim(42635, 43513) to trim(2, 43513)

[StainlessS]

now I need to make it better, and made another function to abridgement

Abridgement.avs

ColorBars().ShowFrameNumber

SCMD="""
2,198
199,242
243, 2410
2411, 20767
20768, 39916
39917, 42634
42635, 43513
"""

outFn = ".\Abridgement.txt"

FrameSel_CmdReWrite(Last,Outfn, scmd=SCmd, Reject=False,Ordered=True,Range=True,Space=False,Prune=False)

Return Last


Abridgement.txt

2,43513


EDIT: Or this (Same output result)

ColorBars().ShowFrameNumber

INCMD="""
2,198
243, 2410
20768, 39916
42635, 43513
"""

OUTCMD="""
199,242
2411, 20767
39917, 42634
"""

SCMD = INCMD + OUTCMD

outFn = ".\Abridgement.txt"

FrameSel_CmdReWrite(Last,Outfn, scmd=SCmd, Reject=False,Ordered=True,Range=True,Space=False,Prune=False)

Return Last

Above, Orders and coalesces all the ranges, outputting only distinct ranges, in above case only a single output range.

EDIT: What is the end result of all of this, give complete description of ALL required [and take your time :) ].
EDIT: Also, what about audio ?

[real.finder]

Abridgement.avs

ColorBars().ShowFrameNumber

SCMD="""
2,198
199,242
243, 2410
2411, 20767
20768, 39916
39917, 42634
42635, 43513
"""

outFn = ".\Abridgement.txt"

FrameSel_CmdReWrite(Last,Outfn, scmd=SCmd, Reject=False,Ordered=True,Range=True,Space=False,Prune=False)

Return Last


Abridgement.txt

2,43513


EDIT: Or this (Same output result)

ColorBars().ShowFrameNumber

INCMD="""
2,198
243, 2410
20768, 39916
42635, 43513
"""

OUTCMD="""
199,242
2411, 20767
39917, 42634
"""

SCMD = INCMD + OUTCMD

outFn = ".\Abridgement.txt"

FrameSel_CmdReWrite(Last,Outfn, scmd=SCmd, Reject=False,Ordered=True,Range=True,Space=False,Prune=False)

Return Last

Above, Orders and coalesces all the ranges, outputting only distinct ranges, in above case only a single output range.

EDIT: What is the end result of all of this, give complete description of ALL required [and take your time :) ].

thanks, I was working on this too

function abtrim(string "trimline") {
Assert(defined(trimline), "'intrim' trimline is needed")
abtrimloop(LCase(trimline))
}

function abtrimloop(string "trimline", int "remainder") {
remainder = default(remainder, -1)
otrimline = trimline
trimline = remainder > 0 ? RightStr(trimline, remainder) : trimline
ofindtrim = FindStr(trimline, "trim")
sishtrim = ofindtrim != 0
trimstart = FindStr(trimline, "(")
findtrimstart = trimstart + 1
trimend = FindStr(trimline, ")")
findtrimend = trimend - 1
taketrim = MidStr(trimline, findtrimstart, findtrimend)
trims = FindStr(taketrim, ",")
strim = !sishtrim ? nop() : MidStr(taketrim,1,trims-1).Value.Int
etrim = !sishtrim ? nop() : MidStr(taketrim,trims+1).Value.Int
ntrimr = RightStr(trimline,StrLen(trimline)-trimend)
ofindntrim = FindStr(ntrimr, "trim")
sishntrim = ofindntrim != 0
ntrimstart = sishntrim ? FindStr(ntrimr, "(") : nop()
findntrimstart = !sishntrim ? nop() : ntrimstart + 1
ntrims = !sishntrim ? nop() : FindStr(ntrimr, ",")
sntrim = !sishntrim ? nop() : MidStr(ntrimr,findntrimstart,ntrims-1).Value.Int
ntrimline=LeftStr(otrimline,trims+findtrimstart+(remainder > 0 ? remainder : 0))+(((etrim+1)==sntrim) ? MidStr(ntrimr,ntrims+1) : MidStr(trimline,trims+findtrimstart+1))
pointer = !sishtrim ? 0 : remainder != 0 ? remainder == -1 ? ntrimline != otrimline ? FindStr(ntrimline, "+") != 0 ? trims+findtrimstart-1 : 0 : FindStr(ntrimline, "+") : remainder + FindStr(RightStr(ntrimline, StrLen(ntrimline) - remainder), "+") : nop()
pointer != 0 ? abtrimloop(ntrimline, pointer) : ntrimline
}

it work with abtrim("trim(2, 198)++trim(199, 242)++trim(243, 2410)++Trim(2411, 20767)++Trim(20768, 39916)++Trim(39917, 42634)++Trim(42635, 43513)")

but still not for this (need fix, still working on it) abtrim("trim(2, 198)++trim(243, 2410)++Trim(2411, 20767)++Trim(20768, 39916)++Trim(39917, 42634)++Trim(42635, 43513)")

and it need nothing but avs

[StainlessS]

I'll look again at all of this tomorrow.
Would be a lot easier (I think) if had frames list source (String or file), and processed that, instead of screwing around trying to parse
list with "Trim(xxx, yyy) ++ ..." etc, much easier to create trim command from frames string/file than
to parse trims command and extract frames list from that.

Also, making it all work in Avs script alone [and properley] is not gonna be easy, nor small.
EDIT: Although with recent AVS+ arrays at least can implement frame range sorting and coalescing with less difficulty.
(I'm not using array supporting versions of Avs+, it dont work with XP MeGUI, so I'm pretty much sticking with avs+ 3.5, r3106)

EDIT:
and it need nothing but avs
Is RT_ usage OK ? [you use it for RT_writeFile() ].
For your abridgement thing, you need to parse and verify correctness of your frame ranges, eg end frame same or after start frame,
for coalescing, you also need sorting all ranges beforehand. If you dont do it properley, it will bite your ass the first time
somebody screws up the trims/ranges list.
Without proper parsing/sorting/checking, will run risk of eg Infinite Recursion, stack exhaustion, where script/avisynth will just disappear
without any error message or indication of what went wrong, not a good way to crash, or, it might just badly mangle your result clip.
The things you are attempting are quite tricky even in C, in basic script is gonna be less easy.

EDIT: Also, before you use RT_WriteFile with Append=true, is best to use RT_FileDelete on the target file [in the intrim() func].
Otherwise on multiple use eg

0,1
199,242
2411,20767
39917,42634
43514,107891
0,1
199,242
2411,20767
39917,42634
43514,107891
...
etc

[real.finder]

Is RT_ usage OK ? [you use it for RT_writeFile() ].
yes, in fact abtrim is for helping intrim

EDIT: Also, before you use RT_WriteFile with Append=true, is best to use RT_FileDelete on the target file [in the intrim() func].

yes, I note this, will use it when I done with abtrim

edit: I think this did it!

function abtrim(string "trimline") {
Assert(defined(trimline), "'intrim' trimline is needed")
abtrimloop(LCase(trimline))
}

function abtrimloop(string "trimline", int "remainder") {
remainder = default(remainder, -1)
otrimline = trimline
sisending = !(remainder < StrLen(trimline))
trimline = remainder > 0 ? !sisending ? RightStr(trimline, StrLen(trimline) - remainder) : trimline : trimline
ofindtrim = sisending ? nop() : FindStr(trimline, "trim")
sishtrim = sisending ? false : ofindtrim != 0
trimstart = !sishtrim ? nop() : ofindtrim + FindStr(MidStr(trimline, ofindtrim + 1), "(")
findtrimstart = !sishtrim ? nop() : trimstart + 1
trimend = !sishtrim ? nop() : ofindtrim + FindStr(MidStr(trimline, ofindtrim + 1), ")")
findtrimend = !sishtrim ? nop() : trimend - 1
taketrim = !sishtrim ? nop() : MidStr(trimline, findtrimstart, findtrimend - findtrimstart + 1)
trims = !sishtrim ? nop() : FindStr(taketrim, ",")
strim = !sishtrim ? nop() : MidStr(taketrim,1,trims-1).Value.Int
etrim = !sishtrim ? nop() : MidStr(taketrim,trims+1).Value.Int
ntrimr = !sishtrim ? nop() : RightStr(trimline,StrLen(trimline)-trimend)
ofindntrim = !sishtrim ? nop() : FindStr(ntrimr, "trim")
sishntrim = !sishtrim ? nop() : ofindntrim != 0
ntrimstart = !sishtrim ? nop() : sishntrim ? FindStr(ntrimr, "(") : nop()
findntrimstart = !sishtrim ? nop() : !sishntrim ? nop() : ntrimstart + 1
ntrims = !sishtrim ? nop() : !sishntrim ? nop() : FindStr(ntrimr, ",")
sntrim = !sishtrim ? nop() : !sishntrim ? nop() : MidStr(ntrimr,findntrimstart,ntrims-1).Value.Int
domerge = !sishtrim ? nop() : sishntrim ? ((etrim+1)==sntrim) : false
ntrimline=!sishtrim ? nop() : !domerge ? otrimline : LeftStr(otrimline,(remainder > 0 ? remainder : 0) + findtrimstart + trims - 1) + MidStr(ntrimr,ntrims+1)
remainder = !sishtrim ? nop() : domerge ? remainder : max(0,remainder) + trimend
!sishtrim ? otrimline : sishntrim ? abtrimloop(ntrimline, remainder) : ntrimline
}

edit: even better for both

function abtrim(string "trimline") {
Assert(defined(trimline), "'intrim' trimline is needed")
function abtrimloop(string "trimline", int "remainder") {
remainder = default(remainder, -1)
otrimline = trimline
sisending = !(remainder < StrLen(trimline))
trimline = remainder > 0 ? !sisending ? RightStr(trimline, StrLen(trimline) - remainder) : trimline : trimline
ofindtrim = sisending ? nop() : FindStr(trimline, "trim")
sishtrim = sisending ? false : ofindtrim != 0
trimstart = !sishtrim ? nop() : ofindtrim + FindStr(MidStr(trimline, ofindtrim + 1), "(")
findtrimstart = !sishtrim ? nop() : trimstart + 1
trimend = !sishtrim ? nop() : ofindtrim + FindStr(MidStr(trimline, ofindtrim + 1), ")")
findtrimend = !sishtrim ? nop() : trimend - 1
taketrim = !sishtrim ? nop() : MidStr(trimline, findtrimstart, findtrimend - findtrimstart + 1)
trims = !sishtrim ? nop() : FindStr(taketrim, ",")
strim = !sishtrim ? nop() : MidStr(taketrim,1,trims-1).Value.Int
etrim = !sishtrim ? nop() : MidStr(taketrim,trims+1).Value.Int
ntrimr = !sishtrim ? nop() : RightStr(trimline,StrLen(trimline)-trimend)
ofindntrim = !sishtrim ? nop() : FindStr(ntrimr, "trim")
sishntrim = !sishtrim ? nop() : ofindntrim != 0
ntrimstart = !sishtrim ? nop() : sishntrim ? FindStr(ntrimr, "(") : nop()
findntrimstart = !sishtrim ? nop() : !sishntrim ? nop() : ntrimstart + 1
ntrims = !sishtrim ? nop() : !sishntrim ? nop() : FindStr(ntrimr, ",")
sntrim = !sishtrim ? nop() : !sishntrim ? nop() : MidStr(ntrimr,findntrimstart,ntrims-1).Value.Int
domerge = !sishtrim ? nop() : sishntrim ? ((etrim+1)==sntrim) : false
ntrimline=!sishtrim ? nop() : !domerge ? otrimline : LeftStr(otrimline,(remainder > 0 ? remainder : 0) + findtrimstart + trims - 1) + MidStr(ntrimr,ntrims+1)
remainder = !sishtrim ? nop() : domerge ? remainder : max(0,remainder) + trimend
!sishtrim ? otrimline : sishntrim ? abtrimloop(ntrimline, remainder) : ntrimline
}
abtrimloop(LCase(trimline))
}

function intrim(clip i, string "trimline", string "output") {
Assert(defined(trimline), "'intrim' trimline is needed")
i
lastframe = framecount() - 1
TrimFile=defined(output) ? output : "TrimFile.TXT"
Exist(TrimFile) ? RT_FileDelete(TrimFile) : nop()
function intrimloop(clip i, string "trimline", string "output", int "lastframe", bool "firstcall") {
firstcall = default(firstcall, false)
ofindtrim = FindStr(trimline, "trim")
sishtrim = ofindtrim != 0
trimstart = !sishtrim ? nop() : FindStr(trimline, "(")
findtrimstart = !sishtrim ? nop() : trimstart + 1
trimend = !sishtrim ? nop() : FindStr(trimline, ")")
findtrimend = !sishtrim ? nop() : trimend - 1
taketrim = !sishtrim ? nop() : MidStr(trimline, findtrimstart, findtrimend-findtrimstart+1)
trims = !sishtrim ? nop() : FindStr(taketrim, ",")
strim = !sishtrim ? nop() : MidStr(taketrim,1,trims-1).Value.Int
etrim = !sishtrim ? nop() : MidStr(taketrim,trims+1).Value.Int
etrim = !sishtrim ? nop() : etrim == 0 ? lastframe : etrim
ntrimr = !sishtrim ? nop() : rightStr(trimline,StrLen(trimline)-trimend)
ofindntrim = !sishtrim ? nop() : FindStr(ntrimr, "trim")
sishntrim = !sishtrim ? nop() : ofindntrim != 0
ntrimstart = !sishtrim ? nop() : sishntrim ? FindStr(ntrimr, "(") : nop()
findntrimstart = !sishtrim ? nop() : !sishntrim ? nop() : ntrimstart + 1
ntrims = !sishtrim ? nop() : !sishntrim ? nop() : FindStr(ntrimr, ",")
sntrim = !sishtrim ? nop() : !sishntrim ? nop() : MidStr(ntrimr,findntrimstart,ntrims-1).Value.Int
istrim = !sishtrim ? nop() : !sishtrim ? nop() : strim !=0 && firstcall ? 0 : etrim + 1
ietrim = !sishtrim ? nop() : !sishtrim ? nop() : firstcall && istrim == 0 ? strim-1 : sishntrim ? sntrim - 1 : lastframe
writeline = sishtrim ? sishntrim ? strim != 0 : etrim < lastframe : false
writeline ? RT_WriteFile(output, \
"%d,%d\n", istrim, ietrim, append=true) : nop()
sishtrim ? intrimloop(i, firstcall && istrim == 0 ? trimline : rightStr(trimline,StrLen(trimline)-trimend), output, lastframe) : i
}
intrimloop(abtrim(trimline), TrimFile, lastframe, true)
eval(trimline)
}

I think it's ok, unless there are hidden bug, for now only RT_Stats is needed for intrim

edit: I added them in https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/AnimeIVTC.avsi

if there any update in future it will be there

[real.finder]

Nope cant be done unless arrays implemented in script.
[No idea if arrays version of 3.6 would work, I have been sticking to 3.52 noarray version].
Assuming that arrays as args to script functions do work, then try to just pass the array to the dll. [see what happens]

EDIT: Actually I'm using 3.5 r3106.

avs+ 3.6 and array work, I did it in UniversalPluginsFolders recently (in plugins64+ folder)

[StainlessS]

I really should not be spending any time on D9, too busy.
But, when I do get time, I'll dissect UniversalPluginsFolders and steal all of the good bits.

[StainlessS]

RT_Stats 2.0 Beta 13, Update:- https://www.mediafire.com/file/xa3t1wx234gyzfq/RT_Stats_25%252626_x86_x64_dll_v2.00Beta13_20201229.zip/file
Also on SendSpace below this post.

[real.finder]

lets say that there are comments (with #) like this:-

#~ Trim(0, 9006)++Trim(9007, 23602)++Trim(23603, 40776)++Trim(40777, 43472)++Trim(43473, 44402)
Trim(44403, 48808)++Trim(48809, 64002)++Trim(64003, 85162)++Trim(85163, 87858)++Trim(87859, 88788)
#~ ++Trim(88789, 94064)++Trim(94065, 108480)++Trim(108481, 129550)++Trim(129551, 132246)++Trim(132247, 133356)

is there a way to delete comment lines from string? so it will be as

Trim(44403, 48808)++Trim(48809, 64002)++Trim(64003, 85162)++Trim(85163, 87858)++Trim(87859, 88788)

need this since those lines will break inTrim() (https://github.com/realfinder/AVS-Stuff/blob/85aa25a36c9cb9a17b5ad941eb9be9d3b0f6d76b/avs%202.5%20and%20up/AnimeIVTC.avsi#L634)

[StainlessS]

Does this sort you out OK R.F., ?

(I've left in non required Chr testing funcs)


Function ChrIsNul(String S) {return RT_Ord(S)== 0} # End of String
Function ChrIsTAB(String S) {return RT_Ord(S)== 9} # TAB
Function ChrIsLF(String S) {return RT_Ord(S)==10} # Line Feed (New Line)
Function ChrIsCR(String S) {return RT_Ord(S)==13} # Carriage Return
Function ChrIsSPACE(String S) {return RT_Ord(S)==32} # SPACE
Function ChrIsBang(String S) {return RT_Ord(S)==33} # !
Function ChrIsDQuote(String S) {return RT_Ord(S)==34} # "
Function ChrIsHash(String S) {return RT_Ord(S)==35} # #
Function ChrIsDollar(String S) {return RT_Ord(S)==36} # $
Function ChrIsPercent(String S) {return RT_Ord(S)==37} # %
Function ChrIsAmp(String S) {return RT_Ord(S)==38} # &
Function ChrIsSQuote(String S) {return RT_Ord(S)==39} # '
Function ChrIsOParen(String S) {return RT_Ord(S)==40} # (
Function ChrIsCParen(String S) {return RT_Ord(S)==41} # )
Function ChrIsStar(String S) {return RT_Ord(S)==42} # *
Function ChrIsPlus(String S) {return RT_Ord(S)==43} # +
Function ChrIsComma(String S) {return RT_Ord(S)==44} # ,
Function ChrIsHyphen(String S) {return RT_Ord(S)==45} # -
Function ChrIsDot(String S) {return RT_Ord(S)==46} # .
Function ChrIsSlash(String S) {return RT_Ord(S)==47} # /
Function ChrIsColon(String S) {return RT_Ord(S)==58} # :
Function ChrIsSColon(String S) {return RT_Ord(S)==59} # ;
Function ChrIsLT(String S) {return RT_Ord(S)==60} # <
Function ChrIsEqual(String S) {return RT_Ord(S)==61} # =
Function ChrIsGT(String S) {return RT_Ord(S)==62} # >
Function ChrIsQuery(String S) {return RT_Ord(S)==63} # ?
Function ChrIsAt(String S) {return RT_Ord(S)==64} # @
Function ChrIsOSquare(String S) {return RT_Ord(S)==91} # [
Function ChrIsBSlash(String S) {return RT_Ord(S)==92} # /
Function ChrIsCSquare(String S) {return RT_Ord(S)==93} # ]
Function ChrIsHat(String S) {return RT_Ord(S)==94} # ^
Function ChrIsUScore(String S) {return RT_Ord(S)==95} # _
Function ChrIsOCurly(String S) {return RT_Ord(S)==123} # {
Function ChrIsPipe(String S) {return RT_Ord(S)==124} # |
Function ChrIsCCurly(String S) {return RT_Ord(S)==125} # }
Function ChrIsTilde(String S) {return RT_Ord(S)==126} # ~
#
Function ChrIsWhite(String S) {C=RT_Ord(S) return C==32||C>=8&&C<=13} # SPC, BackSPC, TAB, LF, VT, FF, CR
Function ChrIsDigit(String S) {C=RT_Ord(S) return C>=48&&C<=57}
Function ChrIsUCase(String S) {C=RT_Ord(S) return C>=65&&C<=90}
Function ChrIsLCase(String S) {C=RT_Ord(S) return C>=97&&C<=122}
Function ChrIsAlpha(String S) {C=RT_Ord(S) return C>=65&&C<=90||C>=97&&C<=122}
Function ChrIsHex(String S) {C=RT_Ord(S) return C>=48&&C<=57||C>=65&&C<=70||C>=97&&C<=102}
# Req GScript or AVS+
Function ChrEatWhite(String S) {i=1 C=RT_Ord(S,i) While(C==32||C>=8&&C<=13) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatAlpha(String S) {i=1 C=RT_Ord(S,i) While(C>=65&&C<=90||C>=97&&C<=122) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatDigits(String S) {i=1 C=RT_Ord(S,i) While(C>=48&&C<=57) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatHex(String S) {i=1 C=RT_Ord(S,i) While(C>=48&&C<=57||C>=65&&C<=70||C>=97&&C<=102) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}

#################################### Above, we only use ChrEatWhite

SSS="""
#~ Trim(0, 9006)++Trim(9007, 23602)++Trim(23603, 40776)++Trim(40777, 43472)++Trim(43473, 44402)
Trim(44403, 48808)++Trim(48809, 64002)++Trim(64003, 85162)++Trim(85163, 87858)++Trim(87859, 88788)
#~ ++Trim(88789, 94064)++Trim(94065, 108480)++Trim(108481, 129550)++Trim(129551, 132246)++Trim(132247, 133356)
"""

NewS = ""
LineCount = SSS.RT_TxtQueryLines
for(i=0,LineCount-1) { # Will Ignore Blank lines and lines where first non-white char is hash '#'.
TmpS = SSS.RT_TxtGetLine(i).ChrEatWhite # Get line of text from multi-line string and remove leading whitespace.
if(TmpS.RT_Ord != 35 && TmpS.RT_Ord != 0) { # NOT '#' or End Of Line
NewS = (NewS=="") ? TmpS : NewS + Chr(10) + TmpS
}
}

BlankClip(Width=1024,height=640,length=1).RT_Subtitle("Stripped =\n%s",NewS)

[real.finder]

yes, thanks StainlessS

:goodpost:

[Dogway]

StainlessS is RT_QwikScan() aimed to align matching shots or only frames (ala MatchFrames). I spend a lot of time syncing videos (to match audio actually) and I grew tired of it, some shots are longer, shorter, slided, added or omitted, as such an example manually crafted script would look like this:

v=ffvideosource("E:\video01.mkv")
a=LWLibavaudioSource("E:\video01.mkv")

v2=ffvideosource("E:\video02.mkv")
a2=LWLibavaudioSource("E:\video02.mkv").ConvertAudioToFloat()

a.ConvertAudioToFloat()

audiodub(v,last)
assumefps(24000,1001)

av2=audiodub(v2,a2)

TimeStretch(rate=((24000.0/1001.0)*100.0)/25.0)

trim(746,0)
trim(0,28046)+trim(av2,28046+1,28502)+trim(28202+1,0)
trim(0,30886)+trim(av2,30886+1,30886+14*24+4)+trim(30886+1,0)
trim(0,37583)+trim(av2,37583+1,37583+13*24+3)+trim(37583+1,0)
trim(0,39334)+trim(av2,39334+1,39334+18*24-7)+trim(39334+1,0)
trim(0,43695)+trim(av2,43695+1,43695+5*24)+trim(43695+1,0)
(...)
trim(0,137057)+trim(av2,137057+1,137666-4)+trim(137057+1,0)
trim(0,137907)+trim(av2,137907+1,138531-1)+trim(137907+1,0)
trim(0,139372)+trim(av2,139372+1,139665)+trim(139372+1,0)
trim(0,141478)+trim(av2,141478+1,142295)+trim(141478+1,0)
trim(0,145026)+trim(av2,145026+1,145411+8)+trim(145026+1,0)
trim(0,149794)+trim(av2,149794+1,150601+48+19)+trim(149794+1,0)
trim(0,151913)+trim(av2,151913+1,151913+24*13+38)+trim(151913+1,0)
trim(0,155769)+trim(av2,155769+1,155769+24*32-14)+trim(155769+1,0)
trim(0,157588)+trim(av2,157588+1,158418)+trim(157683,0)
trim(0,158766)+trim(av2,158766+1,158986)+trim(158766+1,0)
trim(0,160356)+trim(av2,160356+1,160356+24*14-6)+trim(160356+1,0)
trim(0,161050)+trim(av2,161050+1,161976)+trim(161050+1,0)
trim(0,162076)+trim(av2,162076+1,162205+24+25)+trim(162076+1,0)

delayaudio(32*0.0417)
killvideo()

Is it possible to automatize this or do any of your tools facilitate the process?

[StainlessS]

RT_QwikScan, was a bit of a nightmare.
Is to find similar frames only.
It started off good, where spec was to find IDENTICAL frames only, it did this pretty much perfectly, and after creation
of DBase, did so at an incredible ~3 millions frames per second searched [or thereabouts, on an XP Q6600 2.40GHz CPU].
Sadly, it turned out frames required were not identical at all, had moving cursor [art package thingy] which made not identical,
and then it was found out that art package itself did not do a perfect UNDO operation, so frames to seek [UNDO] were different to
the frames they were supposed to UNDO to. There began my nightmare 9 months getting it to work at all, and it also dramatically
slowed search to something like 25,000 frames per second. If provided identical frames then should still be quite fast but
providing for error made it slower, maybe something like 250,000 FPS searched or thereabouts.
DBase on Q6600 took about 18 minutes to create on 200,000 frame clip [DVD], but it took about same to do single averageluma scan on same length clip.
It is also not certain that frame will indeed be matched, if too different or if LumaTol [luma tolerance] is set too low.

If clips are spatially aligned, then might help to look at SirPlant(), first posted here:- https://forum.doom9.org/showthread.php?p=1787898#post1787898
Its in the current RT_Stats 2.0Beta13 AVS directory.

EDIT: Link above is only description, not script. Is 540+ lines script. EDIT: 556 lines, 41KB.
EDIT: It uses RT_QwikScan, which I'de forgotten about.

[Dogway]

Thanks a lot, I will have a look, yes I typically align them to easily spot frame differences.

[Dogway]

Hey, I had some time to test RT_QwikScan thanks to carefully reading this post (https://forum.doom9.org/showpost.php?p=1787899&postcount=376) of yours and let me tell you, it works magnificiently. I tested on a 6000 frames chunk and it worked great, it's going to save me a lot of time.
I admit it took me a while to get used to the switches workflow but once it clicks in your brain it makes it all the way easier.

A few remarks I wanted frame accuracy to the frame, so I had to comment one assert line:
# Assert(16 <= MatchLen,RT_string("%s16 <= MatchLen(%d)",myName,MatchLen))
Another thing is I had the clips reverted, the source clip is the one to replace, the extended version so to speak, in the end the prune call will replace everything in source except for those unique in it.


I will post here my code as a reminder and help for other people:

LOG Files creation

r0=ffvideosource("Extended version.mkv")
r1=ffvideosource("Theatrical version.mkv") # both should be temporal and spatially aligned

src = r0.trim(917,7000) # Source clip
rep = r1.trim(917,7000) # Replace clip, Holds sequences that will replace those in source clip.
# Sequences in Replace clip should be in SAME order as in source clip.
# Clips must be same size, colorspace.
# When splicing sequences with Prune (later) everything must be similar, audio etc,
# also audio should both be normalized for similar voume levels, this detection stage ignores audio.
# In later script, Prune will replace with both video and audio from replace clip.
SRCFN = "Source_"
REPFN = "Replac_"
srcARR = SRCFN+".ARR" # Name of RT_QwikScan FingerPrint Array
srcNxtARR = SRCFN+".Next.ARR" # Name of RT_QwikScan Locator Next Array (Next == scan forward Array)
repARR = REPFN+".ARR" # Name of RT_QwikScan FingerPrint Array
repNxtARR = REPFN+".Next.ARR" # Name of RT_QwikScan Locator Next Array (Next == scan forward Array)

#
Th = 0.8 # Default 0.9(not close matching frames maybe about 0.8)
Th2 = 0.65 # Default 0.75(not close matching frames maybe about 0.65)
LUMATOL = 16.0 # Default 8.0(not close matching frames maybe up to about 16.0)
MatchLen = 1 # Default 60(Minimum sequence length that must match (frames). Default 60(not close matching frames maybe about 32)
SEARCHSTEP = 1 # Default MatchLen/4(not close matching frames maybe about 4) Frames to step after match failure.
STRICT = True # Default True=strict 1:1 matching of frames, False=allow slight mismatch ie missing/extra frames in replacement clip.
VERBOSITY = 5 # Debug output verbocity level, to DebugView.
INSMODE = 2 # Default 0=Replace Only, 1=Insert Extras Only, 2=Replace and Insert Extras
BLKW = 64 # default 64
BLKH = 64 # default BLKW
OLAPX = 0 # default BLKW/2
OLAPY = 0 # default BLKH/2

X = 8 # Avoid edge noise or logos. (All default 0)
Y = 8
W = -8
H = -8

### !!! MUST SET TRUE FIRST, THEN FALSE to RUN !!! ###
CREATE = True # Set True to Create QWikScan arrays for SirPlant. (View progress in DebugView)


(CREATE) ? RT_QwikScanCreate(Src,srcARR,"",srcNxtARR,debug=true,x=X,y=Y,w=W,h=H) : NOP # Only need create these once, Re-Create if change coords or clips.
(CREATE) ? RT_QwikScanCreate(rep,RepARR,"",repNxtARR,debug=true,x=X,y=Y,w=W,h=H) : NOP

### !!! USE DEBUGVIEW !!! ###
(!CREATE) ? SirPlant(src,rep,srcARR,srcNxtARR,repARR,repNxtARR,
\ Th=Th,Th2=Th2,MatchLen=MatchLen,SearchStep=SEARCHSTEP,LumaTol=LUMATOL,Strict=STRICT,InsMode=INSMODE,VerBosity=VERBOSITY,
\ Blkw=BLKW,BlkH=BLKH,OLapX=-OLAPX,OLapY=OLAPY
\ ) : NOP

return Messageclip("All Done" + ((Create)?", Set Create=False":""))



DEBUG Result

vSrc="Extended version.mkv"
vRep="Theatrical version.mkv"

Project="Default"
Cmd=Project+"_Command.txt"
SHOW = True
SRC_AUDIO = False # Use only Source audio. ONLY if STRICT and not inserting EXTRA replacement ranges, else out of sync audio likely
STACK = False # Same for STACK, only if Strict and no EXTRA replacement ranges.

A=ffvideosource(vSrc).trim(917,7000)
B=A C=A
D=ffvideosource(vRep).trim(917,7000)
E=D F=D

A = (SHOW) ? A.AddBorders(0,0,0,16,$000000) : A
B = (SHOW) ? B.AddBorders(0,0,0,16,$404040) : B
C = (SHOW) ? C.AddBorders(0,0,0,16,$808080) : C
D = (SHOW) ? D.AddBorders(0,0,0,16,$00FF00) : D
E = (SHOW) ? E.AddBorders(0,0,0,16,$FF0000) : E
F = (SHOW) ? F.AddBorders(0,0,0,16,$0000FF) : F

SrcAudio=A.KillVideo

Nicks="""
InterveningSource=0
MatchingSource=1
EndRangeSource=2
MatchingReplacement=3
ExtraReplacement=4
EndRangeReplacement=5
"""

Prune(A,B,C,D,E,F,cmd=Cmd,NickName=Nicks,Show=SHOW)

(SRC_AUDIO) ? AudioDub(SrcAudio) : NOP

return (STACK) ? StackVertical(A,Last) : Last


Result Script

A=ffvideosource(vSrc).trim(917,7000)
B=A C=A
D=ffvideosource(vRep).trim(917,7000)
E=D F=D

Nicks="""
InterveningSource=0
MatchingSource=1
EndRangeSource=2
MatchingReplacement=3
ExtraReplacement=4
EndRangeReplacement=5
"""

Prune(A,B,C,D,E,F,cmd="Default_Command.txt",NickName=Nicks)

I understand Prune will honor the audio of each clip right?

[StainlessS]

Prune splices both video and audio, so both source clips need to be normalised so as not to have jumps in volume level.

Glad you figured out how to use it, not so easy to understand, nor to write, still baffles the hell out of me :)

Maybe I should remove that Assert.

EDIT:

Project="Default"
Cmd=Project+"_Command.txt"
SHOW = True
SRC_AUDIO = False # Use only Source audio. ONLY if STRICT and not inserting EXTRA replacement ranges, else out of sync audio likely
STACK = False

As above, audio will also be replaced together with video.

If True, then will use only original audio from source [ie src audio is audiodubbed over the spliced result].
[possible audio out-of-sync if not 1:1 matches or inserting extra sequences that exist in rep but not in src].

[StainlessS]

BUMP, see above edit.

[Dogway]

I comitted a mistake and lost 3h of processing :(
In any case here are my observations.

It took 3h to process 60000 frames (didn't use Prefetch though), about 1/3 of a 2h film on a i7-4790K, at which point my RAM usage reached 32Gb and had to terminate it.
It fails on fades to dark, I had to go and manually add some frames there on result clip and it eventually matches again so no big issue, probably only it has to scan more time in preprocess stage.
I had some minor issue on a scene where shots were a bit shuffled, and had to manually fix that part, but again minor issue counting that I'm saving to do hundreds of edits.
Clips need to have same number of frames, but in this case (extended vs theatrical) they don't, so I appended a blankclip to the theatrical, not sure if this is fine.

Also I'm wondering if I can lower resolution from 1080p for the prepass stage to avoid the above RAM issue.

[StainlessS]

Th = 0.8 # Default 0.9(not close matching frames maybe about 0.8)
Th2 = 0.65 # Default 0.75(not close matching frames maybe about 0.65)
LUMATOL = 16.0 # Default 8.0(not close matching frames maybe up to about 16.0)
MatchLen = 1 # Default 60(Minimum sequence length that must match (frames). Default 60(not close matching frames maybe about 32)
SEARCHSTEP = 1 # Default MatchLen/4(not close matching frames maybe about 4) Frames to step after match failure.
STRICT = True # Default True=strict 1:1 matching of frames, False=allow slight mismatch ie missing/extra frames in replacement clip.
VERBOSITY = 5 # Debug output verbocity level, to DebugView.
INSMODE = 2 # Default 0=Replace Only, 1=Insert Extras Only, 2=Replace and Insert Extras
BLKW = 64 # default 64
BLKH = 64 # default BLKW
OLAPX = 0 # default BLKW/2
OLAPY = 0 # default BLKH/2



MatchLen = 1 # Default 60(Minimum sequence length that must match (frames). Default 60(not close matching frames maybe about 32)
# Default might be better

SEARCHSTEP = 1 # Default MatchLen/4(not close matching frames maybe about 4) Frames to step after match failure.
# It does bi-directional scan anyway, defaults may be faster

Big LumaTol would severely affect times, much much slower. [but if is very different frames, then has to be high-ish]
Fails on fade to dark
Not really surprising, esp if one fade and other dont.
I had some minor issue on a scene where shots were a bit shuffled, and had to manually fix that part,
MatchLen = 1, and SEARCHSTEP=1 probably did not help.
also, INSMOD=2, 2=Replace and Insert Extras. Only if rep clip has sequences which are not in src, otherwise could cause problems with out of order
seqs where matches fail [where Default 0=Replace Only, might be better].

Clips need to have same number of frames, but in this case (extended vs theatrical) they don't, so I appended a blankclip to the theatrical, not sure if this is fine.

NOPE, same length dont matter. [would just make it slower adding blankclip, more to fingerprint, more to search]

Also I'm wondering if I can lower resolution from 1080p for the prepass stage to avoid the above RAM issue.
Dont think would be a problem [maybe even eg 320,240 would work, MAYBE, but 640x480 or thereabout a bit safer].
The arrays are not used after scan script, so size would not be problem to Prune splicing, Prune command file already created.

think these are biggest hogs for memory [defaults might also be faster]

MatchLen = 1 # Default 60(Minimum sequence length that must match (frames). Default 60(not close matching frames maybe about 32)
SEARCHSTEP = 1 # Default MatchLen/4(not close matching frames maybe about 4) Frames to step after match failure.

[Dogway]

Thanks for the help.

[Morzzz]

After butchering the source code quite hard, I got this to compile on Linux. Seems to be running Duplicity2/DropDeadGorgeous just fine, but it'll probably set the house on fire when trying anything else.

It does contain some more portable alternatives to some of the really Win32/VSC centered code though. If anyone interested (StainlessS?) drop me a PM.

[StainlessS]

Thanks.
I'm not using Avs+ on Linux just yet, but when I do, I may bug you a bit :)

[real.finder]

After butchering the source code quite hard, I got this to compile on Linux. Seems to be running Duplicity2/DropDeadGorgeous just fine, but it'll probably set the house on fire when trying anything else.

It does contain some more portable alternatives to some of the really Win32/VSC centered code though. If anyone interested (StainlessS?) drop me a PM.

isn't better to post the changes directly?

[StainlessS]

I have no problem is Morzzz wants to post the updates.

[Morzzz]

Ok, I found out I'm in need of a far less intelligent dedup algorithm, so I have no need for this after all :-D

Anyways, here is the rudimentary code:

https://github.com/mischa85/rt_stats

[StainlessS]

Thanks.

[VoodooFX]

I want to write one line to existing txt file, but I want it to be the first line. Is it possible?

[StainlessS]

Nope sorry, nothing like that implemented.
Best I can suggest [so long as only rarely used], is to read entire file, then re-write with something like

FN=".\test.txt"
Preamble = "Something something somethng"
S=RT_ReadTxtFromFile(FN)
RT_WriteFile(FN,"%s\n%s",Preamble,S) # Will create anew, NO Append

Untested.
EDIT: Added "\n" between %s strings formatting, assumes that Preamble does not end in n/l.
EDIT: If n/l status at end of Preamble UNKNOWN, could use

FN=".\test.txt"
Preamble = "Something something somethng"
S=RT_ReadTxtFromFile(FN)
RT_WriteFile(FN,"%s",Preamble) # Will create anew, NO Append [with guaranteed n/l at end {EDIT: Add n/l if not already present}]
RT_WriteFile(FN,"%s",S,Append=true) # glue on the rest. [with guaranteed n/l at end]

[VoodooFX]

I thought that to read a whole file and then write it is the only option.
Your script works well.

[StainlessS]

Glad it works for U. Another peculiar one added in one of the more recent betas was

RT_FileDuplicate(String Filename1,String Filename2,bool "Overwrite"=False)
Makes a copy of the source Filename1, to a second destination Filename2, automatically overwriting existing Filename2 if
Overrwrite is true (default false). Fails in existing Filename2 if Overrwrite= false (default).

Cant remember why I added it.
No reply necessary.

EDIT: I think I added above for binary duplicate, ie dont have to contain text only. [original requirement probably for short-ish template clip duplicate, so multi-megabytes, probably multi-GigaBytes OK]

[StainlessS]

Test for above mentioned RT_FileDuplicate()

# Infile:- SHA-512: c15605364db905631581032b32f5984b85d2ada7fcdf75058a1e72d7685e1920b01aeff4dc5ccea49519fd54638e052906184ee1a640e9e54d25e9ef43324747
# OutFile:- SHA-512: c15605364db905631581032b32f5984b85d2ada7fcdf75058a1e72d7685e1920b01aeff4dc5ccea49519fd54638e052906184ee1a640e9e54d25e9ef43324747

BlankClip(Width=520,height=128)
FN1="F:\Infile.mp4"
FN2="E:\Outfile.mp4"
TS = RT_TimerHP()
RT_FileDuplicate(FN1,FN2,OverWrite=True)
TE = RT_TimerHP()

T = TE - TS

Subtitle(RT_string("2.50 GB (2,693,697,922 bytes)\\nFile Duplicated in %f seconds",T),lsp=0,size=32)

https://i.postimg.cc/6pSWGnqm/filesize.jpg (https://postimages.org/)
SHA-512 for both at top of code block. [time for two {in and out} physical hard drives]

EDIT: Did not time it but windows copies same file in about 2+ seconds.

EDIT: And there is also this one in latest beta.

RT_FileRename(String OldFilename,String NewFileName)
Renames a file from OldFilename to NewFilename.
This function renames the file or directory specified by OldFilename to the name given by NewFilename.
The OldFilename must be the path of an existing file or directory.
The NewFilename must not be the name of an existing file or directory.
You can use the function to move a file from one directory or device to another by giving a different path in the NewFileName argument.
However, you cannot use the function to move a directory. Directories can be renamed, but not moved.

Return 0 on success.
On Error, returns
-1, Access denied or read only, or NewFileName already exists or could not be created (invalid path),
or oldname is a directory and newname specifies a different path.
-2, OldFilename File or Path not found.
-3, NewFileName contains invalid characters.
-4, Unknown error.

[VoodooFX]

I wish DBASE could store masks. :)

[StainlessS]

I wish DBASE could store masks. :)

Thats always been in the back of my mind.

EDIT: Well not for DBase, but for RT_ Arrays, would likely also have to implement type UShort [maybe as BIN16] in both Array and DBase, for unsigned 16 bit int.
Currently implements non standard types BIN [8 bit unsigned int] and [EDIT: Intended] for internal RT_stats use only, Double [returned to AVS as 32 bit float].

[VoodooFX]

For some reason I thought that it supports it and I'll make like logodata storage out of it.
I can store stuff to ebmp but there wouldn't be coords and other settings.

[StainlessS]

Roughly whats implemented for array [additional to the array itself] (same for DBase)


*****************************************************
********** RT_ARRAY FUNCTIONS ***********
*****************************************************

RT_Stats ARRAY functions allow fast access to a file based Array of up to 3 dimensions of fixed element type,
Bool, Int, Float, String, or BIN. BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in the Array
as an unsigned 8 bit Int, upper 24 bits ignored.
The maximum possible file size is about 1TB, and depends on number and size of dimensions and element type and size, also
depends upon avilable disk space.
Element types Bool and BIN require 1 byte, Int and Float 4 bytes, and String is of fixed maximum length (user chosen) up to
256KB (256 * 1024, no nul terminating character is stored in an Array String).
Maximum possible size of dimension 1 is $7FFFFFFE but depends upon available disk space and also limited to max 1TB.
RT_ArrayAlloc, allows you to create an array file with a fixed number of dimensions and where the 2nd and/or 3rd dimensions
of a multi-dimensional array are fixed in size but the size of the first dimension can 'grow', single dimension arrays can
'grow' too. You can create an array where the first dimension is 0 (any other dimensions fixed), ie no elements pre-allocated
in array file, in this case you can use RT_ArrayExtend to 'grow' the first dimension of the array so that it can be used. In addition,
single dimension arrays can also use RT_ArrayAppend to add elements to the end of the array.
An Array file allows up to 1024 Attributes of types Int or Float, which can be used for whatever a user desires, these
attributes are stored in the Array file header which is 32768 bytes, so the actual data starts at $8000 hex.
In addition to user attributes, an Array has currently 10 user String Attributes that can be be used for whatever you want, the
string attributes are currently a maximum of 1024 characters in length.
In addition to user attributes and user string attributes, there are currently 128 ID's (Int of Float) that are intended to be used
by either RT_Stats itself, or script developers, intended function of ID's are to in some way eg link an array to a particular
clip, by storing maybe Width, Height, Framecount and perhaps other values, the stored ID's might be used to Validate an array
against a clip.
You could use the Array functions to return multiple results from a script function to the caller via a caller supplied Array
filename, perhaps with some kind of status variable returned directly.


EDIT: IIRC, in pre AVS+ [not sure about avs+], Array element and DBase field reads were faster than access to Global Vars. [and that was probably on Core Duo with SATA I drive].

[StainlessS]

Here a few functions than might be handy. [RT_Stats required]


# ChrEatWhite is posted elsewhere, used in Chrxxxxx funcs library.
Function ChrEatWhite(String S) {i=1 C=RT_Ord(S,i) While(C==32||C>=8&&C<=13) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function StrSQuote(string S) {Return "'" + S + "'" } # Surrounded String with Single Quotes.
Function StrDQuote(string S) {Return Chr(34) + S + Chr(34) } # Surrounded String with Double Quotes.
Function StrTrim(String S, Int "Md") {Md=Default(Md,0) S=((Md!=2)?S.RevStr.ChrEatWhite.RevStr:S) Return((Md!=1)?S.ChrEatWhite():S) } # Md=1 Trim WhiteSpace from right : Md=2 Trim WhiteSpace from Left : Md=0(Default) or anything else, Left + Right Trim
Function StrLeftAlign(String S,int "PadLen") {PadLen=Default(PadLen,S.StrLen) Return S.StrTrim(0).RT_StrPad(PadLen)} # Left Align S, result SPACE padded on Right to at least PadLen(default StrLen(S)) length.
Function StrRightAlign(String S,int "PadLen") {PadLen=Default(PadLen,S.StrLen) S = S.StrTrim(0) Return RT_StrPad("",PadLen-S.StrLen) + S} # Right Align S, result SPACE padded on Left to at least PadLen(default StrLen(S)) length.
Function StrCenterAlign(String S,int "PadLen") {PadLen=Default(PadLen,S.StrLen) S=S.StrTrim(0) Return(RT_StrPad("",(PadLen-StrLen(S))/2)+S).RT_StrPad(PadLen)} # Center Align S, result SPACE padded evenly on both Left & Right to at least PadLen(default StrLen(S)) length.
Function StrAlign(String S,Int "Md",Int "PadLen") {Md=Default(Md,0) Return(Md==1)?S.StrRightAlign(PadLen):(Md==2)?S.StrLeftAlign(PadLen):S.StrCenterAlign(PadLen)}# Md=1 Right Align : Md=2 Left Align : Md=0(Default) or anything else, Center Align. Result is L/R/Center padded to at least PadLen(default StrLen(S)) Length.


Demo client


############ DEMO STUFF ###################
#Import(.\SomethingOrOther.avsi")

Src =" some test text "

RT_DebugF("LEN=%d",Src.Strlen)

Global SBUF = StrSQuote(Src).RT_StrPad(48) + "SOURCE STRING # (len between quotes=32)" + "\n" + Chr(10)

Add2BUF(Src, "StrTrim(S) # Left & Right Trim : Default " )
Add2BUF(Src, "StrTrim(S,0) # Left & Right Trim" )
Add2BUF(Src, "StrTrim(S,1) # Right Trim" )
Add2BUF(Src, "StrTrim(S,2) # Left Trim" )

Add2BUF(Src, "StrLeftAlign(S) # Left Align, Default Pad to original length" )
Add2BUF(Src, "StrLeftAlign(S,42) # Left Align, Pad to 42" )
Add2BUF(Src, "StrLeftAlign(S,20) # Left Align, Pad to 20" )

Add2BUF(Src, "StrRightAlign(S) # Right Align, Default Pad to original length" )
Add2BUF(Src, "StrRightAlign(S,42) # Right Align, Pad to 42" )
Add2BUF(Src, "StrRightAlign(S,20) # Right Align, Pad to 20" )

Add2BUF(Src, "StrCenterAlign(S) # Center align, Default Pad to original length" )
Add2BUF(Src, "StrCenterAlign(S,42) # Center Align, pad to 42" )
Add2BUF(Src, "StrCenterAlign(S,20) # Center Align, pad to 20" )

Add2BUF(Src, "StrAlign(S) # Center Align, Default Pad to original length" )
Add2BUF(Src, "StrAlign(S,0) # Center Align, Default Pad to original length" )
Add2BUF(Src, "StrAlign(S,1) # Right Align, Default Pad to original length" )
Add2BUF(Src, "StrAlign(S,2) # Left Align, Default Pad to original length" )

Add2BUF(Src, "StrAlign(S,0,42) # Center Align, Pad to 42" )
Add2BUF(Src, "StrAlign(S,1,42) # Right Align, Pad to 42" )
Add2BUF(Src, "StrAlign(S,2,42) # Left Align, Pad to 42" )

Add2BUF(Src, "StrAlign(S,0,20) # Center Align, Pad to 20" )
Add2BUF(Src, "StrAlign(S,1,20) # Right Align, Pad to 20" )
Add2BUF(Src, "StrAlign(S,2,20) # Left Align, Pad to 20" )

BlankClip(width=1152,Height=640)

Subtitle(SBUF,Font="Courier New",lsp=0)

Return Last

Function Add2BUF(String S, String FnS) {
s3 = Eval(Fns)
Global SBUF = SBUF + StrSQuote(s3).RT_StrPad(48) + FnS + "\n" + Chr(10)
}


Result
https://i.postimg.cc/fygRfzs5/TESTING-00.jpg (https://postimages.org/)

EDIT: There are String Trim type functions implemented in AVS+, this does not use them. {TrimLeft, TrimRight, TrimAll}
EDIT: You can also use PadLen=0 with some, to not pad at all.

[rgr]

Hi.
How big of an array can RT_ArrayAlloc hold?
Avisynth works for a few tens of thousands, but for a few hundred thousand it crashes :)

[StvG]

Are you sure you didn't hit your memory limit? In this case also RT_ArrayAlloc will crash/fail.

[StainlessS]

RT_Array's are file based, nothing to do with Avs+ arrays.
Same for RT_DBase's.


*****************************************************
********** RT_ARRAY FUNCTIONS ***********
*****************************************************

RT_Stats ARRAY functions allow fast access to a file based Array of up to 3 dimensions of fixed element type,
Bool, Int, Float, String, or BIN. BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in the Array
as an unsigned 8 bit Int, upper 24 bits ignored.
The maximum possible file size is about 1TB, and depends on number and size of dimensions and element type and size, also
depends upon avilable disk space.
Element types Bool and BIN require 1 byte, Int and Float 4 bytes, and String is of fixed maximum length (user chosen) up to
256KB (256 * 1024, no nul terminating character is stored in an Array String).
Maximum possible size of dimension 1 is $7FFFFFFE but depends upon available disk space and also limited to max 1TB.
RT_ArrayAlloc, allows you to create an array file with a fixed number of dimensions and where the 2nd and/or 3rd dimensions
of a multi-dimensional array are fixed in size but the size of the first dimension can 'grow', single dimension arrays can
'grow' too. You can create an array where the first dimension is 0 (any other dimensions fixed), ie no elements pre-allocated
in array file, in this case you can use RT_ArrayExtend to 'grow' the first dimension of the array so that it can be used. In addition,
single dimension arrays can also use RT_ArrayAppend to add elements to the end of the array.
An Array file allows up to 1024 Attributes of types Int or Float, which can be used for whatever a user desires, these
attributes are stored in the Array file header which is 32768 bytes, so the actual data starts at $8000 hex.
In addition to user attributes, an Array has currently 10 user String Attributes that can be be used for whatever you want, the
string attributes are currently a maximum of 1024 characters in length.
In addition to user attributes and user string attributes, there are currently 128 ID's (Int of Float) that are intended to be used
by either RT_Stats itself, or script developers, intended function of ID's are to in some way eg link an array to a particular
clip, by storing maybe Width, Height, Framecount and perhaps other values, the stored ID's might be used to Validate an array
against a clip.
You could use the Array functions to return multiple results from a script function to the caller via a caller supplied Array
filename, perhaps with some kind of status variable returned directly.

***
***
***

RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,int "Dim3"=0,int "StringLenMax"=256)

Creates a file to hold a one, two or three dimensional Array of elements of type Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in array as an unsigned 8 bit int, upper 24 bits ignored.

Filename, Array Filename, no default.

Type, int, default 1(0->4). 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN. Type of the array elements, fixed for lifetime of array.
v1.40, NOTE, There is also a type 5 (double) which is not of use to the user, although a type 5 array will accept Float
(as double is not an Avisynth type). Type 5 is implemented for use by RT_Stats internal functions to store higher precision results.
Array type double (5) will be returned as Float to Avisynth script.

Dim1, Dim2, Dim3, all Default 0, range 0 or greater. Number of array elements per dimension (max 3 dimensions).
Where Dim1 & Dim2 & Dim3 all at default 0, will be 1 dimensional unallocated Array.
Dim1 governs the pre-allocated size of the array first dimension.
The first dimension can 'grow' for 1, 2 or 3 dimensional arrays. (RT_ArrayExtend, or for single dimension only RT_ArrayAppend)
Dim2 and Dim3 fix the number of array dimensions for the lifetime of the array, and also the size of dimensions 2 and 3.

Dim1 Dim2 Dim3 Dimension 1 is [EDIT: default] 0, NO elements allocated, can use RT_ArrayExtend to 'grow' size of first dimension for use.
0 0 0 SINGLE dimension array. Can ALSO use RT_ArrayAppend to add elements to end of the single dimension array
0 0 >0 ILLEGAL, Throws error condition.
0 >0 0 TWO dimension array, Dim2 fixes size of 2nd dimension. CANNOT use RT_ArrayAppend.
0 >0 >0 THREE dimension array, Dim2 & Dim3 fix size of 2nd & 3rd dimensions. CANNOT use RT_ArrayAppend.

Dim1 Dim2 Dim3 Dimension 1 is Dim1 in size, elements ARE pre-allocated, initialized to zeros and can be used. Can also 'grow'.
>0 0 0 SINGLE dimension array. Can ALSO use RT_ArrayAppend as well as RT_ArrayExtend.
>0 0 >0 ILLEGAL, Throws error condition.
>0 >0 0 TWO dimension array, Dim2 fixes size of 2nd dimension. Can ALSO use RT_ArrayExtend.
>0 >0 >0 THREE dimension array, Dim2 & Dim3 fix size of 2nd & 3rd dimensions. Can ALSO use RT_ArrayExtend.
(initialized to zeros, Bool=False,Float=0.0,String="").

StringLenMax, default 256(1 -> (256 *1024)). Fixed Maximum string length excluding nul term. Error if string length exceeds.
Only used for Type=3 (string).

Returns maximum size of dim1 dimension given current flesystem and disk space available to the user.
All RT_ArrayXXXX() errors produce an error alert.

It makes little sense to set either Dim2 or Dim3 to 1 although this will not cause an error condition, just be less efficient,
a 3 dimensional array with all dimensions 1, refers only to a single element but will take slightly longer to calculate the
linear array offset.

There are quite a few additional RT_Array functions.

Also, RT_DBase,

*****************************************************
********** DBASE FUNCTIONS ***********
*****************************************************

RT_Stats DBASE functions allow fast access to a file based DataBase where each record can have up to 1024 fields of variable type,
Bool, Int, Float, String, or BIN. BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in the DBase field
as an unsigned 8 bit Int, upper 24 bits ignored. there is also a type double used in DBase, this is only of use to internal RT_Stats
functions, setting or getting type double takes or returns type Float (as Avisynth cannot handle double precision float).
The maximum possible DBase file size is about 1TB, and the number of records is restricted to $7FFFFFFE, about 2 Billion.
Field types Bool and BIN require 1 byte, Int and Float 4 bytes, and String is of fixed maximum length (user chosen) up to
256KB ((256 * 1024), no nul terminating character is stored in a DBase String). The private type Double requires 8 bytes.
A DBase file allows up to 1024 Attributes of types Int or Float, which can be used for whatever a user desires, these
attributes are stored in the DBase file header which is 32768 bytes, so the actual data starts at $8000 hex.
In addition to user attributes, a DBase has currently 10 user String Attributes that can be be used for whatever you want, the
string attributes are currently a maximum of 1024 characters in length.
In addition to user attributes and user string attributes, there are currently 128 ID's (Int of Float) that are intended to be used
by either RT_Stats itself, or script developers, intended function of ID's are to in some way eg link a DBase to a particular
clip, by storing maybe Width, Height, Framecount and perhaps other values, the stored ID's might be used to Validate a DBase
against a clip.
You could use the DBase functions to return multiple results from a script function to the caller via a caller supplied DBase
filename. The DBase functions also make it more likely that script functions could be written to eg find where edits between
two clips have been made, or previously impossible/implausible tasks performed.

***
***
***

RT_DBaseAlloc(String Filename,Int Records,String TypeString,Int "StringLenMax"=256)
Creates a file to hold DataBase records. Each record can have up to 1024 fields of varying type, Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in DBase field as an unsigned 8 bit int, upper 24 bits ignored.
v1.40, NOTE, There is also a type double which is not of use to the user, although a type double field will accept Float
(as double is not an Avisynth type). Type double is implemented for use by RT_Stats internal functions to store higher precision results.
Type double will be returned as Float to Avisynth script.

Filename, DBASE Filename, no default.

Records, int zero or more. Number of records to create, initalized to zeros (initialized to zeros, Bool=False,Float=0.0,String="").
If you dont know the final size of your DBase, you can create a DBase with 0 records, and thereafter use RT_DBaseAppend to add
records to the end of the DBase.

TypeString, string holding record construction data. One character for each field, valid characters are "bifsn" or Uppercase.
'b' = BOOL, 'i' = INT, 'f' = FLOAT, 's' = STRING, 'n' = BIN. (the private type double uses 'e' or 'E' in the TypeString)
The 's' STRING construction character can be followed by numeric digits describing fixed maximum string length (excluding
any nul term character), otherwise StringLenMax is used by default instead.
For no particular reason, I have limited the maximum string length to 256K, ie 256 * 1024, throws an error if greater.

StringLenMax Default=256. Default length of String fields if no explicit string length defined following TypeString S constructor char.

Example TypeString:-
"ifffs1024bsn", 1st field Int, 3 Float fields, 1 string field of length=1024, 1 Bool field, 1 string field using length of
StringLenMax, and 1 BIN field. 8 Fields in all (0->7).

Returns number of records created.
All RT_DBaseXXXX() errors produce an error alert.


RT_ARRAY_Functions:- http://avisynth.nl/index.php/RT_Stats#RT_ARRAY_Functions
DBASE_Functions:- http://avisynth.nl/index.php/RT_Stats#DBASE_Functions

EDIT: RT_ Arrays and DBase files cannot store clip variables; but as file based, can be re-used via another script (if not deleted).

[rgr]

Thanks -- 1TB should be enough.
What will be faster -- RT_Stats ARRAY or DBASE?

[StainlessS]

About the same.
Dont know why, but I nearly always prefer to use DBase. [unless 3 dimensional array]

EDIT:
Timings on earlier version RT_Stats, AVS 2.6 Std, and on Core 2 machine (Q9550 CPU) with SATA 2,
Was quicker to access DBase record/field than to access same type var as AVS Global variable.
(but slower than access AVS Local var)

[rgr]

RT_ArrayAlloc(string FileName,int "Type"=1,int "Dim1"=0,int "Dim2"=0,int "Dim3"=0,int "StringLenMax"=256)

Dim1 governs the pre-allocated size of the array first dimension.
The first dimension can 'grow' for 1, 2 or 3 dimensional arrays. (RT_ArrayExtend, or for single dimension only RT_ArrayAppend)

After "RT_ArrayAlloc(arrname,2,1000)" array size (RT_ArrayGetDim(arrname)) will be 0 or 1000? (If 1000, then they will be filled with zeros?)

Edit: OK, I see: "Dimension 1 is Dim1 in size, elements ARE pre-allocated, initialized to zeros and can be used."

[StainlessS]

Edit: OK, I see: "Dimension 1 is Dim1 in size, elements ARE pre-allocated, initialized to zeros and can be used."
Right on brother.
"initialized to zeros", for float <type=2> init to 0.0. (int=0, string="").

[rgr]

Right on brother.
"initialized to zeros", for float <type=2> init to 0.0. (int=0, string="").

It would be nice to be able to choose what element it should be filled with, because for an array of ~100k elements it takes 22mins :)

[StainlessS]

because for an array of ~100k elements it takes 22mins
Not for me, for array took ~1:51 and for DB ~1:52.
ArrayTest.avs

ARR="D:\Z\Test.Arr"
SZ=100*1000
F=42.0
RT_ArrayAlloc(ARR,2,SZ)
Start=RT_TimerHP()
for(i=0,SZ-1) { RT_ArraySet(ARR,F,i) }
End=RT_TimerHP()
Tim = End-Start
M=int(Tim/60) S=Tim-(m*60)
Msg=RT_String("Done: Time=%d:%02.3f",M,S)
MessageClip(Msg)


DBaseTest.avs

DB="D:\Z\Test.DB"
SZ=100*1000
F=42.0
TypeString="f"
RT_DBaseAlloc(DB,SZ,TypeString)
Start=RT_TimerHP()
for(i=0,SZ-1) { RT_DBaseSet(DB,i,F) }
End=RT_TimerHP()
Tim = End-Start
M=int(Tim/60) S=Tim-(m*60)
Msg=RT_String("Done: Time=%d:%02.3f",M,S)
MessageClip(Msg)


Mine, on i7-8700 CPU. <not that fast on single core, Base Freq 3.2GHz, Max Turbo Frequency 4.6GHz>

But I shall keep your comment in mind on next update (but dont hold your breath, last update was 2018, I think)

EDIT: Storage on Gen 3 NVMe. [EDIT: Due to <EDIT: OS and physical drive> disk caching, spinning drive should not be much different]


RT_ArraySet(string FileName,var Var,int ix1,int "ix2",int "ix3")
Filename, Array Filename, no default.
Var, a variable of Type as set in RT_ArrayAlloc.
ixl, int, zero relative. 1st subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,1)-1.
ix2, int, zero relative. 2nd subscript of array. Valid range 0 -> RT_ArrayGetDim(FileName,2)-1.
Only valid if 2 or 3 dimensional array. For single dimension array, can omit, supply RT_Undefined(), or -1.
ix3, int, zero relative. 3rd subcript of array. Valid range 0 -> RT_ArrayGetDim(FileName,3)-1.
Only valid if 3 dimensional array. For 1 or 2 dimension array, can omit, supply RT_Undefined(), or -1.
Returns total number of elements in array.

If single dimension array, only specifiy the 1st dimension ix1, dont supply ix2 and ix3 as 0. {for 2 dim array, only supply ix1 and ix2}
EDIT: From Array docs for RT_ArrayAlloc,
It makes little sense to set either Dim2 or Dim3 to 1 although this will not cause an error condition, just be less efficient,
a 3 dimensional array with all dimensions 1, refers only to a single element but will take slightly longer to calculate the
linear array offset.

EDIT: Also, if you hard code F as 42.0 instead of using Var F, then will not have to read F local variable at every iteration setting the array element <or DB field>.

EDIT: Output single dim array to a text file [quite slow, tested only on Float array]

ARR="D:\Z\Test.Arr"
FILE="D:\Z\TestArr.txt"
NDIMS = RT_ArrayGetDim(ARR,Dim=0) # Number of dimensions in array
Assert(NDIMS==1, "Error, single dimension array only please")
SZ = RT_ArrayGetDim(ARR,Dim=1) # Num Array elements in single dimension array
TYPE=RT_ArrayGetType(ARR) # 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN. # NOTE, There is also a type 5 (double)
Fmt = "%d ] %" + Select(TYPE, "b","d","f","s","d","f") + "\n"
RT_FileDelete(FILE)
Start=RT_TimerHP()
for(i=0,SZ-1) { RT_WriteFile(FILE,Fmt, i, RT_ArrayGet(ARR,i), Append=True) }
End=RT_TimerHP()
Tim = End-Start
M=int(Tim/60) S=Tim-(m*60)
TYPE_S = RT_ArrayTypeName(Type) # Name of type
Msg=RT_String("Written:- Time=%d:%02.3f single Dim array[%d] of type='%s' to file='%s'",M,S,SZ,TYPE_S,FILE)
MessageClip(Msg)


eg Produces msg clip, "Written:- Time=29:11.995 single Dim array[100000] of type='Float' to file='D:\Z\TestArr.txt'"

0 ] 42.000000
1 ] 42.000000
2 ] 42.000000
3 ] 42.000000
4 ] 42.000000
5 ] 42.000000
6 ] 42.000000
7 ] 42.000000
8 ] 42.000000
9 ] 42.000000
10 ] 42.000000
11 ] 42.000000
12 ] 42.000000
13 ] 42.000000
# etc

A bit on the slow side, maybe the formatted string production is what takes the time.
About 57 elements per second written to file.
Result text file, 1,845KB in size.

[rgr]

Not for me, for array took ~1:51 and for DB ~1:52.
ArrayTest.avs

ARR="D:\Z\Test.Arr"
SZ=100*1000
F=42.0
RT_ArrayAlloc(ARR,2,SZ)
Start=RT_TimerHP()
for(i=0,SZ-1) { RT_ArraySet(ARR,F,i) }
End=RT_TimerHP()
Tim = End-Start
M=int(Tim/60) S=Tim-(m*60)
Msg=RT_String("Done: Time=%d:%02.3f",M,S)
MessageClip(Msg)



Hmmm....

"Done: Time=15:5.778"

Ryzen5700G+Win10+NVME
CPU usage was around 0% :)

[StainlessS]

Done it again, this time only 1:39, with Comodo Antivirus running.

https://i.postimg.cc/MTGhQprJ/time.jpg (https://postimg.cc/67DjstXM)image sharing (https://postimages.org/)
AVS running 100% on single core, other stuff file writing [Drive D: 1%] + other system stuff.

EDIT: I used W10, + Vdub2 as app.

Does your Ryzen5700G thingy have similar to 12 gen intel i7,
performance cores + efficiency cores ?
Is it perhaps running on efficiency core ?

EDIT: In performace graph, right click "change graph to logical cores".
EDIT: I think with mine, every 2nd graph is a logical core, the physical cores are the ones doing something.

EDIT: With 12th gen intel i7, 16 performance cores [EDIT: half of those being logical] shown on upper graph, bottom 4 cores [EDIT: half of those being logical] are efficiency thingies.

EDIT: On your Task Manager Details TAB, is your RightClick Player_appName/Set_Priority set to anything other than Normal ?

[rgr]

https://gcdnb.pbrd.co/images/eCTifyufvmFY.png?o=1
Using VDub changes nothing.
My Ryzen has all the performance cores :)

[rgr]

After raising priority to high:
"Done: Time=9:56.085"

[StainlessS]

No idea what you are showing me there (ffmpeg.exe ???).
Show the CPU cores graph.

Also,
EDIT: On your Task Manager Details TAB, is your RightClick Player_appName/Set_Priority set to anything other than Normal ?
answer above.

EDIT: Gone t' pub.

[rgr]

Yes, I'm running the script through ffmpeg.
Now I've running compression, so I'll paste it later. But don't expect 0% to be visible there :)

EDIT: On your Task Manager Details TAB, is your RightClick Player_appName/Set_Priority set to anything other than Normal ?

After raising priority to high:
"Done: Time=9:56.085"

[StainlessS]

Now I've running compression, so I'll paste it later. But don't expect 0% to be visible there
I expect your AVS instance to be running on a single graph core, your CPU % will be for total ALL cores.
[Mine was 100% on single core [top left core graph], but 16% total CPU]

Yes, I'm running the script through ffmpeg.
Cant for the life of me figure out why.

[rgr]

I expect your AVS instance to be running on a single graph core, your CPU % will be for total ALL cores.
[Mine was 100% on single core [top left core graph], but 16% total CPU]

https://gcdnb.pbrd.co/images/JcDKcVJoakvU.png?o=1

Cant for the life of me figure out why.

Why not?

[StainlessS]

Dont know what to make of your graphs. Seems to be sharing AVS work between several cores.
Am curious as to whether you get same via VDub2.

[rgr]

Same. These CPU loads are not from AVS, but from other programs.

[StainlessS]

Maybe not a good idea to be running other apps when you wanna measure CPU usage of a single app.
(is some kind of server computer ?)

[rgr]

It's just that the antivirus is slightly loading the processor.

But that doesn't matter after raising the priority for the program.

[rgr]

I'm starting to think it might be the disk write process that's causing it. But it's a fast NVME and generally doesn't cause problems, so maybe some other problem accessing the data.

[rgr]

Technical question about RT_FrameMovement

How does it measure the difference between two frames?

Assuming for simplicity that we have a 2x2 block (and only luma), then the measured total difference for the block:
1 -1
-1 1 (this is the difference between luma in both frames)
will be 4 (sum of absolute differences)?

And for the block:
0 0
0 4
also 4?

[StainlessS]

RT_FrameMovement(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0,
int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
bool "AltScan"=false,Bool "ChromaI"=False,int "Matrix"=(c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600?3:2),
int "BlkW"=64,int "BlkH"=BlkW, Int "OLapX"=BlkW/2, Int "OLapY"=BlkH/2,Float "BlkTh"=1.0,String "Prefix"="")

Returns FLOAT value (0.0 -> 255.0) movement detection between clip c frame n area x,y,w,h, and clip c2 frame n2 area x2,y2,w,h.
The frames are sectioned into blocks of BlkW x BlkH with block overlaps of OLapX and OLapY, and all blocks are differenced, the result
is the greatest difference for any one block comparison.
Is more sensitive to localized movement. Best control of localized movement sensitivity is by changing blk size, lower is more sensitive.
If Altscan is true, then only every other horizontal scanline in BLKH will be scanned.

Args:-
c, c2, comparison clips, usually the same clip.
n, default current_frame, frame from clip c.
n2, default current_frame, frame from clip c2.
ChromaWeight, default 1.0/3.0. Range 0.0 -> 1.0.
Difference method control when comparing blocks.
Y8, returns same as RT_LumaDifference. ChromaWeight ignored.
YUV,
Weighting applied YUV chroma:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
RGB,
If ChromaWeight > 0.0, then returns same as RT_RGBDifference() [average RGB pixel channel difference].
If ChromaWeight == 0.0, then returns same as RT_LumaDifference() using Matrix arg to convert RGB to YUV-Y Luma.
x,y,w,h, all default 0 (full frame c).
x2,y2. x2 defaults x, y2 defaults y, for clip c2.
Altscan, default false. If true then scan only every other scanline starting at clip c y coord, and clip c2 y2 coord.
ChromaI, default false. If YV12 and ChromaI, then do YV12 interlaced chroma scanning.
Ignored if not YV12. Both YV12 clips must be same, cannot difference one that is progressive and one that has interlaced chroma.
Matrix, default (c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600)?3:2
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709.
Default = (c.Width > 1100 OR c.Height>600 OR c2.Width > 1100 OR c2.Height>600) then 3(PC709) else 2(PC601). YUV not used
Matrix only Used if RGB and ChromaWeight == 0.0, where returns same as RT_LumaDifference().
BlkW, default 64, Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
BlkH, default BlkW, Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
OlapX, default BlkW/2, Horizontal block overlap. 0 -> BlkW/2
OlapY, default BlkH/2, Vertical block overlap. 0 -> BlkH/2
BlkTh, default 1.0. Governs extra information returned when Prefix != "", as local variables.
Prefix, default "" is do not set additional local variables.
If set eg "FM_", then sets local variables as below.
"FM_Xoff" = X Coord of block in Clip c that was greatest.
"FM_Yoff" = Y Coord of block in Clip c that was greatest.
"FM_PercAboveTh" = Percentage of Blocks that were above BlkTh.
"FM_BlkAveDf" = Average Block difference.
"FM_nAboveTh" = Number of Blocks that were above BlkTh.
"FM_TotalBlks" = Total Number of Blocks.

NOTE, 'x2' and 'y2' default to 'x' and 'y' respectively.
c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.


As in blue above, Result range 0.0 -> 255.0.
As for both your examples except divided by the number of pixels in block, so in both your instances result is 1.0.
Sum Of Absolute Difference / nPixels, ie SAD / (w * h). [EDIT: SAD / (Blkw * Blkh), or if AltScan=true ie skip every other scanline then SAD / (Blkw * (Blkh/2))]

I've used RT_FrameMovement in script functions,
CensorSwap :- https://forum.doom9.org/showthread.php?p=1935658#post1935658
SecurityCamExtractMotion.AVS :- From RT_Stats AVS directory
Duplicity2 :- https://forum.doom9.org/showthread.php?t=175357&highlight=duplicity2

Where in eg Duplcity2(), we use 3 simutaneous detectors,
RT_FrameDifference: Frame Global differencing. [whole frame]
# below, more local differencing
RT_FrameMovement: Block differencing (block of greatest difference in entire frame).
RT_LumaPixelsDifferent: Big single pixel difference.

EDIT: NOTE, RT_Stats is still 8 bit only. [EDIT: However, note!, you dont need eg 10 bit to detect dupes, 8 is quite sufficient but may make script a bit more complicookered :) ]

[rgr]

Yes, I read the manual, I was looking for information about the calculation method.

I'm currently experiencing the disadvantages of this method.

For example:
- 256px differing by 1 each = RT_FrameMov will return 1,
- 254 identical pixels, and two differing by 128 will also return 1.

The first example is typical for compressed videos and is imperceptible on the screen, while the second one rather shows movement in a block and is easy to see on the screen. Of course, these are just quick extreme examples, but I see a problem here. I think that the bigger the difference, the bigger the "point penalty" can be applied, i.e. difference 1 is 1 point, difference 2 is 4 points, difference 3 is 9 points, etc.

(Edit: using dfttest on the source clip reduces the problem, but slows down the whole operation a lot)

[StainlessS]

That is the very reason that I use 3 detectors in Duplicity2.
Sounds like you wanna experiment with both of these,

# below, more local differencing
RT_FrameMovement: Block differencing (block of greatest difference in entire frame).
RT_LumaPixelsDifferent: Big single pixel difference.



RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int"y2"=y,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns amount of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Thresh Default 0, returns number of pixels that are not exactly the same.
Return value is in range 0.0 (meaning none) to 255.0 meaning 100% of pixels.
v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.


From Duplicity2 thread:- https://forum.doom9.org/showthread.php?p=1838153#post1838153

/*
Duplicity2(), v2.13, by StainlessS @ Doom9:- https://forum.doom9.org/showthread.php?t=175357

A two faced scheming and double dealing [on your behalf] dupe tool.

Three modes of operation,
0) Write duplicate Frames file only (as single frames or ranges).
1) Replace duplicates with exact duplicates. (Can also write Frames file).
2) Replace duplicates with Interpolated or Blended frames (control via MaxDuplen and MaxInterp). (Can also write Frames file).

Req:-
RT_Stats v2.00_Beta_11 + CallCmd (c) StainlessS,
GSCript, Grunt, (c) Gavino,
MaskTools2, MvTools2, (c) Manoa/Fizick/Pinterf & Others (Pinterf Versions).
RemoveGrain, (c) Kassandro.

GScript not needed if Avs+ [(c) Ultim, Pinterf and others.].
If CallCmd available then will AutoDelete DBase on clip closure, otherwise requires manual deletion.

ColorSpace:- All v2.6 standard PLANAR colorspaces except YV411, no support for YUY2, RGB24 or RGB32.

Function Duplicity2(clip c,"Mode"=0,Float "ThG"=1.0,Float "ThB"=ThG*2.0, Int "ThP"=64,Float "IgP"=0.0,Int "ThK"=7,
\ Int "MaxDupLen"=9,Int "LagMax"=MaxDupLen,Int "MaxInterp"=Min(9,MaxDupLen),
\ int "BlkW"=64,int "BlkH"=BlkW,int "OLapX"=BlkW/2,Int "OLapY=BlkH/2",
\ Int "X"=0,Int "Y"=0,Int"W"=0,Int "H"=0,Float "CW"=0.0,Bool "ChromaI"=False,Bool "AltScan"=False,
\ Int "SPad"=16, Int "SSharp"=1, Int "SRFilter"=4, [* MSuper *]
\ Int "ABlkSize"=16, Int "AOverlap"=4, Int "ASearch"=3,Int "ADct"=0, [* MAnalyse *]
\ Int "RBlkSize"=8, Int "ROverlap"=2, Int "RthSAD"=100, [* MRecalculate *]
\ Float "Iml"=200.0, Bool "IBlend"=True, Int "IthSCD1"=400, Int "IthSCD2"=130, [* MFlowInter *]
\ Int "OthSCD1"=400,Int "OthSCD2"=130, [* Dupe Detect Override @ Scene Change *]
\ String "OverRide"="",String "Frames"="Frames.txt",Bool "Ranges"=True,Bool "FrameAsRange"=False,
\ Bool "Show"=True,Bool "Verb"=True,Bool "ShowBlk"=True,Bool "Dim"=True,Bool "ShowDot"=False,
\ Bool "InterpFast"=True,String "DBase"=""
\ )

Duplicity2() uses 3 RT_Stats measurement functions, RT_FrameDifference(), RT_FrameMovement(), and RT_LumaPixelsDifferent().
the first two may difference using either luma only, or with a combined weighted chroma difference. Also see CW,
and ChromaI args. RT_LumaPixelsDifferent() differences luma only.

DeDuppers use some method to determine if a frame is a duplicate of its predecessor, a typical measuring method might be
RT_FrameDifference (average absolute difference per pixel), unfortunately the result of such measure may be due to a small
frame global change (lots and lots of pixels change by a small amount), or a smaller number of pixels change by a
larger amount, both these cases could produce the exact same result. Another case may be that of a very small number of
pixels change by a very large amount (eg a person blinks an eye with remainder of frame static) and such measure might seem
to be a duplicate as it could produce much smaller numbers than a frame global change of a small amount.

Duplicity2() uses a triple measurement system (maybe it should be called Triplicity), Primary measurement (RT_FrameDifference)
to detect frames 'Unique' to their immediate predecessor (large or 'frame Global' changes), and second and third methods to
determine if 'Non-Unique' frames are either 'Low Motion' or 'Duplicate' frames. The second method is a block differencing
method (RT_FrameMovement) where it splits the frame up into BlkW by BlkH blocks and finds the maximum average pixel difference
of any one block. The third measurement method is RT_LumaPixelsDifferent (result 0.0->255.0 where 255.0 is equivalent to 100.0%,
user args for this method actually require Duplicity2 arg 'IgP' in range 0.0->100.0%, and also show results in range 0->100.0%).
RT_LumaPixelsDifferent, compares frames and shows percent of pixels whose corresponding absolute luma pixel difference is
greater than Int arg 'ThP', if this percent difference is greater than 'IgP', then it is deemed to be a 'Low motion' frame and if
less or equal to 'IgP' then is a duplicate ('IgP' can be 0.0, meaning any pixels at all with difference greater than 'ThP' is a
'Low Motion' Frame). Where both 'ThP' and 'IgP' are 0 and 0.0 respectively, will show percentage of pixels that are not Exactly the same.
'IgP' could be set to 0.0 (Default) in which case the function is governed solely by any pixel differences exceeding 'ThP', or can set
'IgP'= eg 0.001% meaning the it will disregard as noise 1 pixel difference exceeding 'ThP' in every 100,000 pixels
[eg 1 pixel in every 316x316 region]. It is probably easiest to always use IgP of 0.0 and only adjust ThP.

The function examines the DBase to determine if status for current frame is already known, if not already known then,
the function determines if the current frame is a 'Unique' frame or not (via 'ThG' RT_FrameDifference to predecessor), if Unique,
then the job is done and is considered a Non-Duplicate, these Unique frames are marked in a DBase as Unique. If a frame is
'Non-Unique', then the function scans both backwards and forwards looking for 'Unique Start' and 'Unique End' frames (and if
necessary, marking them in DBase). This Backwards/Forwards scanning permits jumping around in the clip for perusal and getting
exact same results as if travelling forwards only (even backwards traversal only, should work just fine). When the
'Unique Start' and 'Unique End' frames are established, we then know the extent of the unknown (unvisited) frames, and can scan
and measure those with the second and third measurement methods (RT_FrameMovement and RT_LumaPixelsDifferent) to determine
Low Motion/Dupe status. When determining Low Motion/Dupe status, we compare with an 'Anchor Frame', which at the beginning of a
'Non-Unique' sequence is the 'Unique Start' frame. When scanning a non-unique sequence the 'Anchor Frame' is the predecessor frame
to the very first frame in the current sub sequence of the 'Non-Unique' sequence, so potential duplicates are not necessarily
compared with their immediate predecessor. When a frame is determined to be 'Low Motion' (not a dupe), any detected duplicates are
written to the DBase and also to the output Frames file, and it switches to detecting sequences of 'Low Motion' frames within the
'Non-Unique' sequence. On each detected 'Low Motion' frame, the Anchor frame for the next frame (whether it be another
'Low Motion' frame or a new 'Dupe' frame) will be switched to that newly detected 'Low Motion' frame. With this method,
all frames BETWEEN (but excluding) the 'Unique Start' and 'Unique End' in the 'Non-Unique' sequence are detected as perhaps
multiple 'Low Motion' and 'Duplicate' sub sequences.
Setting LagMax arg to some low number (min 1), will limit distance of the Anchor frame prior to the current frame, eg setting
LagMax=1, would always do LoMo/Dupe compare with the current frame and its predecessor. A LagMax greater than 1 allows better detection
of Low-Motion frames where there is only very slow creeping movement in the sequence.

The above method allows for reasonably fast processing, where a 'unique' frame is encountered [where it is clearly different from
its predecessor] but where not so clear, will examine the LoMo/dupe suspect frames more closely. ThG sets the boundary between
'Unique' frames and non unique, and so has some effect on speed, setting too high for your source will slow processing speed as many
non dupes will be examined more closely, and setting ThG too low for your source, may increase speed but may miss duplicates in noisy
sources (probably better to set too high than too low).

Although you might be able to scan backwards and get same detections, results are written to the Frames file in visited sequence and
you should NOT jump about whilst writing the final frames file. Also, this is a Multi-Instance script, can use multiple instances
at the same time (view side by side metrics) but you should really choose a unique 'Frames' filename for each instance.

Args:-
Mode, Default 0, (Range 0->2)
0) Write Frames file only (where Frames file="" then not written so only Show=true does anything).
1) Make duplicate sequences Exact Dupes.
2) Interpolate/Blend Duplicates (depends upon MaxDupLen and MaxInterp).

ThG, Default 1.0 (0.0 < ThG <= 255.0) Suggest 1.0, never below 0.5.
Primary Global Unique detect threshold. [current_frame-1 <-> current_frame].
If using 255.0 will likely scan all of the way to both Start and End frames, and then
detect LoMo/Dupe frames of entire clip using the 2nd & 3rd detection methods, will seem to pause for a long time,
dont do it (no error if using daft numbers).

ThB, Default Float ThG * 2.0, (Range ThG <= ThB < 255.0, Suggested about 1.5->3.5).
Secondary Local LoMo/Dupe Block detect threshold, [Anchor<->current_frame]
Tune for source and Block Size. (For smaller block size, just by random chance there may be several noisy pixels
in duplicate frame block, that produce considerably higher metric than most other blocks).

ThP, Default Int 64. (Advised Range ThB < ThP <= 64, or 255=OFF)
Ternary Local [single pixel] LoMo/Dupe detect threshold, [Anchor<->current_frame].
On a test with 256 artificial generated 640x480 identical frames of random coloured pixels, encoded at
MeGUI AVC CRF 23.0(default), RT_LumaPixelsDifferent(Thresh=1) found no frames where any single pixel was
different by more than 1 luma level when compared with the first encoded frame from that sequence. Mpeg2
and flv might not fare as well. This test was on a totally clean artificial source without artefacts
and such, you will likely need considerably higher values of ThP than 1.
I have seen multiple clips exhibiting some kind of minute (perhaps sub pixel) frame shift (both horizontal and
vertical) where frame should have been an exact duplicate, no idea to the cause.
This setting is an additional knob to twiddle, and you could turn it off and rely on ThG and ThB
completely [ThP=255, turns off]. The default of 64 and the below setting IgP=0.0, will consider any single pixel
with a luma difference of 64 or more, to be a non dupe, unfortunately any clip having above mentioned sub pixel
shifting at dupe frames, may need a high setting to not trigger a lomo detection, I have not I think
seen so far any clip that needed ThP more than about 48. 64 might seem ridiculously high but if frames are shifted,
then a little preposterous-ness is likely necessary, in a true non-dupe, there will most usually be at least
one pixel that differs in luma by more than 64.
We have thusly decided to make the default an emergency fallback detect value of 64, the user may decide to lower
this value or indeed turn it off completely[ThP=255].

IgP, Default 0.0, (Range 0.0 <= IgP < 100.0). Percent noise threshold for Ternary Local [pixel] LoMo/Dupe Detect,
Note, default 0.0% means that any frame with any single pixel with abs luma difference greater than ThP will
be detected as LoMotion.

ThK, Default = 7, (0 <= ThK <= 255). 0 = K Overrides off.
If RT_YPlaneMinMaxDifference(Threshold=0.2) of current_frame is less ThK then will Override any detection,
ie black (or narrow luma range) frames are ignored as not duplicates.
[Threshold=0.2 ignores as noise, 1 in 500 extreme pixels when establishing loose minimum and loose maximum & luma range].


Duplicity2 text shortened due to 16KB limit in Usage forum.

[StainlessS]

Suggest RT_LumaPixelsDifferent(Thresh=64) # Detect big single pixel difference.
Would return 0.0 -> 255.0, where 255.0 = 100%, of pixels with absolute difference greater than Thresh.

Also note,

RT_LumaPixelsDifferentCount(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int"y2"=y,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns number of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Thresh Default 0, returns number of pixels that are not exactly the same.
Note, returns the number of pixels whose difference is greater than Thresh, so ranges 0 -> (Width*Height).
v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

Produces int COUNT of absolute pixel differences greater than Thresh, rather than psuedo percent (0.0->255.0).

EDIT:
Sorry for confusing things (I was confused :) )
But Duplicity2 arg ThP, is used for RT_LumaPixelsDifferent(Thresh=ThP).

EDIT:
Maybe something like [the differencing calls require additional args]

ThG = 1.0 # (0.0 < ThG <= 255.0)
ThB = 2.0 * ThG # (Range ThG < ThB < 255.0, Suggested about 1.5->3.5)
ThP = 64 # (Advised Range ThB < ThP <= 64)

IsDIf = (RT_FrameDifference() > ThG || RT_FrameMovement(BlkTh=ThB) > ThB || RT_LumaPixelsDifferent(Thresh=ThP) > 0.0)

If(isDif) {
# ...
} else {
# ...
}

Note, Avisynth || in IsDif line will only process differencing calls until the first that passes, ie if RT_FrameDifference() > ThG then neither RT_FrameMovement nor RT_LumaPixelsDifferent will be tested. [ie affects SPEED]

[rgr]

OK, thanks for the explanation. I see the solution, but it's a pity that RT_FrameMovement doesn't have a "threshold" parameter.

[StainlessS]

Sorry, I told you a load of BullSquirt, RT_FrameMovement(BlkTh=ThB) was rubbish/unnecessary.

RT_FrameMovement(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0,
int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "x2"=x,int "y2"=y,
bool "AltScan"=false,Bool "ChromaI"=False,int "Matrix"=(c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600?3:2),
int "BlkW"=64,int "BlkH"=BlkW, Int "OLapX"=BlkW/2, Int "OLapY"=BlkH/2,Float "BlkTh"=1.0,String "Prefix"="")

Returns FLOAT value (0.0 -> 255.0) movement detection between clip c frame n area x,y,w,h, and clip c2 frame n2 area x2,y2,w,h.
The frames are sectioned into blocks of BlkW x BlkH with block overlaps of OLapX and OLapY, and all blocks are differenced, the result
is the greatest difference for any one block comparison.
Is more sensitive to localized movement. Best control of localized movement sensitivity is by changing blk size, lower is more sensitive.
If Altscan is true, then only every other horizontal scanline in BLKH will be scanned.

Args:-
c, c2, comparison clips, usually the same clip.
n, default current_frame, frame from clip c.
n2, default current_frame, frame from clip c2.
ChromaWeight, default 1.0/3.0. Range 0.0 -> 1.0.
Difference method control when comparing blocks.
Y8, returns same as RT_LumaDifference. ChromaWeight ignored.
YUV,
Weighting applied YUV chroma:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
RGB,
If ChromaWeight > 0.0, then returns same as RT_RGBDifference() [average RGB pixel channel difference].
If ChromaWeight == 0.0, then returns same as RT_LumaDifference() using Matrix arg to convert RGB to YUV-Y Luma.
x,y,w,h, all default 0 (full frame c).
x2,y2. x2 defaults x, y2 defaults y, for clip c2.
Altscan, default false. If true then scan only every other scanline starting at clip c y coord, and clip c2 y2 coord.
ChromaI, default false. If YV12 and ChromaI, then do YV12 interlaced chroma scanning.
Ignored if not YV12. Both YV12 clips must be same, cannot difference one that is progressive and one that has interlaced chroma.
Matrix, default (c.Width>1100||c.Height>600||c2.Width>1100||c2.Height>600)?3:2
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709.
Default = (c.Width > 1100 OR c.Height>600 OR c2.Width > 1100 OR c2.Height>600) then 3(PC709) else 2(PC601). YUV not used
Matrix only Used if RGB and ChromaWeight == 0.0, where returns same as RT_LumaDifference().
BlkW, default 64, Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
BlkH, default BlkW, Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
OlapX, default BlkW/2, Horizontal block overlap. 0 -> BlkW/2
OlapY, default BlkH/2, Vertical block overlap. 0 -> BlkH/2
BlkTh, default 1.0. Governs extra information returned when Prefix != "", as local variables.
Prefix, default "" is do not set additional local variables.
If set eg "FM_", then sets local variables as below.
"FM_Xoff" = X Coord of block in Clip c that was greatest.
"FM_Yoff" = Y Coord of block in Clip c that was greatest.
"FM_PercAboveTh" = Percentage of Blocks that were above BlkTh.
"FM_BlkAveDf" = Average Block difference.
"FM_nAboveTh" = Number of Blocks that were above BlkTh.
"FM_TotalBlks" = Total Number of Blocks.

NOTE, 'x2' and 'y2' default to 'x' and 'y' respectively.
c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

RT_FrameMovement returns FLOAT value (0.0 -> 255.0) for block with greatest difference and is again SAD / (BlkW*BlkH) # or SAD / (BlkW*(BlkH/2)) if AltScan=true.
It is the greatest difference for any one block comparison.
BlkTh, default 1.0. Governs only extra information returned when Prefix != "", as Global variables.
So for BlkTh to have any function, Need to set eg Prefix="FM_" (where will set extra Global variables, where var name starts with Prefix, in example case "FM_").
Changes these "FM_" Global variiables.
"FM_PercAboveTh" = Percentage of Blocks that were above BlkTh.
"FM_nAboveTh" = Number of Blocks that were above BlkTh.

So modified script with no BlkTh set for RT_FrameMovement() # Previous script with RT_FrameMovement(BlkTh=ThB) did not really do anything different.

ThG = 1.0
ThB = ThG * 2.0 # (0.0 < ThG <= 255.0)
ThP = 64 # (Advised Range ThB < ThP <= 64) # EDIT: Could try 48, this is really an emergency detect for massive single pixel difference where RT_FrameDifference and RT_FrameMovement both fail detect.

IsDIf = (RT_FrameDifference() > ThG || RT_FrameMovement() > ThB || RT_LumaPixelsDifferent(Thresh=ThP) > 0.0)

If(isDif) {
# ...
} else {
# ...
}


EDIT:
but it's a pity that RT_FrameMovement doesn't have a "threshold" parameter.
Threshold for what ?
We really wanna detect any single block being greater than ThB. [and for RT_LumaPixelsDifferent(Thresh=ThP) any single pixel greater than ThP, return value is pseudo percent 0.0->255.0 of pixels]
Below are available if setting Prefix="FM_" and BlkTh
"FM_PercAboveTh" = Percentage of Blocks that were above BlkTh. [EDIT: I think this is Pseudo Percent, range 0.0 -> 255.0, use FM_PercAboveTh*100.0/255.0 for actual %]
"FM_nAboveTh" = Number of Blocks that were above BlkTh.
and others not dependant upon BlkTh
"FM_BlkAveDf" = Average Block difference.
"FM_TotalBlks" = Total Number of Blocks. [always same, depends upon BlkW, BlkH, OLapX, OLapY]
"FM_Xoff" = X Coord of block in Clip c that was greatest.
"FM_Yoff" = Y Coord of block in Clip c that was greatest.

[rgr]

(about RT_FrameMovement)


Threshold for what ?
Threshold to ignore small pixel difference. For example:
ThP (default 0) = ignore pixel difference smaller than ThP.
So with ThP=2 block with luma difference:
1 -1
-1 1
RT_FrameMov. will return 0.
And with block:
0 0
0 4
will return 4 (converted to average value, i.e. 1).

[StainlessS]

Threshold to ignore small pixel difference. For example:
ThP (default 0) = ignore pixel difference smaller than ThP.

No, RT_LumaPixelsDifferent(Thresh=ThP) returns Pseudo Percent of single pixels with pixel diff GREATER than ThP. So with default ThP=0.0, returns perc of pixels not EXACTLY the same.
RT_LumaPixelsDifferentCount(Thresh=ThP) returns COUNT of single pixels with pixel diff GREATER than ThP. So with default ThP=0.0, returns number of pixels that are not EXACTLY the same.
RT_LumaPixelsDifferent(Thresh) and RT_LumaPixelsDifferentCount(Thresh) allow for ThP=0 to compare for exactly same, and ThP=255 = switch OFF detection, dont do anything ie return 0 without scan.

So with ThP=2 block with luma difference:
1 -1
-1 1
RT_FrameMov. will return 0.
RT_FrameMov returns average absolute difference of block with greatest average absolute difference, so can simply compare result with your own threshold in script,
additional vars based on some block thresh can be gotten from eg FM_PercAboveTh and FM_nAboveTh as previously described. (envisaged as being only of use in metrics).

EDIT: So maybe something like this where showing/not showing metrics. [NOT TESTED in any way]

ThG = 1.0
ThB = ThG * 2.0 # (0.0 < ThG <= 255.0)
ThP = 64 # (Advised Range ThB < ThP <= 64) # EDIT: Could try 48, this is really an emergency detect for massive single pixel difference where RT_FrameDifference and RT_FrameMovement both fail detect.
Prefix="FM_"
Metrics=TRUE

If (METRICS) { # Short-circuit logic not possible, need all for metrics.
dfG = RT_FrameDifference()
dfB = RT_FrameMovement(Prefix="FM_",BlkTh=ThB)
dfP = RT_LumaPixelsDifferent(Thresh=ThP)
IsDIf = (dfG > ThG || dfB > ThB || dfP > 0.0)
RT_Subtitle("%d] %s\ndfG=%f\ndfB=%f : BlkAveDf=%f : nAboveTh=%f\ndfP=%f\n",n,IsDif,dfG,dfB,FM_BlkAveDf,FM_nAboveTh,dfP)
} Else { # Below short-circuit LOGIC, stops at first test that passes as TRUE
IsDIf = (RT_FrameDifference() > ThG || RT_FrameMovement() > ThB || RT_LumaPixelsDifferent(Thresh=ThP) > 0.0)
}
If(isDif) {
# ...
} else {
# ...
}


EDIT:
Threshold to ignore small pixel difference. For example:
ThP (default 0) = ignore pixel difference smaller than ThP.
OK, I think that you are maybe relating to eg
YPlaneMin(clip [, float threshold = 0, int offset = 0]) # http://avisynth.nl/index.php/Internal_functions#Difference
where YPlaneMin(threshold) can ignore a few extreme pixels to get "loose minimum",
I dont think that functionality is really of use here, we are trying our best to not ignore anything, and to detect where different,
might be suitable (and can do it with) RT_LumaPixelsDifferent(Thresh=ThP) or RT_LumaPixelsDifferentCount(Thresh=ThP), but not
good idea on block based measurements where we are only interested in the single block with greatest average difference.

EDIT:
This from Duplicity2 docs,

ThP, Default Int 64. (Advised Range ThB < ThP <= 64, or 255=OFF)
Ternary Local [single pixel] LoMo/Dupe detect threshold, [Anchor<->current_frame].
On a test with 256 artificial generated 640x480 identical frames of random coloured pixels, encoded at
MeGUI AVC CRF 23.0(default), RT_LumaPixelsDifferent(Thresh=1) found no frames where any single pixel was
different by more than 1 luma level when compared with the first encoded frame from that sequence. Mpeg2
and flv might not fare as well. This test was on a totally clean artificial source without artefacts
and such, you will likely need considerably higher values of ThP than 1.
I have seen multiple clips exhibiting some kind of minute (perhaps sub pixel) frame shift (both horizontal and
vertical) where frame should have been an exact duplicate, no idea to the cause.
This setting is an additional knob to twiddle, and you could turn it off and rely on ThG and ThB
completely [ThP=255, turns off]. The default of 64 and the below setting IgP=0.0, will consider any single pixel
with a luma difference of 64 or more, to be a non dupe, unfortunately any clip having above mentioned sub pixel
shifting at dupe frames, may need a high setting to not trigger a lomo detection, I have not I think
seen so far any clip that needed ThP more than about 48. 64 might seem ridiculously high but if frames are shifted,
then a little preposterous-ness is likely necessary, in a true non-dupe, there will most usually be at least
one pixel that differs in luma by more than 64.
We have thusly decided to make the default an emergency fallback detect value of 64, the user may decide to lower
this value or indeed turn it off completely[ThP=255].

Is the reason for ThP=64 for RT_LumaPixelsDifferent ThP=64 value. Emergency detect.

[rgr]

No, RT_LumaPixelsDifferent(Thresh=ThP) returns Pseudo Percent of single pixels with pixel diff [B]GREATER than ThP.

Yes, but for the whole image, not for a block. If it were just for a block, the accuracy of change detection would be better (and more unambiguous).

RT_FrameMov returns average absolute difference of block with greatest average absolute difference, so can simply compare result with your own threshold in script,
additional vars based on some block thresh can be gotten from eg FM_PercAboveTh and FM_nAboveTh as previously described. (envisaged as being only of use in metrics).

Yes, but it's not exact, because it treats multiple invisible changes in a block as one big change.

For one set of frames it might give:
50.00 5.00 1.00 50.00 -- where 5.00 and 1.00 are dups
and for another it might give:
2.00 0.50 0.50 2.00 -- where 0.50 are dups
(Same problem with RT_LumaPixelsDifferent)
Return variables won't help here.

Generally lacks threshold option for better motion detection with RT_FrameMov. Or block definition with RT_LumaPixelsDifferent. One of the two would improve motion detection.

[StainlessS]

I doubt it, but I hear you.

[rgr]

I doubt it, but I hear you.

I'm testing my clip (48k frames, about 30k duplicates) so I see problems :)
RT_Frame Mov. captures "noise", small pixel changes in area, but it's not clear whether it's a lot of invisible changes or one big one.
RT_LumaPixelsDifferent (with threshold) captures a specific number of changes, but since it's in the whole image, it's not clear whether it's a lot of small changes (but exceeding the threshold) captured throughout the image (i.e. rather noise) or a large change in one area (movement).

[StainlessS]

but it's not clear whether it's a lot of invisible changes or one big one.
Yes, thats always gonna be the case. [Life's a bitch, and then you die.]

The provided default thresholds for RT_FrameDifference(), RT_FrameMovement(), and RT_LumaPixelsDifferent/Count do work very well,
suggest that you do try the provided defaults, but you can lower the ThP pixel thresh (so long as you dont have wierd sub pixel 'shaking video'), but suggest no less than 16, defo no less than 7.

EDIT:
RT_LumaPixelsDifferent (with threshold) captures a specific number of changes, but since it's in the whole image, it's not clear whether it's a lot of small changes (but exceeding the threshold) captured throughout the image (i.e. rather noise) or a large change in one area (movement).
Suggest ALWAYS compare RT_LumaPixelsDifferent/Count(Thresh=ThP) > 0, ie ANY single pixel diff greater than ThP is a detect, the actual number does not matter, any one of them is a detect.

EDIT: I think I did have an arg something like IgP (pixel ignore threshold) in Duplicity2(), it proved less than useless and I dont ever use it, I always compare RT_LumaPixelsDifferent(ThP) > 0. [IgP probably be removed]
EDIT: The result of RT_LumaPixelsDifferent(ThP) or RT_LumaPixelsDifferentCount(ThP) could be shown in metrics and help to select good ThP, but the actual detect should only be if > 0.
Same with the RT_FrameMovement(Prefix="FM_") style extra global vars, can show in metrics where can asisist in choosing good thresholds for a particular source. (see metrics in following post image, eg nAboveTh).

EDIT:
I'm testing my clip (48k frames, about 30k duplicates) so I see problems
Sounds like you might like to take a peek at SecurityCamExtractMotion.AVS :- From RT_Stats AVS directory
It works very well, and even can lock on to 'fluctuating movement'.

# HOLD frames are combination of all KEEP frames and, DROP frames where block with greatest difference is in close proximity to previous
# KEEP frame block with greatest difference (ie there seems to be possible movement very near previously identified movement).


The script is in the RT_Stats AVS directory, but I've never posted on D9 (I think), so I'll post in 2 parts following this post.

[StainlessS]

Find SecurityCamExtractMotion.AVS v2.02 :- https://www.mediafire.com/file/c6kqosweqmk82my/SecurityCamExtractMotion.AVS/file

Looks like this
https://s20.postimg.cc/5ix37af7h/SCMD2_zpsiuzq505m.png (https://postimg.cc/image/uc6n7xy7t/)

Or,

SecurityCamExtractMotion.AVS [Part 1 of 2]


AVISource("DayCAP2.avi")
Trim(550,31142)
crop(460,200,0,0) # Chop off waving bushes

############################################
# SecurityCamExtractMotion.AVS, v2.02, By StainlessS.
#
# Requires RT_Stats v2.0, FrameSel v1.03, GScript, Grunt, mt_tools_2. [GScript not needed if Avs+]
#
# Clip, Planar, YUY2 or RGB clip, YV411 not supported due to non support in Overlay.
# (RGB, will likely require different thresholds)
#
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN MotDetect.
#
# MotDetect() creates four .txt command files for KEEP frames (movement), DROP frames (not movement), DUPE frames and HOLD frames.
# HOLD frames are combination of all KEEP frames and, DROP frames where block with greatest difference is in close proximity to previous
# KEEP frame block with greatest difference (ie there seems to be possible movement very near previously identified movement).
# Using SelectFrames() you can select frames types of each of the 4 command files, or using the Reject mode of FrameSel, you can
# instead return the frames NOT in the command file, so using the 4 created command files, you can extract 8 variations of frame types
# after a single detection scan.
#
############################################
Th = 6.0 # Greater or equal KEEP (movement), below is DROP(or DUPE, or possibly HOLD).
# Need check against source, block size and almost all other settings.
#
dTH = 4.0 # Below or equal is DUPE, 0.0 <= dTh < Th (above dTh and below Th is DROP [or possibly HOLD]).
#
BLKW = 64 # Suggest about 16 -> 64 (even only, mod 4 else probably slower). Metrics will change with block size.
BLKH = BLKW # Localized movement, adjust for smallest size of moving objects to detect, makes only little difference to speed,
# bigger, may be a bit faster.
# For human target, suggest box encompassing head + shoulders of most distant human.
# RT_FrameMovement returns greatest average pixel difference for any one of the blocks.
# Block searches may be overlapped by up to half a block both horizontally and vertically and so total x,y,w,h area
# of each pair of frames is compared up to about 4 times due to the half block overlaps.

#
X = 16 # Area Of Interest, avoid eg passing traffic or timecode (or could use eg Overlay to blot out stuff for dclip).
Y = 16 # Suggest cut out eg sky unless looking for flying saucers (less to search, faster).
W = -16 # As for Crop. Best MOD 4 only
H = -16
#
CW = 0.0 # ChromaWeight, 0.0->1.0. 0.0 = Luma ONLY, about 0.1 suggested for color (luma ONLY faster for YUV).
# For RGB, Matrix used where CW==0.0 to convert to Luma-Y. If CW != 0.0, then returns average pixel differences,
# (apdR+apdG+apdB)/3.0.
# Perhaps always CW=0.0 [luma only] for YUV night captures.
# Can convert RGB to YUV for Luma-Y detect and SelectFrames() using original RGB, if required.
#
AltScan=false # If True, Scan only every other raster line starting from Y (maybe faster, especially YUV Luma scan only).
#
ChromaI=False # If True, YV12 chroma is interlaced (ignored if not YV12 or if Luma ONLY scan [CW==0.0]).
#
Matrix=(Width>1100||Height>600)?3:2 # Matrix used for RGB conversion to Luma-Y.
# Default, If (Width>1100 || Height>600) then 3(PC.709), else 2(PC.601). (0=Rec.601, 1=Rec.709)
# NOTE, default PClevels conversions will likely require different thresholds to YUV because of the
# greater differences in Y (after RGB-Luma-Y conversion), probably about (255.0/219.0) times greater.
#
OLapX= BlkW/2 # X Block Overlap. 0->BLKW/2 (bigger more accurate, slower)
OLapY= BlkH/2 # Y Block Overlap. 0->BLKH/2
# Where YUV, CW==0.0 and AltScan==True, might try eg OLapY=(BlkH/2)-1, where would be faster due to AltScan and,
# also in overlapped 'Y_Block' scan will then scan alternate scanlines that were not scanned in previous non overlapped
# scan (if that makes any sense, ie even scanlines sampled in first block, odd lines sampled in overlapped block scan).

LAG = 1 # Lag (default 1, ie no lag. 1 or more). Compares current frame n with frame n-Lag. Detection is always
# initially compared(n,n-1), if neither DUPE nor KEEP detected and Lag>1, then will redetect using comparison(n,n-Lag)
# where n-lag frame is limited to no earlier than the most recent KEEP frame.
# A moderate Lag (up to about 50 or so), may be more sensitive to very small amount of movement,
# it introduces a time delay between compared frames. There is no upper limit to Lag arg.
# NOTE, Lag > 1 will likely produce more KEEP and HOLD frames (ie identify more movement), and also a bit slower
# due to 2nd detection scan where initial scan identified DROP frame (ie DROP may be converted to KEEP or HOLD).
# Lag will not make any huge difference to results, small amount of movement at beginning of sequence of movements may be
# detected, use lag > 1 only where every possible movement frame is required.

SHOWDCLIP=false # If True, then shows DClip in MotDetect rather than source clip (Only useful if dclip pre-processed in some fashion)
#
dclip=Last # Detection clip.
# Or some kind of denoise or whatever, used for detection (visible in ShowMetrics if SHOWDCLIp==True).
# Use SPATIAL denoise ONLY.
#dclip=dclip.MaxContrast(Samples=64,Gamma=1.0).VagueDenoiser(threshold=0.8,method=1,nsteps=6,chromaT=0,interlaced=false) # SLOWISH
#dclip=dclip.MaxContrast(Samples=64,Gamma=1.0).hqdn3d(2.0, 0.0,0.0,0.0,0) # A bit FASTER

###
### Show Metrics ONLY
DIM=False # Dim/darken area outside of the x,y,w,h search area. A little faster with Dim==False when showing metrics only.
SHOWMISS =true # Show block 'Marker' for greatest moving DROP block (miss), (markers never shown for dupes).
MISSCOLOR =$AAFF00 # Color, block 'Marker' for greatest DROP block (when SHOWMISS==True).
HITCOLOR =$FFFF44 # Color, block 'Marker' for greatest KEEP block (hit).
HOLDCOLOR =$FF8800 # Color, block 'Marker' for HOLD block (Miss converted to HIT due to Miss proximity to previous Hit).
VERBOSE=True # Show additional metrics, Th, dTh, BLK Args
###
### Frame Extraction ONLY
SHOW =True # Show FrameSel() Info on frame when extracting frames (ie shows '66] Using 12345').
SHOWTIME=True # Show original clip Time on Frame.
#
# # Name of command frames file returned By AutoGetFrames and SelectFrames. (Keep.Txt, Drop.txt, Dupe.txt, Hold.txt)
#ReturnFN="Keep.Txt"
ReturnFN="Hold.Txt"
#ReturnFN="Drop.Txt"
#ReturnFN="Dupe.Txt"
#
REJECT = False # If True, then return frames NOT in RetunFN rather than frames in ReturnFN.
###

##########################################
# Choose 1 of below options:- (uncomment)
##########################################

# Show metrics (Multi-Instance capable) to get best Threshold 'th' (will create valid frames files if playing straight through).
MotDetect(Last,Th,dTH,BLKW,BLKH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OLapX,OLapY,LAG,dclip,Dim,SHOWMISS,MISSCOLOR,HITCOLOR,HOLDCOLOR,SHOWDCLIP,VERBOSE)

# Make Frames Cmd files for SelectFrames(), Must Play all way through (no metrics shown ie faster).
#MotDetect(Last,Th,dTH,BLKW,BLKH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OLapX,OLapY,LAG,dclip,Hide=True)

# Make Frames command files and then extract ReturnFN frames (Keep.txt default) automatically in 1 pass (Auto scans entire file before display).
# Fastest possible scanning, faster than eg VDub video analysis pass as no video shown at all during scan, Speed FPS result shown via DebugView.
#AutoGetFrames(Last,Th,dTh,BlkW,BLKH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OlapX,OlapY,Lag,dclip,show=SHOW,showTime=SHOWTIME,ReturnFN=ReturnFN,Reject=REJECT)

# Return Frames using EXISTING ReturnFN frames Command file (after making ReturnFN using one of above options)
#SelectFrames(Last,ReturnFN,show=SHOW,showTime=SHOWTIME,reject=REJECT)

#AssumeFPS(250.0)
return Last
##########################################
##########################################
##########################################

Function MotDetect(clip c,float th,Float "dth", int "BlkW",int "BlkH",int "X",int "Y",int "W",int "H",
\ Float "CW",Bool "AltScan",Bool "ChromaI",int "Matrix",int "OLapX",int "OLapY",Int "Lag",
\ clip "dclip",bool "dim",Bool "ShowMiss",int "MissColor",int "HitColor",int "HoldCOLOR",bool "ShowDClip",bool "Verbose",
\ String "KeepFN",String "DropFN",String "DupeFN",String "HoldFN",
\ bool "Hide") {
myName="MotDetect: "
IsAvsPlus=(FindStr(UCase(versionString),"AVISYNTH+")!=0) HasGScript=RT_FunctionExist("GScript")
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(IsAvsPlus||HasGScript,myName+"Essential GScript installed, http://forum.doom9.org/showthread.php?t=147846")
Assert(RT_FunctionExist("RT_FrameMovement"),myName+"Essential RT_Stats v2.0 installed, http://forum.doom9.org/showthread.php?t=165479")
dth=Float(Default(dth,0.001))
BlkW=Default(BlkW,64) BlkH=Default(BlkH,BlkW)
X=default(X,0) Y=default(Y,0) W=default(W,0) H=default(H,0)
CW=Default(CW,1.0/3.0) AltScan=Default(AltScan,false) CI=Default(ChromaI,false)
Lag=Default(Lag,1)
dclip=Default(dclip,c)
ShowMiss=Default(ShowMiss,true) MissColor=Default(MissColor,$AAFF00)
HitColor=Default(HitColor,$FFFF44) HoldColor=Default(HoldColor,$FF8800)
ShowDClip=Default(ShowDClip,false)
Verbose=Default(Verbose,true)
c=(ShowDClip)?dclip:c
Hide=Default(Hide,false)
Dim=(!Hide)?Default(dim,true):False
KeepFN=Default(KeepFN,"Keep.Txt") DropFN=Default(DropFN,"Drop.Txt")
DupeFN=Default(DupeFN,"Dupe.Txt") HoldFN=Default(HoldFN,"Hold.Txt")

c
Matrix=Default(Matrix,Width>1100||Height>600?3:2) OLapX=Default(OLapX,BlkW/2) OLapY=Default(OLapY,BlkH/2)
Assert(CW>=0 && CW<=1.0,RT_String("%s0.0 <= CW <= 1.0",myName))
Assert(Width==dclip.Width&&Height==dclip.Height&&FrameCount==dclip.FrameCount, RT_String("%sDissimilar detection clip",myName))
Assert(dTh>=0.0 && dTh<Th && Th<=255.0,RT_String("%s0.0 <= dTh < Th <= 255.0",myName))
Assert(Lag>=1, RT_String("%sLag Must be 1 or more",myName))
W = (W > 0) ? (W + X) - Width : W H = (H > 0) ? (H + Y) - Height : H # Make -ve relative width,height
X = (X+3) / 4 * 4 Y = (Y+3) / 4 * 4 # Round UP (inwards) Mod 4
W = (W-3) / 4 * 4 H = (H-3) / 4 * 4 # Round DOWN(inwards) Mod 4
W = Width - X + W H = Height - Y + H # +ve width,height
Cropped=Crop(X,Y,W,H) BlkW=Min(Cropped.Width,BlkW) BlkH=Min(Cropped.Height,BlkH)
(Dim && (W!=Width || H!=Height)) ? levels(0,1.0,255,0,128).Overlay(Cropped,x=X,y=Y,opacity=1.0) : Last
(Exist(KeepFN)) ? RT_FileDelete(KeepFN) : NOP
(Exist(DropFN)) ? RT_FileDelete(DropFN) : NOP
(Exist(DupeFN)) ? RT_FileDelete(DupeFN) : NOP
(Exist(HoldFN)) ? RT_FileDelete(HoldFN) : NOP
MissMsk = Hit_Marker(BlkW,BlkH,false)
HitMsk = Hit_Marker(BlkW,BlkH,true)
HoldMsk = Hit_Marker(BlkW,BlkH,true)

MissMrk = MissMsk.BlankClip(Color=MissColor,pixel_type="YV12")
HitMrk = HitMsk.BlankClip(Color=HitColor,pixel_type="YV12")
HoldMrk = HoldMsk.BlankClip(Color=HoldColor,pixel_type="YV12")

OutCS=IsRGB24?"RGB24":IsRGB32?"RGB32":IsYUY2?"YUY2":IsYV12?"YV12":VersionNumber<2.6?"UNK"
\ :IsY8?"Y8":IsYV16?"YV16":IsYV24?"YV24":IsYV411?"YV411":"UNK" # YV411 not actually supported by Overlay output.
Fmt=
\ "%d ] %s{%c} BlkMaxDif=%7.3f\n" +
\ "LAG=%-13d : CompFrm=%d\n" +
\ "KEEP=%-6d %5.2f%% : MinKeep=%.3f\n" +
\ "DROP=%-6d %5.2f%% : MaxDrop=%.3f\n" +
\ "DUPE=%-6d %5.2f%% : MaxDupe=%.3f\n" +
\ "HOLD=%-6d %5.2f%% : LastKFrm=%d"
vFmt="nBlks=%d : BlkAveDif=%.3f\n" +
\ "nAboveTh=%2d (%.2f%%)\n" +
\ "Th=%.2f DTh=%.3f : CW=%.3f\nAlt=%c : Blk=%dx%d : OLap=%dx%d"

[StainlessS]

SecurityCamExtractMotion.AVS [Part 2 of 2 - Glue them together]



FuncS="""
Function Fn@@@(clip clp,Float th,Float dth,int BlkW,int BlkH,int x,int y,int w,int h,
\ Float CW,bool AltScan,bool CI,int Matrix,int OLapX,int OLapY,Int Lag,
\ clip dclip,bool ShowMiss,clip MissMrk,clip HitMrk,clip HoldMrk,clip MissMsk,clip HitMsk,clip HoldMsk,bool Verbose,
\ String Fmt,String vFmt,String Prefix,string OutCS,
\ String KeepFN,String DropFN,String DupeFN,String HoldFN,
\ bool Hide) {
clp
n=current_frame
if(PrevN@@@ != n) {
# Always check against n-1 for DROP and KEEP
cmp = n-1
if(PrevN@@@!=cmp) { # Reset (jump about or frame 0)
cmp = max(n-1,0)
Global RecentKeep@@@=cmp # Most recent KEEP frame number
Global PrevMat@@@=1 # Prev vist match type DROP
Global CurrXOff@@@=-6666 # Invalid Coords of current KEEP or HOLD
Global HoldXOff@@@=-6666 # Invalid Coords of Most recent KEEP or HOLD
}
dif=RT_FrameMovement(dclip,dclip,n,cmp,CW,x,y,w,h,x,y,altscan,CI,Matrix,BlkW,BlkH,OLapX,OLApY,Th,Prefix)
mat=(dif>dth) ? (dif<th?1:2) : (n==0) ? 2 : 0 # 0=DUPE : 1=DROP : 2=KEEP : 3=HOLD(Not used here)

if(Lag>1 && mat==1 && cmp-RecentKeep@@@>=1) {
# If DROP will be converted to HOLD, then dont compare LAG frame
if(MOTD_XOff<HoldXOff@@@-BlkW || MOTD_XOff>HoldXOff@@@+BlkW || MOTD_YOff<HoldYOff@@@-BlkH || MOTD_YOff>HoldYOff@@@+BlkH) {
# Limit RecentKeep@@@ usage to inital lag,
# avoid small remnants of objects moving out of frame from being used in later compare
cmp = (n-RecentKeep@@@==2) ? RecentKeep@@@ : max(RecentKeep@@@+1,n-Lag)
dif=RT_FrameMovement(dclip,dclip,n,cmp,CW,x,y,w,h,x,y,altscan,CI,Matrix,BlkW,BlkH,OLapX,OLApY,Th,Prefix)
mat=(dif>dth) ? (dif<th?1:2) : 0
}
}

if(mat==0) { # Dupe
(DupeFN!="") ? RT_WriteFile(DupeFN,"%d",n,append=true) : NOP
} else if(mat==1) { # Drop
if(HoldXOff@@@ >= 0) { # HOLD Disabled ? (will quite often be disabled)
if( MOTD_XOff>=HoldXOff@@@-BlkW && MOTD_XOff<=HoldXOff@@@+BlkW &&
\ MOTD_YOff>=HoldYOff@@@-BlkH && MOTD_YOff<=HoldYOff@@@+BlkH) {
mat=3 # Convert to HOLD
(HoldFN!="") ? RT_WriteFile(HoldFN,"%d",n,append=true) : NOP
} else { # Wandered too far from most recent hold coords, Disable HOLD locking
Global HoldXOff@@@ = -6666
}
}
(DropFN!="") ? RT_WriteFile(DropFN,"%d",n,append=true) : NOP
Global PrevMat@@@ = 1
} else { # Keep
if(n!=0 || PrevN@@@==-6666) { # Block nasty VDub Analysis pass writing 0 on re-seek to 0 at end
(KeepFN!="") ? RT_WriteFile(KeepFN,"%d",n,append=true) : NOP
(HoldFN!="") ? RT_WriteFile(HoldFN,"%d",n,append=true) : NOP
}
Global PrevMat@@@ = 2
Global RecentKeep@@@ = n
Global HoldXOff@@@ = MOTD_XOff # From Locals set by RT_FrameMovement()
Global HoldYOff@@@ = MOTD_YOff
}

if(!Hide) {
Global CmpFrm@@@ = cmp
Global Mat@@@ = mat
Global Dif@@@ = dif
Global CurrXOff@@@ = MOTD_XOff
Global CurrYOff@@@ = MOTD_YOff
if(mat==0) {
Global nDupe@@@ = nDupe@@@ + 1
Global DupeMax@@@ = max(dif,DupeMax@@@)
} else if(mat==1) {
Global nDrop@@@ = nDrop@@@ + 1
Global KeepMin@@@ = 255.0
Global DropMax@@@ = max(dif,DropMax@@@)
} else if(mat==2) {
Global nKeep@@@ = nKeep@@@ + 1
Global KeepMin@@@ = min(dif,KeepMin@@@)
Global DropMax@@@ = 0.0
Global nHold@@@ = nHold@@@ + 1
} else {
Global nHold@@@ = nHold@@@ + 1
}
if(VerBose) {
Global TotalBlks@@@ = MOTD_TotalBlks
Global BlkAveDf@@@ = MOTD_BlkAveDf
Global nAboveTh@@@ = MOTD_nAboveTh
Global PercAboveTh@@@ = MOTD_PercAboveTh
}
}
}

if(!Hide) {
mat = Mat@@@
z=RT_Ord("-123",mat+1)
RT_SubTitle(Fmt,n,(mat==0)?"DUPE":(mat==1)?"DROP":(mat==2)?"KEEP":"HOLD",
\ z,Dif@@@,max(n-CmpFrm@@@,0),CmpFrm@@@,
\ nKeep@@@, nKeep@@@*100.0 / (n+1.0),KeepMin@@@,
\ nDrop@@@, nDrop@@@*100.0 / (n+1.0),DropMax@@@,
\ nDupe@@@, nDupe@@@*100.0 / (n+1.0),DupeMax@@@,
\ nHold@@@, nHold@@@*100.0 / (n+1.0),RecentKeep@@@
\ )
Verbose ? RT_Subtitle(vFmt,TotalBlks@@@,BlkAveDf@@@,nAboveTh@@@,PercAboveTh@@@,th,dth,CW,(altScan)?84:70,BlkW,BlkH,OLapX,OLapY,Align=1) : NOP
if((mat>0 && (mat!=1||ShowMiss)) && n>0) {
Overlay(mat==1?MissMrk:mat==2?HitMrk:HoldMrk,x=x+CurrXOff@@@,y=y+CurrYOff@@@,
\ Mask=mat==1?MissMsk:mat==2?HitMsk:HoldMsk,Mode="Blend",output=OutCS)
}
}
Global PrevN@@@ = n
return Last
}
#######################################
# Unique Global Variables Initialization
#######################################
Global PrevN@@@=-6666 # Previously visited Frame Number
Global RecentKeep@@@=0 # Most recent KEEP frame number
Global PrevMat@@@=1 # Prev vist match type (1,2,only, not DUPE)
Global HoldXOff@@@=-6666 # Coords of Most recent KEEP or HOLD
Global HoldYOff@@@=-6666
# Required for Metrics ONLY
Global CmpFrm@@@=0 # Current comparison frame for decision
Global Mat@@@=0 # Current visit Match type
Global Dif@@@=0.0 # Current visit dif
Global TotalBlks@@@=0
Global BlkAveDf@@@=0.0
Global nAboveTh@@@=0
Global PercAboveTh@@@=0.0
Global CurrXOff@@@=-6666 # Current Coords
Global CurrYOff@@@=-6666
Global nKeep@@@=0 Global nDrop@@@=0 Global nDupe@@@=0 Global nHold@@@=0 # Counts
Global KeepMin@@@=0.0 Global DropMax@@@=0.0 Global DupeMax@@@=0.0 # Dif limits
Prefix="MOTD_"
#######################################
# Unique Runtime Call, GScriptClip must be a one-liner:
#######################################
ARGS = "th,dth,BlkW,BlkH,x,y,w,h,CW,AltScan,CI,Matrix,OLapX,OLapY,Lag," +
\ "dclip,ShowMiss,MissMrk,HitMrk,HoldMrk,MissMsk,HitMsk,HoldMsk,Verbose,Fmt,vFmt,Prefix,OutCS," +
\ "KeepFN,DropFN,DupeFN,HoldFN,Hide"
Last.GScriptClip("Fn@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
#######################################
# Unique Identifier Definition
#######################################
GIFunc ="MOTD" # Function Name, Supply unique name for your multi-instance function.
GIName =GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS) # Resulting Instance script, for debug perusal.
IsAvsPlus?Eval(InstS):HasGScript?GScript(InstS):Assert(False,RT_String("%sNeed either GScript or AVS+",myName))
Return KillAudio
}


Function Hit_Marker(int W, Int H, Bool "Hit", Int "BW", Int "BH",Int "PW",Int "PH",Bool "YV12",Bool "Mod2") {
/*
Req RT_Stats & mt_tools v2. http://forum.doom9.org/showthread.php?t=174527
Creates a marker mask WxH, for use with Overlay as Mask arg.
Intended for use via some kind of detector to show detection Hit, or Miss.
Returns Single frame clip with null audio, the marker top left hand side is aligned to 0,0.
Default return clip colorspace is Y8 for Avisynth v2.6 and above, or YV12 if defunct version.
Args:-
W,H, Size of marker mask (can be odd), return clip dimensions are rounded up to next multiple of 2 if YV12 return clip.
Bare minimum usable size is 4x4 with defaulted BW, BH, PW, PH.
HIT, Default False, returns marker as angled corners (Thickness set by BW and BH).
True, returns marker as angled corners + outer perimeter where perimeter thickness set by PW and PH.
BW, Default Max(W/8,1). Horizontal thickness of the marker corners.
BH, Default Max(H/8,1). Vertical thickness of the marker corners.
PW, Default Max(BW/8,1). Horizontal thickness of the vertical perimeter.
PH, Default Max(BH/8,1). Vertical thickness of the horizontal perimeter.
YV12, Default False if Avisynth version 2.6 or greater(Y8), else true(YV12). Selects return clip colorspace.
Mod2, Default True. If Y8 then round odd dimensions up modulo 2 (the mask is still WxH).

Example:-
Import("Hit_Marker.avs")
WW=128 HH=WW HIT=True
BlankClip
Yell=Last.BlankClip(Width=WW,Height=HH,Length=1,Color=$FFFF00)
Return OverLay(Yell,x=(Width-WW)/2,y=(Height-HH)/2,mask=Hit_Marker(WW,HH,HIT))
*/
Hit=Default(hit,False) BW=Max(Default(BW,W/8),1) BH=Max(Default(BH,H/8),1)
BW2=(W>4)?BW*2:BW BH2=(H>4)?BH*2:BH
PW=Max(Default(PW,BW/8),1) PH=Max(Default(PH,BH/8),1) is26 = VersionNumber>=2.6
YV12=(!is26) ? True : Default(YV12,False) Mod2=Default(Mod2,True) Mod = (YV12||Mod2) ? 2 : 1
CanvasW=(W+Mod-1)/Mod*Mod CanvasH=(H+Mod-1)/Mod*Mod
Rpn= RT_String("x %d < x %d >= | y %d < | y %d >= | x %d < x %d >= | & y %d < y %d >= | & ",BW,W-BW,BH,H-BH,BW2,W-BW2,BH2,H-BH2)
Rpn= (Hit) ? Rpn + RT_String("x %d < x %d >= | y %d < | y %d >= | | ",PW,W-PW,PH,H-PH) : Rpn
Rpn= Rpn + RT_String("x %d < y %d < & & 255 0 ?",W,H)
Blankclip(width=CanvasW,height=CanvasH,Length=1,pixel_type=YV12?"YV12":"Y8").Killaudio
return mt_lutspa(relative = false,yExpr=Rpn, chroma = "-128")
}


Function MaxContrast(clip c,int "Samples",Float "Ignore",Float "Gamma",Bool "Show") { # PC Levels Max Contrast
c Gamma=Float(Default(Gamma,1.0)) Show=Default(Show,False)
Eval(RT_QueryLumaMinMax(Samples=Samples,Ignore=Ignore,X=4,Y=4,W=-4,H=-4,Debug=Show))
Doit = (QLMMMin!=0 || QLMMMax != 255)
RT_DebugF("Levels(%3d,%.2f,%3d,%d,%d,Coring=False)",QLMMMin,Gamma,QLMMMax,0,255,name="MaxContrast: ")
(Doit && Show) ? RT_Subtitle("%sLevels(%3d,%.2f,%3d,%d,%d,Coring=False)","MaxContrast: ",QLMMMin,Gamma,QLMMMax,0,255) : NOP
(Doit) ? Levels(QLMMMin,Gamma,QLMMMax,0,255,Coring=False) : NOP
Return Last
}

Function SelectFrames(clip c,String Fn,bool "Show",bool "Showtime",bool "Reject") {
c Show=Default(Show,False) ShowTime=Default(ShowTime,False) Reject=Default(Reject,False)
(ShowTime) ? ShowTime() : NOP FrameSel(CMD=Fn,show=Show,reject=Reject) # Returns NO audio
}

Function AutoGetFrames(clip c,float th,Float "dth", int "BlkW",int "BlkH",int "X",int "Y",int "W",int "H",
\ Float "CW",Bool "AltScan",Bool "ChromaI",int "Matrix",int "OLapX",int "OLapY", Int "Lag",
\ clip "dclip",String "KeepFN",String "DropFN",String "DupeFN",String "HoldFN",
\ bool "Show",Bool "Showtime",String "ReturnFN",Bool "Reject"
\ ) {
Default(dclip,c) # Use detection clip (denoised or whatever) for making frames command files.
KeepFN = Default(KeepFN,"Keep.txt") # Filename for Keep Frames
ReturnFN = Default(ReturnFN,KeepFN) # Use ReturnFN if given else KeepFN
Reject = Default(Reject,False)
START = RT_TimerHP()
# Make the command file for FrameSel()
MotDetect(Last,th,dth,BlkW,BlkH,X,Y,W,H,CW,AltScan,ChromaI,Matrix,OLapX,OLapY,Lag,Last,
\ KeepFN=KeepFN,DropFN=DropFN,DupeFN=DupeFN,HoldFN=HoldFN,Hide=True)
RT_ForceProcess(Debug=True) # Scan entire clip forcing command files to be written.
T = RT_TimerHP() - START
RT_DebugF("AutoGetFrames: %.2fsecs (%.2fmins) %.2fFPS",T,T/60.0,FrameCount/T)
c.SelectFrames(ReturnFN,Show=Show,showtime=Showtime,Reject=Reject) # Extract ReturnFN source frames and return to client
}


EDIT: When using AutoGetFrames(), Progress/Speed FPS shown via DebugView[From Micro$oft]:- https://learn.microsoft.com/en-us/sysinternals/downloads/debugview

[rgr]

RT_LumaPixelsDifferent/Count(Thresh=ThP) > 0, ie ANY single pixel diff greater than ThP is a detect, the actual number does not matter, any one of them is a detect.

As I wrote, the problem is that it applies to the entire image, not the block.

EDIT: The result of RT_LumaPixelsDifferent(ThP) or RT_LumaPixelsDifferentCount(ThP) could be shown in metrics and help to select good ThP, but the actual detect should only be if > 0.

This approach isn't perfect -- I do it comprehensively by comparing the differences in frames next to each other, so I don't have to have a perfect result. But I do have to have a comparable result, not one where duplicates can generate such different results.

Same with the RT_FrameMovement(Prefix="FM_") style extra global vars, can show in metrics where can asisist in choosing good thresholds for a particular source. (see metrics in following post image, eg nAboveTh).

They don't help at all, because they can't. They don't show what contributed to the block measurement result.

[StainlessS]

I guess that its down to you writing your own routine then,
Good luck.

[rgr]

I guess that its down to you writing your own routine then,
Good luck.

Unfortunately, it's not for me anymore :)

[Selur]

Any plans to port RT_Stats to Vapoursynth?

[StainlessS]

Any plans to port RT_Stats to Vapoursynth?
I would not hold your breath, (I dont use Vapoursynth),
and have cold sweats just thinking about making it HBD for avisynth.
(also, dont think I've done any C/CPP coding at all since Covid).

[qyot27]

Due to the issue reported in https://forum.doom9.org/showthread.php?t=186549, I needed to run RT_Stats on Linux. I saw the post from a few years ago that mentioned doing this (https://forum.doom9.org/showthread.php?p=1943103#post1943103), but the repository had been deleted in the time since.

Thankfully, the Internet Archive had preserved it, so I was able to reconstruct the changes (with the Linux patch being its own commit). The result is here:
https://github.com/qyot27/RT_Stats

Whether the change in behavior I noted in the thread above is the result of the changes involved in the patch for Linux or not, I don't know.

[StainlessS]

From the linked thread,

RT_DebugF("TestRecursion : Debut")
i = Compte(0,1000,1)
RT_DebugF("TestRecursion Fin : %d",i)
Return BlankClip(pixel_type="YUV420P8").Subtitle("Fini !")

Function Compte(Int Ind,Int Fin, Int Incr) {
RT_DebugF("TestRecursion : %d",Ind)
Return (Ind < Fin) ? Compte(Ind+Incr,Fin,Incr) : Ind
}

I get this [on W10 x64]

00000102 01:51:05 [3576] RT_DebugF: TestRecursion : Debut
00000103 01:51:05 [3576] RT_DebugF: TestRecursion : 0
00000104 01:51:05 [3576] RT_DebugF: TestRecursion : 1
00000105 01:51:05 [3576] RT_DebugF: TestRecursion : 2
00000106 01:51:05 [3576] RT_DebugF: TestRecursion : 3
00000107 01:51:05 [3576] RT_DebugF: TestRecursion : 4

# cut

00000641 01:51:05 [3576] RT_DebugF: TestRecursion : 538
00000642 01:51:05 [3576] RT_DebugF: TestRecursion : 539
00000643 01:51:05 [3576] RT_DebugF: TestRecursion : 540
00000644 01:51:05 [3576] RT_DebugF: TestRecursion : 541

VDub2 disappears here.