RT_Stats, Compile-time/Runtime Functions v1.43 - 08 Oct 2014

[Forensic]

@StainlessS

I marvel at your awesome skill. It worked like a charm. Thank you.

[StainlessS]

@Forensic

Ooooo, you really are a sycophant, but dont stop, me likee. :)

[StainlessS]

RT_Stats new version v1.08, see 1st post.

Also contains new script function in zip:


Function ImageSplicer(String "FileName",int "SzLimit",Float "FAR",Bool "AutoCrop",Float "AutoLevelStrength",bool "ShowAutoLevel",bool "ShowBorder", \
bool "Subs", Float "CropThresh",Float "FPS",int "WMod",int "HMod",int "Baffle",Float "ScanPerc", \
bool "Debug", bool "Debug_QBC",bool "Debug_QLMM",String "Resizer") {
##########################################
# ImageSplicer v1.00 - 08 Jan 2013
#
# Requires GScript, RT_Stats v1.08 and QueryBorderCrop.avs, QueryLumaMinMax.avs
#
# ImageSplicer, joins together images of different sizes into a sort of slideshow, auto cropping borders, auto levelling, resizing or original size,
# border padding to chosen aspect ratio.
# Gets max width/height of all images (after optional auto cropping) and creates a background 'canvas' based on SzLimit & FAR.
# SzLimit==0: Images kept at original dimensions (post cropping), canvas made to encompass all cropped images.
# SzLimit>0: The shorter canvas dimension (FAR) limited to AT MOST SzLimit, where images will be resized to fit canvas.
# SzLimit<0: The shorter canvas dimension (FAR) set at abs(SzLimit) and images resized to fit canvas.
#
# Args:-
# Filename: Default "*.BMP|JPG|JPE|JPEG|PNG|TGA|TIF|GIF|TIFF". Image Path and wildcard with mulitple pipe separated extensions.
# SzLimit: Default=0, 0=No resize. Greater than 0 is limit on shorter dimension (see FAR), -ve resize to abs(SzLimit) shorter dimension.
# FAR: Default 4.0/3.0. Output Frame Aspect Ratio, if FAR>=1.0 then SzLimit affects height, else affects width.
# AutoCrop: Default=True. Crops borders off images.
# AutoLevelStrength: Default 0.0, Auto luma leveling, range 0.0 -> 1.0. 0.0 is OFF.
# ShowAutoLevel: Default False. If true then shows Auto leveling on right hande side of image ONLY.
# ShowBorder: Default False. When padding to maintain Frame Aspect Ratio, borders are shown in red.
# Subs: Default False. Shows Subtitled info on frames.
# CropThresh: Default -32.0. Leave at default -32.0 to allow auto adjustment (EDIT by QueryBorderCrop).
# CropThresh <=0.0 is auto thresh where thresh is set to (YPlaneMin + abs(CropThresh))
# CropThresh >0.0 is explicit threshold @ PC Levels, probably best avoided.
# Default -32.0 will for single frames auto adjust cropthresh to -0.58219178 (-0.5 * 255.0/(235-16)), ie -0.5 scaled to PC levels.
# CropThresh is then set to (YPlaneMin + abs(CropThresh)). This very sensitive threshold for single
# frame clips is to avoid possibility of overcropping. User specified cropthresh of eg -1.0 would scale to YPlaneMin + (1.0*255/219)
# RGB clips are sampled as YUV-Y @ PCLevels in QueryBorderCrop. See QueryBorderCrop.
# FPS: Default 1.0. Frames Per Second of output clip.
# WMod: Default 2. Output clip width is a multiple of WMod.
# HMod: Default 2. Output clip height is a multiple of HMod.
# Baffle: Default 3. Number of scanlines that must break a CropThresh to be accepted as Image. Avoids noise. Not usually altered.
# ScanPerc: Default 49.0. Percentage of image dimensions to scan per side. Not usually altered. See QueryBorderCrop.
# Debug: Default False. Show debug info via DebugView. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
# Debug_QBC: Default False. Debug QueryBorderCrop.
# Debug_QLMM: Default False. Debug QueryLumaMinMax.
# Resizer: Default "Spline36Resize(_W_,_H_)", the resizer to use. "_W_" and "_H_" are replaced by the resize width & height.
##########################################


PlanetCrop.avs v1.01 updated in zip.

[StainlessS]

v1.09 Bugfix, Fixed RT_RgbChanAve w==1 bug (always blue channel).

[StainlessS]

Coming, bit pissed at the moment but thought someone may like the below adds's (what It've got sooooo far)..

RT_GetWorkingDir()
Return current Working directory, directory of script, or dir set by SetWorkingDir().
The returned directory ends with a backslash.


RT_GetFullPathName(string "name")
Creates an absolute or full path name for the specified relative path 'name'.
eg 'RT_GetFullPathName("test.avs")' might return "D:\avs\test.avs".
Throws and error if cannot return path. (PS, pretty much there).


RT_FileDelete(string fn)
Delete file fn. Use with care.
Returns 0 = OK. -1 = access denied / read only. -2 = Path not found OR is Directory. -3 = Unknown error.


RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the
parts joined together.
'Get' (default 15, 1 -> 15), If set, Bit 0=DRIVE, 1=Dir, 2=Name, 4=Extension.
Add 1 for Drive (bit 0), add 2 for Dir (bit 1), add 4 for Name (bit 2), add 8 for Extension (bit 3).
Some combinations do not make sense, eg Drive + Extension (1+8=9). Below sensible options.
1 = Drive (includes trailing ':')
2 = Dir (includes trailing '\')
3 = Drive + Dir
4 = Name
6 = Dir + Name
7 = Drive + Dir + Name
8 = Extension (includes leading '.')
12 = Name + Extension
14 = Dir + Name + Extension
15 = Drive + Dir + Name + Extension
Assuming a current working directory of eg "D:\avs\avi", 'filename'="test.avi" and 'get'=15, returns "D:\avs\avi\test.avi",
so given a relative filename and default 'get'=15, is equivalent to GetFullPathName(filename).

Also, gonna add a func to get all Y(luma) stats info from a single call that is little more expensive than a single call.
ie, Ave luma, min / max / max-min / Median / Yinrange / stddev, all in a single call, and all costing little more than
a single RT call (single RT call about 30% slower than a built in Avisynth call (I thiink).
Need some sleep. (wish I had some beer) [EDIT: pretty much there,.]

[StainlessS]

This script to extract security camera footage with movment only, removing static scenes.
In response to this thread here:
http://forum.doom9.org/showthread.php?t=166974

Requires GScript, Grunt, RT_Stats v1.09+, FrameSelect and DegrainMedian Plugins.


AVISource("D:\AVS\AVI\CAP.avi")

# DO NOT USE DirectShowSource OR DENOISER THAT NEEDS LINEAR ACCESS eg HQDN3D
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN ShowMetrics OR MakeFramesCmd.

Function ShowMetrics(clip c,float th,int "x",int "y",int "w",int "h",bool "dim",bool "fast") {
# SUBTITLES, 1st number Frame number, 2nd Anchor frame number, Action, Diff between Frame and Anchor.
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) dim=Default(dim,true) Fast=Default(Fast,False)
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
cropped=Crop(x,y,w,h)
(dim && (w!=width || h!=height)) ? levels(0,1.0,255,0,128).Overlay(cropped,x=X,y=Y,opacity=1.0) : Last
Global ShowMetrics_Start=0 # Frame to start comparing with (Anchor)
Global ShowMetrics_KEPT=0 # Count of kept frames
Global ShowMetrics_PREV=0 # The 2nd most recent kept frame
ScriptClip("""
d = RT_LumaDifference(Last,Last,n2=ShowMetrics_Start,x=x,y=y,w=w,h=h)
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th)
Global ShowMetrics_KEPT = (keep) ? ShowMetrics_KEPT + 1 : ShowMetrics_KEPT
SS=String(current_frame) + " ] (" + String(ShowMetrics_Start) + ") " + ((keep)?" KEEP":" DROP") + " Diff="+String(d,"%03.2f") +\
" KEPT=" + String(ShowMetrics_KEPT) + " " + String(ShowMetrics_KEPT*100.0 / (current_frame+1.0),"%03.2f") + "%"
Global ShowMetrics_Start=(keep)?ShowMetrics_Prev:ShowMetrics_Start # Follow but NOT too closely (delayed by 1)
Global ShowMetrics_Prev=(keep)?current_frame:ShowMetrics_Prev # 1 kept frame delay
Subtitle(SS)
""",After_Frame=True,args="th,x,y,w,h") # Needs Grunt for args
SubTitle("Thresh = "+String(th,"%02.2f"),Align=1)
return ((Fast)?AssumeFPS(250.0):Last) .KillAudio()
}

Function MakeFramesCmd(clip c,float th,int "x",int "y",int "w",int "h",String "FileName") {
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) FileName=Default(FileName,"FrameSelect_Cmd.Txt")
(Exist(FileName)) ? RT_Call("cmd /c DEL /Q "+RT_QuoteStr(FileName),hide=true) : NOP # Delete existing
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
Global MakeFramesCmd_Start=0 # Frame to start comparing with (Anchor)
Global MakeFramesCmd_PREV=0
ScriptClip("""
# We DONT compare with previous frame, we compare with previous KEEP frame to avoid slow (and sneaky) burglars.
d = RT_LumaDifference(Last,Last,n2=MakeFramesCmd_Start,x=x,y=y,w=w,h=h) # 2 clip func, use 2 frame nos, not delta
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th) # Keep BOTH FIRST and LAST frame ALWAYS
Global MakeFramesCmd_Start=(keep)?MakeFramesCmd_Prev:MakeFramesCmd_Start # Follow but NOT too closely
Global MakeFramesCmd_Prev=(keep)?current_frame:MakeFramesCmd_Prev
WriteFileIf(FileName, "(keep)", "current_frame", append=True)
""",After_Frame=True,args="th,x,y,w,h,FileName") # Needs Grunt for args
return AssumeFPS(250.0).KillAudio()
}

Function SelectFrames(clip c,String "FileName",bool "Show") {
c
FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Show=Default(Show,False)
(Show) ? ShowTime() : NOP
FrameSelect(CMD=FileName) # Returns NO audio
}

Function GetFrames(clip c,float th,int "x",int "y",int "w",int "h",String "FileName",clip "dclip",bool "Show") {
Default(dclip,c) # Use denoised clip for scanning
MakeFramesCmd(th,x,y,w,h,FileName) # Make the command file for FrameSelect()
GSCript("""
For(i=0,framecount-1) {
RT_YankChain(n=i) # Force Process, MakeFrames
}
c.SelectFrames(FileName,Show=Show) # Extract frames and return to client
""")
}

###################

X=128 Y=172 W=-128 H=-0 # Area Of Interest, avoid eg passing traffic or timecode.

THRESH =1.3 # Whatever, use ShowMetrics, need adjust, suggest 0.5 -> 1.5, need check against source.

DENOISE=7 # DegrainMedian LimitY, 0 = OFF, higher stronger, Suggest 4 -> 8
DN_MODE=0 # DegrainMedian mode, see DegrainMedian docs, suggest 0 -> 2, higher weaker
SHOW=True # Show original Time on final output

# Optional Denoise, perhaps there are better ones, Produces identical results in below functions. (OK for non-linear access).
dclip = (DENOISE>0) ? ConvertToYV12().DeGrainMedian(limitY=DENOISE,limitUV=0,mode=DN_MODE).GreyScale() : Last

###################

ShowMetrics(dclip,THRESH,X,Y,W,H) # Show metrics to get best Threshold 'th'. Denoised clip shown in GrayScale.
#MakeFramesCmd(dclip,THRESH,X,Y,W,H) # Make Frames Cmd file for SelectFrames()
#SelectFrames(show=SHOW) # Return Frames with EXISTING frames Command file (after MakeFramesCmd)
#GetFrames(THRESH,X,Y,W,H,dclip=dclip,show=SHOW) # Make Frames command file and then extract frames automatically in 1 pass.


UPDATED:

[StainlessS]

Ref previous post:
I stuck a webcam out my window and captured about 2750 frames at about midnight.
Running through previous script with Thresh set to 1.0, it reduced frame count to 145 fames
with 1 bike and 3 cars passing, lights behind curtains going off and a fox passing close by.
So, it does not seem too bad as it is, but might be happier if more sensitive to small localized movement.
Would probably do better with LumaDifference (SAD, Sum Of Absolute Differences) being replaced with
Sum Of Squared Differences. I'll add a func to RT_Stats to do that.
Ooooh, there's that fox making a weird racket.

EDIT: Old Webcam 640x480@15FPS Codec Indeo IV50, murky window glass with street lamp shining upon it.

[StainlessS]

Updated post #106.
Added dclip denoised detection clip + other mods.

[StainlessS]

New Version RT_Stats v1.10 17 Mar 2013. See first post.

Updates in brief:

RT_Debug(string s1, ... , string sn,bool "name"=true)
v1.10, Added name arg, setting false inhibits title "RT_Debug:" output, set s1 to your own title instead.

RT_GetLastError()
Return a error number describing previous system error (if available). Perhaps use after eg RT_Call.

RT_GetLastErrorString()
Return a string describing previous system error (if available).

RT_TxtSort(String,int 'mode'=0)
Function to sort a Chr(10) separated multiline string.
The String arg is the multiline string, and 'mode' can be in range 0 to 11.
v1.10, added modes 8 -> 11.
8 = Sort ascending, Integer number Strings
9 = Sort ascending, float number Strings
10= Sort decending, Integer number Strings
11= Sort decending, float number Strings

RT_StrPad(String,Int n,string "c"=" ")
Pads an unnamed string to at least int n (unnamed) length with first character of string c (Default SPACE).

RT_GetWorkingDir()
Return current Working directory, directory of script, or dir set by SetWorkingDir().
The returned directory ends with a backslash.

RT_GetFullPathName(string "name")
Creates an absolute or full path name for the specified relative path 'name'.
eg 'RT_GetFullPathName("test.avs")' might return "D:\avs\test.avs".
Throws and error if cannot return path.

RT_FilenameSplit(string filename,int "get"=15)
Splits the un-named filename string into component parts selected by 'get' bit flags arg and returns the
parts joined together.
'Get' (default 15, 1 -> 15), If set, Bit 0=DRIVE, 1=Dir, 2=Name, 4=Extension.
Add 1 for Drive (bit 0), add 2 for Dir (bit 1), add 4 for Name (bit 2), add 8 for Extension (bit 3).
Some combinations do not make sense, eg Drive + Extension (1+8=9). Below sensible options.
1 = Drive (includes trailing ':')
2 = Dir (includes trailing '\')
3 = Drive + Dir
4 = Name
6 = Dir + Name
7 = Drive + Dir + Name
8 = Extension (includes leading '.')
12 = Name + Extension
14 = Dir + Name + Extension
15 = Drive + Dir + Name + Extension
Assuming a current working directory of eg "D:\avs\avi\", 'filename'="test.avi" and 'get'=15, returns "D:\avs\avi\test.avi",
so given a relative filename and default 'get'=15, is equivalent to RT_GetFullPathName(filename).

RT_GetFileExtension(string s)
Returns filename extension from the supplied filename, including '.' prefix.


RT_FileDelete(string fn)
Delete unnamed file fn. Use with care.
Returns 0 = OK. -1 = access denied / read only. -2 = Path not found OR is Directory. -3 = Unknown error.

RT_Undefined()
Returns same as undefined() in v2.6. Defined(RT_Undefined()) would return false.

RT_VarExist(string)
Given the name (string) of the variable that you want to test for existence, returns true if exists, else false.
Eg, #a=32
RT_Debug(string(RT_VarExist("a"))) # would output false to debugview unless '#a=32' uncommented. {Defined(a) would fail with error}.
return colorbars()

RT_Ord(string)
Returns same as 'Ord' in v2.6, the ASCII code for the 1st letter in string (0 if empty string "").

RT_Timer()
Returns time in seconds since start of process.

RT_Ystats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=(width<=720?2:3),int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="YS_")
Returns multiple results as for above single frame Luma sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", and "Matrix", are as for all other clip functions, "threshold" used only for "RT_YPlaneMin", "RT_YPlaneMax" and
"RT_YPlaneMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_YInRange" equivalent routine with same functionality.
The only new args are int "Flgs" which selects which results you want returned and the string "Prefix" that is prepended
to the returned global variables.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.

Flgs_Bit_Number Add_To_Flgs Equivalent_Funcion Global_Var_Set_Excluding_Prefix
0 1($01) RT_YPlaneMin() "yMin" (0->255)
1 2($02) RT_YPlaneMax() "yMax" (0->255)
2 4($04) RT_YPlaneMinMaxDifference() "yMinMaxDiff" (0->255)
3 8($08) RT_YPlaneMedian() "yMed" (0->255)
4 16($10) RT_AverageLuma() "yAve" (0.0->255.0)
5 32($20) RT_YPlaneStdev() "yStdev" (0.0->255.0)
6 64($40) RT_YInRange() "yInRng" (0.0->1.0)
RT_Ystats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=127($7F) are all bits set and so sets ALL global vars at once.
RT_Ystats(flgs=1+2+16) would set global vars "YS_yMin", "YS_yMax" and "YS_yAve" for full frame current_frame.


****** SCRIPT CONVERSION TO PLUGIN FUNCTIONS ********

Below fns converted from previously existing script functions which will also be supplied so that you can see the logic,
any changes after v1.10 plugin versions may not be reflected in the later script versions.

RT_GetSAR(clip,dar float)
Gets Sample (pixel) Aspect Ratio from DAR Display Aspect Ratio.

RT_GetDAR(clip,sar float)
Gets DAR Display Aspect Ratio from SAR, Sample (pixel) Aspect Ratio.

RT_SignalDAR(float) # See Megui wiki (name change from SetDAR)
Signal DAR to MEGUI. Sets global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

RT_GetCropDAR(clip,float DAR,float "X"=0,float "Y"=0,Float"W"=0,float "H"=0)
Call prior to Crop/Resize with (possibly fractional) cropping to calc resultant DAR, X,Y,W,H are cropping coords
DAR = FAR * SAR ::: FAR = DAR / SAR ::: SAR = DAR / FAR
Cropping affects FAR & usually DAR, resizing does not affect DAR (unless also cropped). Resizing affects FAR and maybe SAR.
We dont allow eg -ve X

RT_QueryLumaMinMax(clip c,int "Samples"=24,float "Ignore"=0.2,String "Prefix"="QLMM",bool "DEBUG"=false,int "X"=0,int "Y"=0,int "W"=0,int "H"=0, \
int "Matrix"=(width<=720?2:3))
Prescan function to get Luma Min,Max for clip scanning Samples frames area x,y,w,h.
Returns luma min/max as string Default eg "QLMMMin=25 QLMMMax=244", use eg "Eval(RT_QueryLumaMinMax())" to set variables for use in script.
Samples:=24 = frames in clip to sample.
Ignore:=0.2=Percentage of extreme pixels to ignore (noise) when getting luma min/max.
Prefix:="QLMM"=Prefix for return string values.
X=Y=W=H=0=Full frame, as crop, area to examine.
DEBUG:=false= dont show. True=debug info. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
Matrix:, For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
Default for is 2(PC601) if width <= 720 else 3(PC709) : YUV not used

Function RT_QueryBorderCrop(clip c,int "Samples"=24,Float "Thresh"=-32,bool "Laced"=true,int "XMod",int "YMod",int "WMod",int "HMod", \
bool "Relative"=false, String "Prefix"="QBCrop",int "RLBT"=15,bool "DEBUG"=false,float "Ignore"=0.2,int "Matrix"=(width<=720?2:3), \
Float "ScanPerc"=49.0,int "Baffle"=4, bool "ScaleAutoThreshRGB"=true,bool "ScaleAutoThreshYUV"=false)

Prescan function to get coords for eg cropping black borders by sampling Samples frames. Planar, YUY2, RGB.
Borders are detected by sampling at Samples frames, at scanlines (h/v) using AverageLuma (RGB is converted to Luma Y at either TV or
PC levels, See Matrix). This sampling is done on all 4 sides of the frame.
If a scanline Average luma is below or equal to Thresh, then is considered possible black border, above considered possible image,
if Baffle [default=4] adjacent scanlines above Thresh, then it IS image.

Simultaneously returns 4 sets of strings holding crop coords, here is one set: "QBCropX=8 QBCropY=8 QBCropW=640 QBCropH=480".
String sets are Chr(10) separated, the 'exact found' coords set as above, not rounded at all and possibly throwing an error if
you try to crop, although could use in eg resize.
Second set using eg "QBCropXL=8", which is CropLess("L" trailer), ie when rounding for Xmod,WMod etc may leave some black border.
Third set using eg "QBCropXM=8", which is CropMore, ie when rounding for Xmod,WMod etc may crop some image.
Forth set using eg "QBCropXP=8", which is CropPlus, moves border positions an extra 2 pixels inwards on each edge and then as CropMore.
The non-exact coords try to center the coords when cropping for larger WMod/HMod's so as to evenly crop from eg both left and right instead of
just one side. Also returned in the return string is the used Threshold, perhaps set by AutoThresh, as eg "QBCropThresh=32.0"
You can use eg Eval(QueryBorderCrop()) to set values into QBCropX, QBCropY,QBCropW,QBCropH etc variables.



Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(width<=720?2:3),String "Prefix"="YIRL_",bool "Debug"=false)

Clip function to scan frame n+delta for an area where percentage of pixels in luma range Lo to Hi (inclusive) breaks a threshold over at least Baffle
consecutive number of scanlines (all four sides). Returns True on Successful location else false. On success set Global var Coords.
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold. Default is any pixel in the range Lo to Hi will break threshold.
Lo: Default 128, lower bound of pixel values to search for.
Hi: Default 255, upper bound of pixel values to search for.
Matrix: Default for RGB is 2(PC601) if width <= 720 else 3(PC709) : YUV not used
Prefix: Default "YIRL_", prefix for Global Var coords set on successful location of object, eg YIRL_X, YIRL_Y, YIRL_W, YIRL_H.
Debug: Default false. If true outputs info to debugView.

[StainlessS]

See also previous post:

Here result of script included in RT_Stats v1.10 (EDIT: Required GScript)


00000024 20:06:29 RT_Debug: RT_Stats Timing Test
00000025 20:06:29 RT_Debug: Timing 14x5(70) runs on 2500 Frames (Total Frames=175000)
00000026 20:07:06 RT_Debug: RT_YankChain 7.19Secs : 347.56FPS :: Basic GetFrame() [ADJUSTMENT]
00000027 20:07:58 RT_Debug: YPlaneMin 10.55Secs : 237.03FPS :: ADJUSTED 3.35Secs 745.38FPS
00000028 20:08:47 RT_Debug: RT_YPlaneMin 9.66Secs : 258.75FPS :: ADJUSTED 2.47Secs 1012.69FPS
00000029 20:09:39 RT_Debug: YPlaneMax 10.55Secs : 237.03FPS :: ADJUSTED 3.35Secs 745.38FPS
00000030 20:10:28 RT_Debug: RT_YPlaneMax 9.67Secs : 258.48FPS :: ADJUSTED 2.48Secs 1008.47FPS
00000031 20:11:23 RT_Debug: YPlaneMinMaxDif 11.03Secs : 226.74FPS :: ADJUSTED 3.83Secs 652.23FPS
00000032 20:12:13 RT_Debug: RT_YPlaneMinMaxDif 10.00Secs : 250.00FPS :: ADJUSTED 2.81Secs 890.63FPS
00000033 20:13:06 RT_Debug: YPlaneMedian 10.59Secs : 235.99FPS :: ADJUSTED 3.40Secs 735.15FPS
00000034 20:13:55 RT_Debug: RT_YPlaneMedian 9.73Secs : 256.82FPS :: ADJUSTED 2.54Secs 983.73FPS
00000035 20:14:38 RT_Debug: AverageLuma 8.63Secs : 289.86FPS :: ADJUSTED 1.43Secs 1745.81FPS
00000036 20:15:20 RT_Debug: RT_AverageLuma 8.44Secs : 296.30FPS :: ADJUSTED 1.24Secs 2009.12FPS
00000037 20:17:57 RT_Debug: Avisynth_ALL 31.44Secs : 79.52FPS :: ADJUSTED 24.24Secs 103.12FPS
00000038 20:18:56 RT_Debug: RT_Avisynth_ALL 11.83Secs : 211.35FPS :: ADJUSTED 4.64Secs 539.30FPS
00000039 20:19:56 RT_Debug: RT_ALL 11.87Secs : 210.62FPS :: ADJUSTED 4.68Secs 534.57FPS
00000040 20:19:56 RT_Debug: Total Runtime = 13.45 Mins


720x576@25 UT Video Codec. Avisynth v2.6a4 Standard. (RT Compiled TK3, VS6 not too much different)

Does 5 runs on each func, throws away fastest and slowest times and gets average of remaining three.

Adjusted times have the timing for reading frames (eg HardDrive, codec, sourcefilter) subtracted from them.
Avisynth_ALL samples each frame via all the Avisynth Native Luma Sampling runtime funcs,
RT_Avisynth_ALL() [RT_YStats] does same but in 1 function. RT_ALL includes two additional Runtime funcs that
are not native to Avisynth, ie 7 sampling funcs altogether at once.

EDIT: If your results contradict above, post them here (above surprises the hell out of me, RT_ just standard CPP).
Above results on Core 2 Duo duel core. Similar results speed ratios on single core Prescot and another P4 machine.

EDIT: Below is the RT_Ystats() doc which is repeated here from previous post (is a bit hidden there):

RT_Ystats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "Matrix"=(width<=720?2:3),int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="YS_")
Returns multiple results as for above single frame Luma sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", and "Matrix", are as for all other clip functions, "threshold" used only for "RT_YPlaneMin", "RT_YPlaneMax" and
"RT_YPlaneMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_YInRange" equivalent routine with same functionality.
The only new args are int "Flgs" which selects which results you want returned and the string "Prefix" that is prepended
to the returned global variables.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.

Flgs_Bit_Number Add_To_Flgs Equivalent_Funcion Global_Var_Set_Excluding_Prefix
0 1($01) RT_YPlaneMin() "yMin" (0->255)
1 2($02) RT_YPlaneMax() "yMax" (0->255)
2 4($04) RT_YPlaneMinMaxDifference() "yMinMaxDiff" (0->255)
3 8($08) RT_YPlaneMedian() "yMed" (0->255)
4 16($10) RT_AverageLuma() "yAve" (0.0->255.0)
5 32($20) RT_YPlaneStdev() "yStdev" (0.0->255.0)
6 64($40) RT_YInRange() "yInRng" (0.0->1.0)
RT_Ystats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=127($7F) are all bits set and so sets ALL global vars at once.
RT_Ystats(flgs=1+2+16) would set global vars "YS_yMin", "YS_yMax" and "YS_yAve" for full frame current_frame.

[StainlessS]

OOPS, mismatch in docs, script YInRangeLocate.avs and RT_YInRangeLocate default Prefix,
docs say "YIRL" avs script "YIRL" and "YIRL_" in RT_YInRangeLocate. Have changed online docs to "YIRL_"
and will update doc and avs script in next version to "YIRL_".


Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(width<=720?2:3),String "Prefix"="YIRL_",bool "Debug"=false)


Affects none of the included scripts (PlanetCrop.avs) as they use an explicit Prefix.

[StainlessS]

Oh dear!
Another update, v1.11 19 Mar 2013.
Last minute change to v1.10, I missed out a '!' in check on clips in RT_LumaCorrelation and RT_LumaDifference, so they will
give a "Must Have Two Clips" type error. Fixed here in v1.11. See 1st post or in sig.

Also posted a request for a name of new func here (in developers forum):-
LINK REMOVED, THINK I MADE A BOOBOO

If you have a better name then please submit.

As a temp measure, v1.11 includes func as RT_LumaSNSSD() # Scaled Normalized Sum Of Squared Differences.


RT_LumaSNSSD(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value Scaled Normalized Sum of Squared Differences. (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h,
and clip2 frame (n2+delta2) area x2,y2,w,h. Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Calculates Sum Of Squared Differences, takes the Square root of it and then scales in range 0.0 -> 255.0.


EDIT: Will also add RT_LumaSSSD() {or other name} for Scaled Sum of Squared Differences.

[kypec]

Another update, v1.11 19 Mar 2013.

yet you updated the thread title too much ahead in time "19 May 2013" :D

[StainlessS]

Thank you kind sir, making lots of mistakes lately, must be getting senile.

Think I may also have dropped a B@ll@ck with the RT_LumaSNSSD() thing,
it seems to be graphing linearly (not much different to SAD LumaDifference).
Perhaps an early April fool thing. :)

I was going to experiment with the RT_LumaSNSSD thing and had just put back the
source into RT_Stats source when I found the "!" error in v1.10, as I was reluctant
to remove it again for the v1.11 update I just pushed it out, but not sure that it is terribly
useful.
Have just now implemented the RT_LumaSSSD and that seems to be working as expected.

[StainlessS]

And again. New Version v1.12.


# PROBABLY REMOVED IN NEXT VERSION, I THINK I MADE A BOOBOO
RT_LumaSNSSD(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value Scaled Normalized Sum of Squared Differences. (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h,
and clip2 frame (n2+delta2) area x2,y2,w,h. Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Calculates Sum Of Squared Differences, takes the Square root of it and then scales in range 0.0 -> 255.0.



# CONSIDER IT EXPERIMENTAL
RT_LumaSSSD(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value Scaled Sum of Squared Differences. (0.0 -> 255.0) between clip frame (n+delta) area x,y,w,h,
and clip2 frame (n2+delta2) area x2,y2,w,h. Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Calculates Sum Of Squared Differences, then scales in range 0.0 -> 255.0.


EDIT: @ Jmac, I think the Moon2 sample took about 6.5 minutes in old version of PlanetCrop, knocked it down to about
18.5 seconds in current version, :) :) :) :) :) :) :) :)

[StainlessS]

New version, RT_Stats v1.13, see first post.
RT_QueryBorderCrop CropLess/CropMore modified (did not quite work as it should have).

[StainlessS]

New Version, RT_Stats v1.14, see 1st post.

Added RT_FindStr. Mods:- RT_Debug, StrReplace fns, RT_NumberValue and RT_HexValue.

Newly Added:


RT_FindStr(String S,string Substr,bool "Sig"=True,int "Pos"=1)
Finds unnamed string Substr in unnamed string S.
Optional bool Sig: Default=True, does case significant comparison, insignificant if false.
Optional int Pos: Default=1, is start position in string S to find string Substr. If Pos smaller than 1 or greater than length of string s,
then will return 0, Not Found.
Returns position (1 relative) of found string within the FULL LENGTH string S, or 0 if not found.
Differs from Avisynth FindStr in Sig and Pos args and in that if either string S or string Substr is "" then will ALWAYS return 0, 'Not Found',
Avisynth FindStr always returns 1 if SubStr="" even if S is also "".
v1.14, New.


Mods:


RT_Debug(string s1, ... , string sn,bool "name"=true)
Non-clip function. Takes one or more strings, joins them together and then outputs the combined
string to OutputDebugString(), if strings are NOT space separated, it inserts a single space between them.
Useful during development Compile-time/Run-time scripts to monitor status in real time.
Can also output multiline string obtained via eg RT_ReadTxtFromFile.
Name: bool Default=True. If False, then does NOT output "RT_Debug: " at the beginning of each line, you
might want to replace with your own title in string s1.
Use DebugView utility to view and capture output. http://technet.microsoft.com/en-gb/sysinternals/bb545027
Usage eg:- RT_Debug("["+String(42)+"]","Going pretty good so far @","Line "+String(19))
Would Produce this "RT_Debug: [42] Going pretty good so far @ Line 19"
The above "RT_Debug:" title is present in all RT_Debug() output to the debugview window and you can (in DebugView),
filter out all lines that do not contain "RT_Debug:" so as to skip stuff that is of no interest (eg AVSPMod generates
some output, not sure but I think Avisynth v2.58 also generates lots of debug stuff).
NOTE, Any program can send output to DebugView window, if no debug viewer is present, it normally just disappears
down the plughole.
In DebugView, filter for eg "RT_Debug:" to skip all the extraneous stuff generated by other plugins/programs.
(In DebugView "Menu:Edit-Filter/Highlight", and enter "RT_Debug:".
v1.14, No longer converts characters 127->255 to SPACE, only chr(2) converted to chr(1).
Characters displayed via debugview are same as those via SubTitle, but those above 127 are dissimilar to the ones
produced via Avisynth source Info.h, used during eg plugin metrics display.
Example to show characterset with both RT_Debug & Subtitle:
# ShowCharacterset.avs # Requires GScript & RT_Stats v1.14+
S=""
GSCript("""
for(y=0,15) {
s=s+"$"+RT_Hex(y*16,2)+ " "
for(x=0,15) {
n=y*16+x
c=(n==0 || n==9 || n==10 || n==13) ? Chr(1) : Chr(n)
# Convert Odd Effect chars to NO GRAPHIC {Square} Chr(1)
# NOTE, DebugView
# Chr(0) {NUL} [Null or End of String], converted here to NO GRAPHIC Chr(1).
# Chr(9) {TAB} [Horizontal TAB], produces SPACE, converted here to NO GRAPHIC Chr(1).
# Chr(10) {LF} [Line Feed or NewLine], converted here to NO GRAPHIC Chr(1).
# Chr(13) {CR} [Carriage Return], converted here to NO GRAPHIC Chr(1).
#
# Chr(2) {SOH} [Start Of Heading ], Produces ODD effect , converted internally by RT_Debug to a NO GRAPHIC Chr(1).
s=s+c
}
s=s+Chr(10)
}
""")
colorbars(width=320,height=320).Trim(0,-1)
RT_Debug(s)
s=RT_StrReplace(s,Chr(10),"\n")
Subtitle(s,font="Courier New",lsp=0)
return Last

RT_StrReplace(string source,string find,string replace,bool "sig"=True) # Based on Algorithm by Vampiredom, Gavino & IanB.
String args 'source', 'find' and 'replace' unnamed and compulsory.
Takes a source string, searches for all occurences of find string and replaces the found strings with the replace string.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Newlines are treated no differently to other characters, and could be replaced/deleted.
v1.14, added 'sig' arg,default true is Case Significant (as previous version). Set false for case insignificant find string.

RT_StrReplaceDeep(string source,string find,string replace,bool "sig"=True)
String args 'source', 'find' and 'replace' unnamed and compulsory.
Takes a source string, searches for all occurences of find string and replaces the found strings with the replace string.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Newlines are treated no differently to other characters, and could be replaced/deleted.
Differs from RT_StringReplace in that if length of replace < length of find string will rescan after a replacement to see if
the replaced string combines with currently existing string to create another find string, If length replace >= find, then calls
RT_StringReplace instead. Useful to replace eg " " with " ", (two spaces with 1 space), where more than two consecutive
spaces would end up with more than one consecutive resulting space if using plain RT_StrReplace.
v1.14, added 'sig' arg,default true is Case Significant (as previous version). Set false for case insignificant find string.


RT_StrReplaceMulti(string source,string find,string replace,bool "sig"=True)
String args 'source', 'find' and 'replace' unnamed and compulsory.
Takes a source string, searches for all occurences of find substrings and replaces the found substrings with the replace substrings.
Find string and Replace string are both multiline [Chr(10) separated] strings, both of n lines where find[i] and replace[i]
are matched pairs, ie the first line of a multiline find is replaced by the first line of a multiline replace, etc.
Can use "" in replace string (only in replace) to delete the found substrings from the source string.
Differs from RT_StrReplace, in that both find and replace strings are multiline strings and MUST match in number of lines.
Cannot use either carriage return [Chr(13)] or newline [Chr(10)] in find or replace as these are interpreted as end of current line,
you would need to use RT_StrReplace() instead.
eg S=RT_StrReplaceMulti("THE cat SAT.","SAT"+Chr(10)+"THE","sat"+Chr(10)+"The") produces "The cat sat.".
v1.14, added 'sig' arg,default true is Case Significant (as previous version). Set false for case insignificant find string.


RT_HexValue(String,"pos"=1)
Returns an int conversion of the supplied hexadecimal string.
Conversion will cease at the first non legal number base digit, without producing an error
Fixes HexValue bug in 2.58 & 2.6a3. eg "FFFFFFFF" returns 2147483647 (0x7FFFFFFF) instead of the correct -1 (As most/all calculators)
Bug is fixed in Avisynth v2.6a4.
v1.14, Added optional pos arg default=1, start position in string of the HexString, 1 denotes the string beginning. Will return 0
if error in 'pos' ie if pos is less than 1 or greater than string length.


RT_NumberValue(String,int "base"=10,int "pos"=1)
Returns an int conversion of the supplied number base string.
Base, (10, 2 -> 36) is the number base that the string uses.
To convert a Decimal string to a Float, use Avisynth native "Value()" func.
Conversion will cease at the first non legal number base digit, without producing an error.
eg RT_NumberValue("1100",2) returns 12 from the binary string.
v1.14, Added optional pos arg default=1, start position in string of the number string, 1 denotes the string beginning. Will return 0
if error in 'pos' ie if pos is less than 1 or greater than string length.




Result of the ShowCharacterSet.avs Script (doom9 swallows some of the chars, shows space,should look like NO GRAPHIC char $7F, square):


00000004 04:48:40 RT_Debug: $00
00000005 04:48:40 RT_Debug: $10
00000006 04:48:40 RT_Debug: $20 !"#$%&'()*+,-./
00000007 04:48:40 RT_Debug: $30 0123456789:;<=>?
00000008 04:48:40 RT_Debug: $40 @ABCDEFGHIJKLMNO
00000009 04:48:40 RT_Debug: $50 PQRSTUVWXYZ[\]^_
00000010 04:48:40 RT_Debug: $60 `abcdefghijklmno
00000011 04:48:40 RT_Debug: $70 pqrstuvwxyz{|}~
00000012 04:48:40 RT_Debug: $80 €�‚ƒ„…†‡ˆ‰Š‹Œ�Ž�
00000013 04:48:40 RT_Debug: $90 �‘’“”•–—˜™š›œ�žŸ
00000014 04:48:40 RT_Debug: $A0 *¡¢£¤¥¦§¨©ª«¬*®¯
00000015 04:48:40 RT_Debug: $B0 °±²³´µ¶·¸¹º»¼½¾¿
00000016 04:48:40 RT_Debug: $C0 ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏ
00000017 04:48:40 RT_Debug: $D0 ÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß
00000018 04:48:40 RT_Debug: $E0 àáâãäåæçèéêëìíîï
00000019 04:48:40 RT_Debug: $F0 ðñòóôõö÷øùúûüýþÿ

[StainlessS]

New Version v1.15, see 1st post.
v1.15 pretty much same as v1.14, made small alteration to RT_YInRangeLocate, Internally, Baffle separately limited to dimensions on both axis.

Also added below script to v1.15 zip.


AVISource("D:\avs\test.avi")

# Pick a test below

#Return TestClip($FFFFFF).TweenFrames(LimitLo=200.0,LimitHi=255.0,Show=true) # Fix White frames PROGRESSIVE
#Return TestClip($000000).TweenFrames(LimitLo=0.0,LimitHi=32.0,Show=true) # Fix Black frames PROGRESSIVE
#Return TestClipI($FFFFFF).TweenFramesI(LimitLo=200.0,LimitHi=255.0,Show=true) # Fix White frames INTERLACED
#Return TestClipI($000000).TweenFramesI(LimitLo=0.0,LimitHi=32.0,Show=true) # Fix Black frames INTERLACED

Function TweenFrames(clip c,float "LimitLo",float "LimitHi",bool "show") {
# TweenFrames() by StainlessS
# PROGRESSIVE. Requires RT_Stats & MvTools 2
# Replace isolated bad frames eg Black or White with a frame tweened from those either side using MvTools, and RT_Stats.
# LimitLo: is minimum luma value of frame to replace. Allows select, eg black/White.
# LimitHi: is maximum luma value of frame to replace.
# Show: Puts indicator on fixed frames.
c
LimitLo=Float(Default(LimitLo,200.0)) # Average Luma greater or equal to this eligible for replacement
LimitHi=Float(Default(LimitHi,255.0)) # Average Luma lesser or equal to this eligible for replacement
show=Default(show,False)
Prev=DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("FRAME FIXED") : inter
CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>=LimitLo && ave<=LimitHi && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"]","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""
CondS=RT_StrReplaceMulti(CondS,"LimitLo"+Chr(10)+"LimitHi",String(LimitLo)+Chr(10)+String(LimitHi))
ConditionalSelect(Last,CondS,Last,inter) # Fix bad frames
return Last
}

Function TweenFramesI(clip c,float "LimitLo",float "LimitHi",bool "show") {
# TweenFramesI() by StainlessS
# INTERLACED.
# Replace isolated bad fields eg Black or White with a field tweened from those either side using MvTools, and RT_Stats.
# LimitLo: is minimum luma value of field to replace. Allows select, eg black/White.
# LimitHi: is maximum luma value of field to replace.
# Show: Puts indicator on fixed fields.
c
LimitLo=Float(Default(LimitLo,200.0)) # Average Luma greater or equal to this eligible for replacement
LimitHi=Float(Default(LimitHi,255.0)) # Average Luma lesser or equal to this eligible for replacement
show=Default(show,False)
SEP=SeparateFields()
Even=SEP.SelectEven()
Prev=Even.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("EVEN FIELD FIXED") : inter
CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>=LimitLo && ave<=LimitHi && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"] EVEN","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""
CondS=RT_StrReplaceMulti(CondS,"LimitLo"+Chr(10)+"LimitHi",String(LimitLo)+Chr(10)+String(LimitHi)) # Import explicit into script
EvenFixed=ConditionalSelect(Even,CondS,Even,inter) # Fix bad even fields
Odd=SEP.SelectOdd()
Prev=Odd.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("ODD FIELD FIXED",align=1) : inter
CondS=RT_StrReplace(CondS,"EVEN","ODD")
OddFixed=ConditionalSelect(Odd,CondS,Odd,inter) # Fix bad Odd fields
Interleave(EvenFixed,OddFixed)
Weave()
return Last
}

Function TestClip(clip c,int "Color") {
# Replace every 8th frame with blank.
c Color=Default(Color,$FFFFFF) KillAudio() BAD=Last.BlankClip(color=Color)
A=SelectEvery(8,0) B=SelectEvery(8,1) C=SelectEvery(8,2) D=SelectEvery(8,3)
E=SelectEvery(8,4) F=SelectEvery(8,5) G=SelectEvery(8,6)
Interleave(A,B,C,D,E,F,G,BAD) # every 8th frame replaced with BAD
Return Trim(0,1000)
}

Function TestClipI(clip c,int "Color") {
# Replace every 8th EVEN field and every 12th ODD field with a blank.
Color=Default(Color,$FFFFFF) c=c.KillAudio() SEP=c.SeparateFields BAD=SEP.BlankClip(color=Color) EVEN=SEP.SelectEven()
A=EVEN.SelectEvery(8,0) B=EVEN.SelectEvery(8,1) C=EVEN.SelectEvery(8,2) D=EVEN.SelectEvery(8,3)
E=EVEN.SelectEvery(8,4) F=EVEN.SelectEvery(8,5) G=EVEN.SelectEvery(8,6) EVEN=Interleave(A,B,C,D,E,F,G,BAD)
ODD=SEP.SelectODD() A=ODD.SelectEvery(12,0) B=ODD.SelectEvery(12,1) C=ODD.SelectEvery(12,2)
D=ODD.SelectEvery(12,3) E=ODD.SelectEvery(12,4) F=ODD.SelectEvery(12,5) G=ODD.SelectEvery(12,6)
H=ODD.SelectEvery(12,7) I=ODD.SelectEvery(12,8) J=ODD.SelectEvery(12,9) K=ODD.SelectEvery(12,10)
ODD=Interleave(A,B,C,D,E,F,G,H,I,J,K,BAD) Interleave(EVEN,ODD) Weave()
Return Trim(0,1000)
}


Mod of script posted here:-
http://forum.doom9.org/showthread.php?t=167632

EDIT: Maybe TweenFramesI should be called TweenFields.

[StainlessS]

New version RT_Stats v1.16. See 1st post.

Function added:

RT_LumaSceneChange(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3))
Returns FLOAT value (0.0 -> 255.0) scene change detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
Is implemented sort of like:- mt_lutxy(mt_polish("((x-y)/4)^2")).AverageLuma() where x=clip(n+delta) and y=clip2(n2+delta2).
Scene change metric between current frame and next frame, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,delta2=1,h=-32)
and exactly the same thing again:
SC = RT_LumaSceneChange(Last,Last,n2=current_frame+1,h=-32)
There are two arg clips for flexibility, might usually both be same, but might want to measure against two separate clips and so
implemented as above.
Scene change detection between current frame top field and bottom field, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,y2=1,interlaced=true,h=-32) # EDIT: Expect higher metrics here
A threshold of about 80.0 is pretty good for a hi/lo motion clip, above is scene change, below not scene change although metrics
before and after scene change would best be used as well.


See this post for usage example.
http://forum.doom9.org/showthread.php?p=1625975#post1625975

[StainlessS]

New Version RT_Stats v1.17, see 1st post.

v1.17, Added RT_LumaMovement. RT_Ord mod.


RT_LumaMovement(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3),
int "Mode"=0,int "BlkW"=32,int "BlkH"=32)
Returns FLOAT value (0.0 -> 255.0) movement detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
There are two modes available, 0 (Default) = RT_LumaDifference() : 1 = RT_LumaSceneChange().
BLKW/BLKH (default 32), Block size. Minimum 8 (silently limited to frame dimensions).
RT_LumaMovement() splits the frames into block sized lumps and calls the desired mode RT_LumaDifference or RT_LumaSceneChange
function to get a metric for each of those blocks. The metric returned is the metric with the highest value, and so is more
sensitive to localized movement.
If Interlaced is true, then only every other horizontal scanline in BLKH will be scanned.
v1.17, New.

RT_Ord(string,int "pos"=1)
Returns same as 'Ord' in v2.6, the ASCII code for the 1st letter in string (0 if empty string "").
v1.17, Added pos arg, default 1, ie 1st character in string. Returns 0 if pos < 1 or greater than string length.
Could possibly use a string as a sort of BYTE sized int array containing non zero values.


New Addition SecurityCamExtractMotion.AVS


AVISource("D:\AVS\AVI\CAP4.avi") #.Trim(32,0)

############################################
# SecurityCamExtractMotion.AVS, v1.0, By StainlessS.
# Requires RT_Stats v1.17, FrameSelect v1.03, GScript, Grunt, Maybe DeGrainMedian if required.
#
# DO NOT USE DirectShowSource OR DENOISER THAT NEEDS LINEAR ACCESS eg HQDN3D
# MUST SCAN STRAIGHT THROUGH, NO JUMPING ABOUT IN ShowMetrics OR MakeFramesCmd.
############################################

X=32 Y=32 W=-32 H=-0 # Area Of Interest, avoid eg passing traffic or timecode.

MODE =1 # For RT_LumaMovement. 0=RT_LumaDifference, 1=RT_LumaSceneChange, Metrics will change with Mode.
BLKW =32 # Metrics will change with block size. For RT_LumaMovement().
BLKH =32 #
THRESH =2.25 # Below KEEP. Use ShowMetrics, need check against source. About 3.0 for Mode 1.

DENOISE=0 # DegrainMedian LimitY, 0 = OFF, higher stronger, Suggest 4 -> 8
DN_MODE=0 # DegrainMedian mode, see DegrainMedian docs, suggest 0 -> 2, higher weaker
SHOW=True # Frame Info
SHOWTIME=True # Show original Time on output
ROBOLEVELS=False # AutoLevel, need differnt Thresh
PC=False # PC Levels for ROBOLEVELS

dclip = (ROBOLEVELS) ? RoboLevels(x=X,y=Y,W=W,h=H,pc=PC) : Last
# Optional Denoise, perhaps there are better ones, (OK for non-linear access).
dclip = (DENOISE>0) ? dclip.ConvertToYV12().DeGrainMedian(limitY=DENOISE,limitUV=0,mode=DN_MODE).GreyScale() : dclip

##########################################
# Choose 1 of below options:- (uncomment)
##########################################

# Show metrics to get best Threshold 'th'. Denoised clip shown in GrayScale.
ShowMetrics(dclip,THRESH,X,Y,W,H,fast=true,showTime=SHOWTIME,mode=MODE,blkw=BLKW,blkh=BLKH)

# Make Frames Cmd file for SelectFrames(), Must Play all way through.
#MakeFramesCmd(dclip,THRESH,X,Y,W,H,mode=MODE,blkw=BLKW,blkh=BLKH)

# Return Frames with EXISTING frames Command file (after MakeFramesCmd)
#SelectFrames(show=SHOW,showTime=SHOWTIME)

# Return NON-selected frames, for scrutiny, with EXISTING frames Command file (after MakeFramesCmd)
#RejectedFrames(show=SHOW,showTime=SHOWTIME)

# Make Frames command file and then extract frames automatically in 1 pass.
#AutoGetFrames(THRESH,X,Y,W,H,dclip=dclip,show=SHOW,showTime=SHOWTIME,mode=MODE,blkw=BLKW,blkh=BLKH)

AssumeFPS(250.0)
return Last
##########################################

Function RoboLevels(clip c,float "Strength",int "Samples",float "Ignore",Bool "QBC",Float "QBC_Thresh",int "X",int "Y",int "W",int "H", \
int "Matrix",bool "PC",bool "DEBUG",bool "Interlaced",bool "DoCrop",Bool "AutoGain") {
c
Frames=FrameCount
Strength=Float(Default(Strength,0.95))
Samples=(!Defined(Samples)) ? 24 : (Samples > Frames) ? Frames : (Samples < 1) ? 1 : Samples
Ignore=Float(Default(Ignore,0.2))
QBC=Default(QBC,False)
QBC_Thresh=Float(Default(QBC_Thresh,-32.0))
QBCropXM = Default(X,0)
QBCropYM = Default(Y,0)
QBCropWM = Default(W,-0)
QBCropHM = Default(H,-0)
DoCrop = Default(DoCrop,True)
AutoGain = Default(AutoGain,false)
QBCropWM = (QBCropWM<=0) ? QBCropWM : Width -(QBCropXM+QBCropWM)
QBCropHM = (QBCropHM<=0) ? QBCropHM : Height-(QBCropYM+QBCropHM)
QBCropXP = QBCropXM+2
QBCropYP = QBCropYM+2
QBCropWP = QBCropWM-2
QBCropHP = QBCropHM-2
Matrix = Default(Matrix,(width <= 720) ? 2 : 3)
PC=Default(PC,False)
DEBUG=Default(DEBUG,False)
Interlaced=Default(Interlaced,true)
Assert(Strength>=0.0 && Strength <=1.0, "RoboLevels: 0.0 <= Strength <= 1.0")
GScript("""
if(QBC) {
QBCS=RT_QueryBorderCrop(samples=Samples,thresh=QBC_Thresh,debug=DEBUG,ignore=Ignore,laced=Interlaced,matrix=Matrix)
Eval(QBCS)
}
if(Strength>0.0||AutoGain) {
if(AutoGain) {
(DEBUG)?RT_Debug("AutoGain"):NOP
TMPW=QBCropWP/4*4 # Mod 4 for AutoGain
TMPH=QBCropHP/4*4 # Mod 4 for AutoGain
TMP=Crop(QBCropXP,QBCropYP,TMPW,TMPH)
AutoGain(Last,TMP)
} else {
QLMMS=RT_QueryLumaMinMax(samples=Samples,ignore=Ignore,debug=DEBUG,X=QBCropXP,Y=QBCropYP,W=QBCropWP,H=QBCropHP,matrix=Matrix)
(DEBUG)?RT_Debug(QLMMS):NOP
Eval(QLMMS)
if(IsRGB() || PC) {
CSMin = 0
CSMax = 255
} else {
CSMin = 16
CSMax = 235
}
ALMin = Int(CSMin - ((CSMin - QLMMMin) * Strength) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * Strength)) # Round down
(DEBUG)? RT_Debug("Levels("+String(ALMin)+",1.0,"+String(AlMax)+","+String(CSMin)+","+String(CSMax)+",Coring=False)"):NOP
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring
}
} else {
(DEBUG) ? RT_Debug("No Auto Levels") : NOP
}
if(QBC && DoCrop) {
RT_Debug("RoboLevels",String(QBCropXM),String(QBCropYM),String(QBCropWM),String(QBCropHM))
Crop(QBCropXM,QBCropYM,QBCropWM,QBCropHM)
}
""")
return Last
}



Function ShowMetrics(clip c,float th,int "x",int "y",int "w",int "h",bool "dim",bool "fast",bool "Showtime",int "Mode",int "BlkW",int "BlkH") {
# SUBTITLES,
# LINE 1, 1st number Frame number, 2nd Anchor frame number (current comparison frame), Remainder, delayed KEPT frames (avoid dups).
# LINE 2, Diff between Frame and Anchor, Action, KEPT Frame count, % of frames KEPT.
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) dim=Default(dim,true) Fast=Default(Fast,False)
ShowTime=Default(ShowTime,True) Mode=Default(Mode,0) BlkW=Default(BlkW,32) BlkH=Default(BlkH,32)
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
cropped=Crop(x,y,w,h)
(dim && (w!=width || h!=height)) ? levels(0,1.0,255,0,128).Overlay(cropped,x=X,y=Y,opacity=1.0) : Last
Global ShowMetrics_KEPT=0 # Count of kept frames
Global ShowMetrics_DEL7=0 Global ShowMetrics_DEL6=0 Global ShowMetrics_DEL5=0 Global ShowMetrics_DEL4=0
Global ShowMetrics_DEL3=0 Global ShowMetrics_DEL2=0 Global ShowMetrics_DEL1=0 Global ShowMetrics_DEL0=0
ScriptClip("""
d = RT_LumaMovement(Last,Last,n2=ShowMetrics_DEL7,x=x,y=y,w=w,h=h,mode=Mode,blkw=BlkW,blkh=BlkH)
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th)
Global ShowMetrics_KEPT = (keep) ? ShowMetrics_KEPT + 1 : ShowMetrics_KEPT
SS=String(current_frame) + "]{" + \
String(ShowMetrics_DEL7) + "}("+ String(ShowMetrics_DEL6) + "," + \
String(ShowMetrics_DEL5) + "," + String(ShowMetrics_DEL4) + "," + \
String(ShowMetrics_DEL3) + "," + String(ShowMetrics_DEL2) + "," + \
String(ShowMetrics_DEL1) + "," + String(ShowMetrics_DEL0) + ")\n" + \
"Diff="+String(d,"%-6.2f ") + ((keep)?"KEEP":"DROP") + \
" KEPT=" + String(ShowMetrics_KEPT) + " " + String(ShowMetrics_KEPT*100.0 / (current_frame+1.0),"%03.2f") + "%\n" + \
"Thresh="+String(th,"%.2f"+" Mode="+String(Mode)+" BlkW="+String(BlkW)+" BlkH="+String(BlkH))
Global ShowMetrics_DEL7=(keep)?ShowMetrics_DEL6:ShowMetrics_DEL7 # Follow but NOT too closely (delayed by 7)
Global ShowMetrics_DEL6=(keep)?ShowMetrics_DEL5:ShowMetrics_DEL6
Global ShowMetrics_DEL5=(keep)?ShowMetrics_DEL4:ShowMetrics_DEL5
Global ShowMetrics_DEL4=(keep)?ShowMetrics_DEL3:ShowMetrics_DEL4
Global ShowMetrics_DEL3=(keep)?ShowMetrics_DEL2:ShowMetrics_DEL3
Global ShowMetrics_DEL2=(keep)?ShowMetrics_DEL1:ShowMetrics_DEL2
Global ShowMetrics_DEL1=(keep)?ShowMetrics_DEL0:ShowMetrics_DEL1
Global ShowMetrics_DEL0=(keep)?current_frame:ShowMetrics_DEL0
Subtitle(SS,lsp=0,font="Courier New",size=18)
""",After_Frame=True,args="th,x,y,w,h,Mode,BlkW,BlkH") # Needs Grunt for args
(ShowTime) ? ShowTime() : NOP
return ((Fast)?AssumeFPS(250.0):Last) .KillAudio()
}

Function MakeFramesCmd(clip c,float th,int "x",int "y",int "w",int "h",String "FileName",int "Mode",int "BlkW",int "BlkH") {
c
x=default(x,0) y=default(y,0) w=default(w,0) h=default(h,0) FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Mode=Default(Mode,0) BlkW=Default(BlkW,32) BlkH=Default(BlkH,32)
(Exist(FileName)) ? RT_FileDelete(FileName) : NOP # Delete existing
x = (x / 4) * 4
y = (y / 4) * 4
w = (w<=0) ? width() - X + W : w
h = (h<=0) ? height() - Y + H : h
w = (w / 4) * 4 h = (h / 4) * 4 # Mod 4 both dimensions
Global MakeFramesCmd_DEL7=0 # Frame to start comparing with (Anchor)
Global MakeFramesCmd_DEL6=0 Global MakeFramesCmd_DEL5=0 Global MakeFramesCmd_DEL4=0 Global MakeFramesCmd_DEL3=0
Global MakeFramesCmd_DEL2=0 Global MakeFramesCmd_DEL1=0 Global MakeFramesCmd_DEL0=0
ScriptClip("""
# We DONT compare with previous frame, we compare with a previous KEPT frame to avoid slow (and sneaky) burglars.
d = RT_LumaMovement(Last,Last,n2=MakeFramesCmd_DEL7,x=x,y=y,w=w,h=h,mode=Mode,blkw=BlkW,blkh=BlkH)
keep=(current_frame==0 || current_frame==FrameCount()-1 || d>=th) # Keep BOTH FIRST and LAST frame ALWAYS
Global MakeFramesCmd_DEL7=(keep)?MakeFramesCmd_DEL6:MakeFramesCmd_DEL7
Global MakeFramesCmd_DEL6=(keep)?MakeFramesCmd_DEL5:MakeFramesCmd_DEL6
Global MakeFramesCmd_DEL5=(keep)?MakeFramesCmd_DEL4:MakeFramesCmd_DEL5
Global MakeFramesCmd_DEL4=(keep)?MakeFramesCmd_DEL3:MakeFramesCmd_DEL4
Global MakeFramesCmd_DEL3=(keep)?MakeFramesCmd_DEL2:MakeFramesCmd_DEL3
Global MakeFramesCmd_DEL2=(keep)?MakeFramesCmd_DEL1:MakeFramesCmd_DEL2
Global MakeFramesCmd_DEL1=(keep)?MakeFramesCmd_DEL0:MakeFramesCmd_DEL1
Global MakeFramesCmd_DEL0=(keep)?current_frame:MakeFramesCmd_DEL0
WriteFileIf(FileName, "(keep)", "current_frame", append=True)
""",After_Frame=True,args="th,x,y,w,h,FileName,Mode,BlkW,BlkH") # Needs Grunt for args
return AssumeFPS(250.0).KillAudio()
}


Function SelectFrames(clip c,String "FileName",bool "Show",bool "Showtime") {
c
FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Show=Default(Show,False)
ShowTime=Default(ShowTime,False)
(ShowTime) ? ShowTime() : NOP
FrameSelect(CMD=FileName,show=Show) # Returns NO audio
}


Function RejectedFrames(clip c,String "FileName",bool "Show",bool "Showtime") {
c
FileName=Default(FileName,"FrameSelect_Cmd.Txt")
Show=Default(Show,False)
ShowTime=Default(ShowTime,False)
(Showtime) ? ShowTime() : NOP
FrameSelect(CMD=FileName,show=Show,reject=True) # Returns NO audio
}


Function AutoGetFrames(clip c,float th,int "x",int "y",int "w",int "h",String "FileName",clip "dclip",bool "Show",bool "Showtime", \
int "Mode",int "BlkW",int "BlkH") {
Default(dclip,c) # Use denoised clip for scanning
Mode=Default(Mode,0) BlkW=Default(BlkW,32) BlkH=Default(BlkH,32)
MakeFramesCmd(th,x,y,w,h,FileName,mode=MODE,blkw=BLKW,blkh=BLKH) # Make the command file for FrameSelect()
GSCript("""
For(i=0,framecount-1) {
RT_YankChain(n=i) # Force Process, MakeFrames
}
c.SelectFrames(FileName,Show=Show,showtime=Showtime) # Extract source frames and return to client
""")
}


Works really well. :)

One clip about 11:00 PM, cut down to about 10% of frames, where about 25 vehicles, 3 cyclists, 2 pedestrians and one fox caused
motion detection. Original clip about 18 mins. Dark, rain shimmering under street lamps.

2nd clip, about 02:00 AM cut down to about 3.5% of frames, where about 9 vehicles caused motion detection.
Original clip about 18 mins. Dark, little activity.

[Forensic]

Stainless. I look forward to testing your motion extraction against DeDup, but I can't locate the download link for the current version.

[StainlessS]

5th link from bottom in sig or here:-
http://www.mediafire.com/StainlessS

however, it's not for removing duplicate frames, its for extraction of movement in eg security camera clips,
eg only having to watch the interesting bits where the burglar is paying your building site a visit.

Or for those of you that like watching the nightlife of the local fauna.

[StainlessS]

@Forensic
It would be nice if you could post findings in comparison with dedup, it did not really occur
to me to try dedup for this purpose.

EDIT: By the way, the delays in the script are intended (in part) to keep duplicate frames in areas of movement,
it perturbs me to see movement in the rejected frames.
I cant really do a good test during daylight hours as outside my window is like a ruddy motorway during the day.

[Forensic]

I can't get past a "FrameSelect: Cannot Open Cmd File FrameSelect_Cmd.Txt" error. So I uploaded my test file to (https://www.yousendit.com/download/WFJWd0VFQXAyWGR4Tk1UQw). The only motion in that video, other than minor real-world camera fluctuations, occur at frames 3390 and 3804.

Three passes with DeDup did a perfect job. I simply nest each resulting output pass into the next input so only the end results gets saved.

[StainlessS]

You need to use MakeFramesCmd to create the text file before the other functions, except
ShowMetrics() which does not need/use frame command file, or AutoGetFrames which makes it's own frame command file.

[StainlessS]

By the way, an https account seems to be private to you, ie cannot access it.

[Forensic]

@StainlessS. I marvel at your programming creations and RT_Stats is quickly becoming a "must have" tool among my fellow Forensic Video Analysts. However, I think I will stick with my DeDup solution to isolate motion frames. Thank you though. As for the test video, it is not a private link but it does require a secure socket connection. Try the unsecure alternative http://www.yousendit.com/download/WFJWd0VFQXAyWGR4Tk1UQw

[StainlessS]

Thankyou, I did try that but presume that failure was due to my crap connection, been having a terrible connection since about November.
Constantly disconnected, Three.co.uk SuperFast Speed Mobile Broadband, 2kb-4kb/s during the day is a regular occurence,
good job I dont ffffing swear much (my neighbour thinks I've got Tourette's).

[StainlessS]

For your clip, I think mode 1 is overkill (over sensitive), need mode 0 with a thresh of maybe ~24, cuts down to 29 frames.
Need to avoid them there flowers, waving around in the wind.

Can you post your dedup script for that clip (bit of a phoney clip by the way).

EDIT: Perhaps mode 1 is better for low light captures (lots of fluctuating dark noise), Mode 0 for daylight.
I've only really tested much with night caps.

[Forensic]

th=1.0 # I do not recall the exact value that I used
# uncomment each successive line per "video analysis pass"
DupMC(log="1.dup")
#DeDup(threshold=th,maxcopies=19,maxdrops=18,log="1.dup").DupMC(log="2.dup")
#DeDup(threshold=th,maxcopies=19,maxdrops=18,log="1.dup").DeDup(threshold=th,maxcopies=19,maxdrops=18,log="2.dup").DupMC(log="3.dup")
#DeDup(threshold=th,maxcopies=19,maxdrops=18,log="1.dup").DeDup(threshold=th,maxcopies=19,maxdrops=18,log="2.dup").DeDup(threshold=th,maxcopies=19,maxdrops=18,log="3.dup")

I am sure that your method is faster, but this works for the rarity that I need such a feature.

[Forensic]

#StainlessS
Is there any way to use RT_Stats to return the estimated resulting compressed file size of the video file and MPEG CoDec name passed to it (without actually saving the file)? If so, I will create a script to automatically test dozens of denoising/degraining filter combinations to determine which produces the clearest result, and thus the smallest file size. When dealing with each surveillance video, the best filter combination is rarely the one that I last used.

This need comes up weekly for me and thousands of times for my industry. Currently we have to encode using VirtualDub (2.5) to view the estimated resulting file size, or let it finish writing the file. This becomes quite time consuming and I am all about automation.

I do not mind, and actually prefer, if the solution runs through the entire encoding process (without writing anything to the hard drive) so it could return the most accurate final file size. If this ability is/can not be in RT-Stats, maybe it can be a portion of the solution. As an added bonus, such a feature would also allow the user to confirm that they have ample hard drive space before committing a save.

[StainlessS]

Sorry, I dont have a clue how to do that.

[StainlessS]

Forensic
You might be able to knock up something in AutoIt script (quite similar to basic), to start VD (or whatever), run encode and after maybe 5 mins, read
estimated output size, then abort encode and try again. You could call the compiled autoit exe from Avisynth script
and read data written by autoit exe into some file.
http://www.autoitscript.com/site/autoit/
Also get the editor (Scite4AutoIt or something like that, linked in downloads, better than the basic editor + tools).

You might find MegUIAutoEncode and DGIndex Batchers source useful especially the utility functions. (Mediafire in sig)

AutoIt has an unbelievable number of libraries to do all sorts of things including eg Post OS SysPrep setup stuff
to set Windows Defaults, install Software, configure registry etc.

EDIT: The DGIndex Batcher, waits for DGIndex to show the "FINISHED" text in info box so that it knows
that it has completed it's operation, similar could be used with VD to get estimated file size.

[Forensic]

I had never heard of autoit. It looks like fun to learn and has tremendous tutorial resources, including this website (Doom9 is amazing). I have also never worked with "DGIndex" or "MeGUI AutoEncode Batcher", the later of which was written by you (no surprise there). I can't fathom how you can maintain a paying career or personal life (let alone ever sleep) and still contribute as much as you do to help others. Your programming contributions have actually saved lives, including RT-Stats role in enhancing images of the Boston Marathon bombers when expediency mattered most. A debt of appreciation we can never repay. Thank you StainlessS.

[StainlessS]

DGIndex Batcher will probably be the better template to use as a starting point rather than MeGUI Auto Encode Batcher.
If you've programmed in BASIC before, it should be reasonably easy to pick up, took me about 1.5 days to feel relatively comfortable.
Not sure, I think I may have seen Selur post some AutoIt stuff.

Thank you for your nice words, glad to see it being of use.

EDIT: As far as coding is concerned, I normally (broke two fingers last night, I think, was a bit alcohol laden, typing has suffered) am a pretty
fast coder. When I was coding C and Assembler full time (until about 1996) I was convinced that I could think in C (sounds daft but true).
EDIT: Wish I could think in Machine Code, that would be real fun, dreaming in M/C. :eek:

[martin53]

StainlessS,
would you mind to spend some thoughts about a potential use case of the built in and RT_Stats statistic functions?

I currently check if it would be possible to make an automatic white balance script.
I think, although I don't yet have proof of it, that the first task is to find areas in the image that show white/grey objects, skin or maybe sky. Then their U/V values should be measured against a standard value and the bias reduced.
For the first task, I made a line of AviSynch code that selects the parts of the frame that are inside of a certain Y/U/V range and deliver the U-V bias (c is the clip)
ymin = "150"
ymax = "240"
uMin = "120"
uMax = "135"
vMin = "120"
vMax = "135"
c = c.ConvertToYV24()
# ( yMin<Y<yMax && uMin<U<uMax && vMin<V<vMax ) ? ( U - V + 128 ) : ( 0 )
mt_lutxyz(c, c.UToY(), c.VToY(), expr="x "+ymin+" > x "+ymax+" < & y "+uMin+" > & y "+uMax+" < & z "+vMin+" > & z "+vMax+" < & y z - 128 + 0 ?", chroma="128")
That function delivers grey values " U - V + 128 ", i.e. something around 128, and if the average is above 128, then I expect the frame to be too blue and vice versa.

The point is now, that it is difficult (ahem, almost impossible) to retrieve the statistics, because most of the frame is black and shall be left unconsidered. But no statistic function supports a mask, all evaluate the whole frame (x,y,w,h not of interest here).

The one solution I know of is: use RT_YInRange(lo=0,hi=0) to get the amount of black, and divide the RT_AverageLuma by (1.0 - YInRange) - no surprise since I asked you for the RT_YInRange() function for a similar problem. It works because I can be sure that black is outside the useful luma range and because it does not contribute to the average. Similar tweaks can be used to get other statistical numbers, e.g. return 255 for the invalid pixels when RT_YPlaneMin() is needed. But that needs repeated calls of the mt_lutxyz() function and slows down things.

Currently I feel it's obvious that all statistic functions can profit from an additional mask clip. With that option, I would probably prepare a clip "clip UV = c.U - C.V + 128" (with mt_lutxy() ), make the mt_lutxyz() function return the mask instead of U-V; and then query min,max,average and all that from the UV clip.

All comments appreciated!

[StainlessS]

Martin53, do I have this right,

You want funcs where only pixels taken into account are those where same coords in mask clip are non zero ?

EDIT: Or if not non zero, what ? I've actually just done the plug but as yet no testing whatever. Only supports Planar.
As for other RT stats Y funcs, n,delta,x,y,w,h etc.

EDIT: At the moment, I've got it returning -1 for all funcs except YStats equivalent where it returns flgs=0,
when there are no pixels to return anything for.

I could implement a range of mask values to process pixels for eg MaskMin, MaskMax inclusive, a single variable value, or a fixed
value eg 0.

What you want ?

[martin53]

I could implement a range of mask values to process pixels for eg MaskMin, MaskMax inclusive, a single variable value, or a fixed
value eg 0.

What you want ?

Hi StainlessS,
makes me happy that you like the idea and even work already on an implementation!

... I did not think all alternatives to the end so I'm also unsure what would be the best way to evaluate the mask for the stats functions. First, I thought that a multiplication with (mask luma value / 255) would be the most intuitive implementation. But I can imagine no real useful application for a weighted average, maximum etc. One could 'dim' the mask edges and give more weight on the center, for example. But what purpose would that be useful for?

Your proposal MaskMin MaskMax sounds much more useful: one could prepare a mask with several discrete values, and then re-use the same mask for a number of statistics queries. E.g. a mask value stands for 'about white', another one for 'sky blue' or 'skin'. That would support a much faster approach than repeated mt_lutxyz() mask creations.

Maybe there is someone else besides us who might want to share his opinion? If not, I think I favour MaskMin, MaskMax.

EDIT: only pixels taken into account where same coords in mask clip are valid. Yes, exactly.

EDIT: and I don't think it is useful to use chroma of the mask with chroma stats. Better always use luma, so luma and chroma query take the same pixels into account and no repeated mask preparation is needed. It's always possible to use UtoY() when someone really needs U of a clip as the mask.

[StainlessS]

OK, I'll proceed and give it a little testing and hand over to you.

Currently with names eg YStatsM, etc.
EDIT: Changed to MYStats

[StainlessS]

Martin53, Here, v2.6 dll + source
http://www.mediafire.com/download/cw93p3rqxp6g5t6/MyStats.zip

Script I used to create the maskclip

c=Avisource("D:\avs\test2.avi")

ymin = "150"
ymax = "240"
uMin = "120"
uMax = "135"
vMin = "120"
vMax = "135"

c = c.ConvertToYV24()
mt_lutxyz(c, c.UToY(), c.VToY(), expr="x "+ymin+" > x "+ymax+" < & y "+uMin+" > & y "+uMax+" < & z "+vMin+" > & z "+vMax+ \
" < & y z - 128 + 0 ?", chroma="128")
Return ConvertToYV12()


A basic test clip, not much other testing done

c=Avisource("D:\avs\test2.avi")
Mask=Avisource("D:\avs\Mask.avi")

LO=128
HI=LO+24
THRESH=0.0
PREFIX="MYS_"

c.ScriptClip("""
v = MYPlaneMin(Mask,threshold=THRESH)
S=(V<0) ? "NONE" : \
"\nMin="+String(v) + \
"\nMax="+String(MYPlaneMax(Mask,threshold=THRESH)) + \
"\nMnMxDiff="+String(MYPlaneMinMaxDifference(Mask,threshold=THRESH)) + \
"\nMed="+String(MYPlaneMedian(Mask)) + \
"\nAve="+String(MAverageLuma(Mask),"%.2f") + \
"\nStdev="+String(MYPlaneStdev(Mask),"%.2f") + \
"\nInRng="+String(MYInRange(Mask,lo=LO,hi=HI),"%.2f")
flgs=MYStats(Mask,threshold=THRESH,lo=LO,hi=HI,flgs=$7F,prefix=PREFIX)
S2="\nflgs="+String(flgs)
S3="\nMin="+(RT_BitTst(flgs,0) ? String(MYS_yMin) : " ---") + \
"\nMax="+(RT_BitTst(flgs,1) ? String(MYS_yMax) : " ---") + \
"\nMnMxDiff="+(RT_BitTst(flgs,2) ? String(MYS_yMinMaxDiff) : " ---") + \
"\nMed="+(RT_BitTst(flgs,3) ? String(MYS_yMed) : " ---") + \
"\nAve="+(RT_BitTst(flgs,4) ? String(MYS_yAve,"%.2f") : " ---") + \
"\nStdev="+(RT_BitTst(flgs,5) ? String(MYS_yStdev,"%.2f") : " ---") + \
"\nInRng="+(RT_BitTst(flgs,6) ? String(MYS_yInRng,"%.2f") : " ---")
Subtitle(String(current_frame)+" ]" + S+S2+S3,lsp=0)
return Last
""")

return StackHorizontal(Mask).convertToRGB32()


Watch out for using Subtitle over YV24, I think it uses OverLay and hence the YV24 Overlay bug will exhibit itself.

EDIT: Args (default Prefix="MYS_")


"MYPlaneMin", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[MaskMin]i[MaskMax]i"
"MYPlaneMax", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[MaskMin]i[MaskMax]i"
"MYPlaneMinMaxDifference", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[MaskMin]i[MaskMax]i"
"MYPlaneMedian", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[MaskMin]i[MaskMax]i"
"MAverageLuma", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[MaskMin]i[MaskMax]i"
"MYPlaneStdev", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[MaskMin]i[MaskMax]i"
"MYInRange", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[lo]i[hi]i[MaskMin]i[MaskMax]i"
"MYStats", "cc[n]i[delta]i[x]i[y]i[w]i[h]i[Interlaced]b[threshold]f[lo]i[hi]i[flgs]i[prefix]s[MaskMin]i[MaskMax]i"



EDIT: Oops, MaskMin default = 128, MaskMax default = 255.

EDIT: I could also implement for both YUY2 and RGB/24/32 but with the Mask being Planar Only, wadya think.
(in which case the Matrix arg will be inserted at positions similar to original funcs).

[martin53]

StainlessS,
thank you!
I am going to test it with my footage and adapted mt_... clips.
Can't guarantee a feedback to you and the doom9 community tomorrow since I'm refurbishing the living room...

[StainlessS]

RT_Stats v1.18, New version.

v1.18, Fixed RT_QueryBorderCrop RGB "Matrix NOT @ PC levels - Check!" bug.
Added RT_ColorSpaceXMod/YMod. Added RT_RgbChan family.
Added v2.6 dll

Stuff added:

RT_ColorSpaceXMod(clip)
Return int, the natural cropping XMod for clip colorspace, eg YV411=4, YV12=2, YUY2=2, RGB=1
v2.5 plugin limited to v2.58 colorspaces.

RT_ColorSpaceYMod(clip,bool "Laced"=true)
Return int, the natural cropping YMod for clip colorspace, eg YV411=1, YV12=2, YUY2=1, RGB=1
Laced, bool. Default true. If Laced==true, returns doubled YMod.
v2.5 plugin limited to v2.58 colorspaces.

RT_RgbChanMin(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0)
Returns int minimum value (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA]). Threshold is a percentage, stating how many percent of the pixels are allowed below minimum.
The threshold is optional and defaults to 0.0


***
***
***

RT_RgbChanMax(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0)
Returns int maximum value (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA]). Threshold is a percentage, stating how many percent of the pixels are allowed below minimum.
The threshold is optional and defaults to 0.0

***
***
***

RT_RgbChanMinMaxDifference(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
bool "interlaced"=false,float "threshold"=0.0,int "chan"=0)
Returns int value range maximum - minimum difference (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA]). Threshold is a percentage, stating how many percent of the pixels are allowed below minimum.
The threshold is optional and defaults to 0.0

***
***
***

RT_RgbChanMedian(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "chan"=0)
Return the int channel median (0 -> 255) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA])

***
***
***

RT_RgbChanAve(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0, int "h"=0,bool "interlaced"=false,int "chan"=0)
Return the float average value (0.0 -> 255.0) for an RGB channel in frame(n+delta) for area x,y,w,h. (RGB 24/32).
(chan default = 0 = R [RGBA])

***
***
***

RT_RgbChanStdev(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "chan"=0)
Returns FLOAT value Standard Deviation (0.0 -> 255.0) for an RGB channel in frame(n+delta) for area x,y,w,h.
Sample standard deviation http://en.wikipedia.org/wiki/Standard_deviation
(chan default = 0 = R [RGBA])


***
***
***

RT_RgbChanInRange(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,int "chan"=0,
int "lo"=128,int "hi"=lo)
Returns float value (0.0 -> 1.0) being the amount of pixels with a channel value in the range "lo" to "hi" (inclusive), 1.0 is
equivalent to 100%. NOTE, differs from other funcs that return range 0.0 to 255.0. NOTE, lo defaults to 128, "hi" defaults to "lo".
(chan default = 0 = R [RGBA])

***
***
***

RT_RgbChanStats(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,bool "interlaced"=false,
float "threshold"=0.0,int "chan"=0,int "lo"=128,int "hi"=lo,int "flgs"=127,string "prefix"="RCS_")
Returns multiple results as for above single frame RGB sampling functions as Global Variables (prefixed with the prefix string arg).
The args up to "interlaced", and "chan", are as for all other RGB clip functions, "threshold" used only for "RT_RgbChanMin",
"RT_RgbChanMax" and "RT_RgbChanMinMaxDifference" equivalent routines with same functionality.
"lo" and "hi" are used only with the "RT_RgbChanInRange" equivalent routine with same functionality.
The only new args are int "Flgs" which selects which results you want returned and the string "Prefix" that is prepended
to the returned global variable names.
The actual return result is a copy of the flgs args with any non valid bits reset, ie the global variables that were set.
Global variables are NOT altered for any function not selected in flgs.

Flgs_Bit_Number Add_To_Flgs Equivalent_Function Global_Var_Set_Excluding_Prefix(Where 'x' is channel number)
0 1($01) RT_RgbChanMin() "Min_x" (0->255)
1 2($02) RT_RgbChanMax() "Max_x" (0->255)
2 4($04) RT_RgbChanMinMaxDifference() "MinMaxDiff_x" (0->255)
3 8($08) RT_RgbChanMedian() "Med_x" (0->255)
4 16($10) RT_RgbChanAve() "Ave_x" (0.0->255.0)
5 32($20) RT_RgbChanStdev() "Stdev_x" (0.0->255.0)
6 64($40) RT_RgbChanInRange() "InRng_x" (0.0->1.0)
RT_RgbChanStats() allows you to inquire multiple results simultaneously, with not much more overhead than calling a single individual
routine, however, you should not select sub functions that you dont need as there may be an additional unnecessary overhead.
The Default flgs=127($7F) are all bits set and so sets ALL global vars at once.
RT_RgbChanStats(chan=0,flgs=1+2+16) would set global vars "RCS_Min_0", "RCS_Max_0" and "RCS_Ave_0" for full frame current_frame.



See 1st post.

[StainlessS]

New version RT_Stats v1.19, see 1st post.


v1.19, RT_LumaSceneChange() moved cap @ 255.0 to just before return instead of on Lut, better metrics.
Originally like:- mt_lutxy(mt_polish("((x-y)/4)^2")).AverageLuma() where was capped @ 255 @ equivalent mt_lutxy step.
Changed RT_QueryBorderCrop() Thresh Massaging.

[Forensic]

How can I use RT-Stats to return the 6 character RGB (or 8 character RGBa) value of one user specified pixel on one specified frame in RGB-24 (or RGB-32) color space?
EDIT: RT_RgbChanMedian did the trick one channel at a time. :)

[StainlessS]

Forensic, glad you got it sorted.
Min, max, median and average should all return identical results for a single pixel.

EDIT: this should also be able to do it in a single call although for RGB32 could return a -ve int.

RT_RGB32AsInt(clip,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0)
Compile/Runtime clip function.
Given an RGB32 clip that had a pixel value created from an Int, gets that pixel and returns as the
original Int. By creating single pixel clips and stacking them horizontally/vertically,
you can use a frame as a two dimensional array of Int, perhaps use the frame number for a third dimension.


EDIT

6 character RGB (or 8 character RGBa) suggests you want Hex, this would convert even -ve
int to hex

RT_Hex(int , width=8)

Or

RT_NumberString(int ,base=16, int width=8)

v2.58 has bug in Hex function for -ve numbers.

[StainlessS]

RT_Stats v1.20, new version.

Changed the way RT_LumaSceneChange works. Added Bias->Pord args. Added RT_Subtitle(), Added RT_LumaSceneChange_Graffer with vb source.

Mod:

RT_LumaSceneChange(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3), \
Float "Bias"=0.0,Float "Gain"=1.0,Float "Cont"=1.0,Float "Rpow"=1.0,Float "Spow"=1.0,Float "SPMid"=0.5,Bool "Pord"=false)

Returns FLOAT value (0.0 -> 255.0) scene change detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
v1.20, Modified how it works. Creates a look up table (lut) which is used for the pixel differences between same pixel in frames
(n+delta) and (n+delta2). The lut is created using args "Bias" through to "Pord" and a sort of levels type internal function
creates the lut. An absolute pixel difference is taken between corresponding pixels and then the contents of the lut for that difference is
added to the sum of differences, and finally the average is taken. Is sort of like mt_lutxy("x y - abs").Lut().AverageLuma()
Additional Args:
Bias, Default 0.0, (-512 -> 512). Equivalent to Bias in RGBAdjust(), ie brightness.
Gain, Default 1.0, (-8.0 -> 8.0). Equivalent to Gain in RGBAdjust(), varies slope.
Cont, Default 1.0, (-8.0 -> 8.0). Contrast. A bit like Cont in ColorYUV.
RPow, Default 1.0, (0.1 -> 4.0). Similar to Gamma.
SPow, Default 1.0, (0.1 -> 4.0). S shaped power curve.
SPMid,Default 0.5, (0.01 -> 0.99). Mid point control for SPow.
Pord, Default False (False -> True). Governs which power function is applied first. False (default) applies RPow 1st,
True Applies SPow 1st. Only applies if both power functions not at default 1.0.
All Defaults for lut creation are linear and will be equivalent to RT_LumaDifference(). As currently implemented, may provide
alternative functionality other than scene change detection, but no idea what those applications could be.
A graphing program with VB source is provided in the zip, so you can play with sliders to choose function args.
Func versions previous to v1.20, used args similar to:- gain=4.0,rpow=0.5 with remaining args at default.
The old default, gave less prominence to big differences so that a few pixels with massive difference were limited in
the overall effect on results. New args produce a linear LUT by default.

Scene change metric between current frame and next frame, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,delta2=1,h=-32)
and exactly the same thing again:
SC = RT_LumaSceneChange(Last,Last,n2=current_frame+1,h=-32)
There are two arg clips for flexibility, might usually both be same, but might want to measure against two separate clips and so
implemented as above.
Scene change detection between current frame top field and bottom field, whilst ignoring 32 pixels of crud at bottom:-
SC = RT_LumaSceneChange(Last,Last,y2=1,interlaced=true,h=-32) # Expect higher metrics here.


***
***
***

RT_LumaMovement(clip,clip2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,
int "n2"=current_frame,int "delta2"=0,int "x2"=x,int "y2"=y,bool "interlaced"=false,int "Matrix"=(width<=720?2:3),
int "Mode"=0,int "BlkW"=64,int "BlkH"=64 \
Float "Bias"=0.0,Float "Gain"=1.0,Float "Cont"=1.0,Float "Rpow"=1.0,Float "Spow"=1.0,Float "SPMid",Bool "Pord"=false)
Returns FLOAT value (0.0 -> 255.0) movement detection between clip frame (n+delta) area x,y,w,h, and clip2 frame (n2+delta2) area x2,y2,w,h.
Note, 'x2' and 'y2' default to 'x' and 'y' respectively.
There are two modes available, 0 (Default) = RT_LumaDifference() : 1 = RT_LumaSceneChange().
BLKW/BLKH (default 64), Block size. Minimum 8 (silently limited to frame dimensions), Must be EVEN.
RT_LumaMovement() splits the frames into block sized lumps and calls the desired mode RT_LumaDifference or RT_LumaSceneChange
function to get a metric for each of those blocks. The metric returned is the metric with the highest value, and so is more
sensitive to localized movement. Best control of localized movement sensitivity is by changing blk size, lower is more sensitive.
If Interlaced is true, then only every other horizontal scanline in BLKH will be scanned.
v1.20, See RT_LumaSceneChange() for additional args when Mode=1 (ignored when Mode = 0).


Added

RT_Subtitle(clip source, string format, dat1,...,datn,int "align",int "x","y",bool "vcent"=false,bool "expx"=false,bool "expy"=false )
Standard filter function.
This function is a standard filter graph function, not a runtime/compile time function.
The v2.5 plugin limited to v2.58 colorspaces.
Prints formatted text to clip frame using Avisynth Info.h source font and DDigit font renderer, faster than using Subtitle
system fonts. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the text string that is
printed on frame, uses C/CPP printf() style formatting. The args 'align', 'x', and 'y', work in a similar fashion to
Avisynth Subtitle() eg
RT_Subtitle(c,"Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013,align=4,x=100,vcent=true)
would print

Hello there Fred and Ted.
Goodbye 2013.

at x pixel position 100 and y pixel position centered, the vcent=true will vertical center a block of text (align=4 or 5 or 6),
whereas Subtitle() starts at centred position and prints downwards from there (as default vcent=false).

Align, default = 7 ie top left (as on numeric keypad).
x and y allow modification of alignment, eg x= -1, centers x at mid point, x=width, would right align,
y=height would align at bottom of frame.
It is necessary to specify the arg names of x,y,align and vcent, as the dat args can take 0 or more variables
of any type, so Avisynth cannot tell when the list of dat args ends and the named optional args begins. I did
try to use the dat args last but Avisynth seems to get confused when you use dat= etc, it throws an error
saying something like "dat arg specified more than once", even if it was not.
printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned.
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

eg
FMT="%d ] \a%cAverageLuma=\a-%6.2f"
ALPHA="0123456789ABCDEFGHIJKLMNOPQRSTUV-!"
ScriptClip("""
RT_Subtitle(FMT,current_frame,RT_Ord(ALPHA,current_frame%16+1),AverageLuma(),align=5)
""")

In the above example you see the sequence "\a%c" where %c is replaced with a character from ALPHA, "\a0"
selects color 0, "\aV" selects the last color (31). "\a-" selects the default color, and "\a!" selects
the hilite color. You can in Scriptclip() select colors programmatically, perhap selecting red to show an
error or bad metrics condition. The initially selected color will always be the default color (white).
Ripped from DDigit source code:
// Color control codes as strings. ASCII Code of Final char
#define DDIGIT_CC_HILITE "\a!" // 33
#define DDIGIT_CC_DEFAULT "\a-" // 45
#define DDIGIT_CC_DARKGRAY "\a0" // 48
#define DDIGIT_CC_DODGERBLUE "\a1" // 49
#define DDIGIT_CC_ORANGERED "\a2" // 50
#define DDIGIT_CC_ORCHID "\a3" // 51
#define DDIGIT_CC_LIME "\a4" // 52
#define DDIGIT_CC_AQUAMARINE "\a5" // 53
#define DDIGIT_CC_YELLOW "\a6" // 54
#define DDIGIT_CC_WHITE "\a7" // 55
#define DDIGIT_CC_SILVER "\a8" // 56
#define DDIGIT_CC_CORNFLOWERBLUE "\a9" // 57
#define DDIGIT_CC_ORANGE "\aA" // 65
#define DDIGIT_CC_PLUM "\aB" // 66
#define DDIGIT_CC_CHARTREUSE "\aC" // 67
#define DDIGIT_CC_POWDERBLUE "\aD" // 68
#define DDIGIT_CC_GOLD "\aE" // 69
#define DDIGIT_CC_GAINSBORO "\aF" // 70
#define DDIGIT_CC_Y_0 "\aG" // 71
#define DDIGIT_CC_Y_1 "\aH" // 72
#define DDIGIT_CC_Y_2 "\aI" // 73
#define DDIGIT_CC_Y_3 "\aJ" // 74
#define DDIGIT_CC_Y_4 "\aK" // 75
#define DDIGIT_CC_Y_5 "\aL" // 76
#define DDIGIT_CC_Y_6 "\aM" // 77
#define DDIGIT_CC_Y_7 "\aN" // 78
#define DDIGIT_CC_Y_8 "\aO" // 79
#define DDIGIT_CC_Y_9 "\aP" // 80
#define DDIGIT_CC_Y_A "\aQ" // 81
#define DDIGIT_CC_Y_B "\aR" // 82
#define DDIGIT_CC_Y_C "\aS" // 83
#define DDIGIT_CC_Y_D "\aT" // 84
#define DDIGIT_CC_Y_E "\aU" // 85
#define DDIGIT_CC_Y_F "\aV" // 86
In addition to color control codes, the filter will accept "\n" or "\r" as newline, and "\f" as forward
space (move 1 right without printing anything), no other control codes are supported. "\t" and TAB control
codes are converted to a single space, I could think of no sensible way to handle tab when center or
right aligned, if there is a demand, I could maybe handle tab for left alignment only and replace with single space
for other alignment.
To embed a single BackSlash ("\") into the format string or even an argument dat string you must insert a double
BackSlash ie "\\". To embed a single percent character ("%") into a format string, you must embed a double percent
sequence ie "%%". Reason is that such characters are used for formatting eg "%f" flags a float arg to embed into
the string and eg "\n" would mean insert newline into string.
Note, the colored text is not great in YV12 and quite awful in YV411, try to avoid YV411 at all times,
tis truly nasty.
To Print contents of multiline text file on frame:

AVISource("D:\AVS\TEST.AVI")
S=RT_ReadTxtFromFile("D:\avs\avi\test.txt")
RT_Subtitle("%s",S,align=5,vcent=True)

As can be seen from above example, you can also use eg Chr(10) instead of "\n" or Chr(13), or Chr(13)+Chr(10)
instead of "\r\n" with identical results, CR, NL and CR/NL pairs are internally replaced by a single Chr(13).
To Print contents of multiline text file on frame, Scrolling upwards as for end credits:
AVISource("D:\AVS\TEST.AVI")
S=RT_ReadTxtFromFile("D:\avs\avi\test.txt")
DELAY=100
Scriptclip("""
RT_Subtitle("%s",S,align=5,y=height+DELAY-current_frame,expy=true)
""")
The above example uses the "expy" arg, which tells the filter to NOT interpret y == -1 as screen vertical
center align and to NOT interpret y=height as bottom aligned. ExpX does the same for the x arg. If you change
above to ExpY=false, as the text scrolls on screen and again when text scrolls off top of frame, there will be
a single frame where y==height OR y==-1, will align the text an cause a glitch in the video clip.
You could also embed eg color control codes into any above such scrolling text file for your amusement, eg
embedding "\a!" into the text file would switch on Hi-Lite-ed text.
If you want you can run above script with this text file as "test.txt" and because it has color control codes
embedded into it (eg the ripped DDigit source code earlier) so it will show the file, some lines of which will be
colored. This text file has NOT been prepared for that particular reason but may demo the embedded color codes.
RT_Subtitle() has the advantage of not requiring string memory as the string is created internally and
released on plugin destruction, whereas SubTitle() within ScriptClip will keep accumulating string memory
which is not released until Avisynth closes. Of course any eg concatenated strings supplied to RT_Subtitle
will be accumulated unless eg assigned to string variable external to Scriptclip.
The RT_Subtitle filter is intended to be used in providing on-frame metrics in a script, and not for final
output, use SubTitle() for that.
For any script using standard Subtitle() for metrics, it would not be at all surprising if that script were
to double (or more) in speed when replaced by RT_Subtitle, it really is quite light weight in comparison.


See 1st post.

[StainlessS]

New function contained in RT_Stats v1.20 DynaCrop.avs, equivalent but more robust version of SCSelect()


Function SC_SceneSelect(clip dclip,clip start,clip end,clip motion,
\ float "dfact",float "minim",
\ Float "Bias",Float "Gain",float "Cont",Float "RPow",Float "SPow",Float "SPMid",bool "Pord",bool "Show") {
# Selects frames depending upon scene change status. Similar to SCSelect()
# Dclip, detection clip. Start, Frames after scene change. End, Frames before scene change. Motion, not scene change.
# Start, End and Motion must be same size and colorspace, Dclip can be other.
myName="SC_SceneSelect: "
df=Float(Default(dfact,4.0)) #
minim=Float(Default(minim,4.0)) # Scene change Diff has to be at least this, avoids eg scene change in near static scene.
Bias=Float(Default(Bias,0.0)) # Arg to RT_LumaSceneChange(), -511.0 -> 511.0, Default 0.0
Gain=Float(Default(Gain,1.0)) # Arg to RT_LumaSceneChange(), -8.0 -> 8.0, Default 1.0
Cont=Float(Default(Cont,1.0)) # Arg to RT_LumaSceneChange(), -8.0 -> 8.0, Default 1.0
RPow=Float(Default(RPow,1.0)) # Arg to RT_LumaSceneChange(), 0.1 -> 4.0, Default 1.0
SPow=Float(Default(SPow,1.0)) # Arg to RT_LumaSceneChange(), 0.1 -> 4.0, Default 1.0
SPMid=Float(Default(SPMid,0.5)) # Arg to RT_LumaSceneChange(), 0.01 -> 0.99, Default 0.5
Pord=Default(Pord,False) # Arg to RT_LumaSceneChange(), Default False
Show=Default(Show,False) # True = Show a little extra Info (Default False)
Assert(df>1.0,myName+"dfact MUST be greater than 1.0")
Assert(minim>0.0,myName+"minim MUST be greater than 0.0")
dclip=dclip.RoboCrop(WMod=4,HMod=4) # Dont allow common border to 'water down' metrics
Global SCM_B=0.0 Global SCM_C=0.0 Global SCM_D=0.0 Global SCM_E=0.0
Global SCM_SC=False Global TrimN=0 Global SCM_Prev=-2 # Init vars, SCM_Prev=-2 forces initalize
Start =(Show)? Start.SubTitle("START OF SCENE",align=1,size=30) : Start
End =(Show)? End.SubTitle("END OF SCENE",align=3,size=30) : End
Motion=(Show)? Motion.SubTitle("MOTION",align=5,size=30) : Motion
# Make format string only once, not at every frame
Fmt=(Show)? "%d ] Trim=%d {\a%c%.1s\a-:\a%c%.1s\a-:\a%c%.1s\a-} L=%.2f : R=%.2f\n" +
\ "%6.2f \a%c%6.2f %6.2f %6.2f\a- %6.2f\n %6.2f %6.2f %6.2f %6.2f\n \a%c%6.2f %6.2f %6.2f\a-\n" +
\ "DFact=%6.2f : Minim=%6.2f\nBias =%6.2f : Gain =%6.2f : Cont =%6.2f\nRPow =%6.2f : SPow =%6.2f : SPMid=%6.2f : Pord=%.1s":""
Motion.ScriptClip("""
NotNext = (current_frame!=SCM_Prev+1)
# If jumped about, get values for Previous frame, else keep same.
# Below Eval avoids extra calcs when scanning forward but is slower going backwards.
psc=(NotNext) ? Eval("
Global SCM_A=RT_LumaSceneChange(dclip,dclip,n=current_frame-3,n2=current_frame-2,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_B=RT_LumaSceneChange(dclip,dclip,n=current_frame-2,n2=current_frame-1,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_C=RT_LumaSceneChange(dclip,dclip,n=current_frame-1,n2=current_frame ,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_D=RT_LumaSceneChange(dclip,dclip,n=current_frame ,n2=current_frame+1,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
Global SCM_E=RT_LumaSceneChange(dclip,dclip,n=current_frame+1,n2=current_frame+2,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
AB=SCM_B-SCM_A BC=SCM_C-SCM_B CD=SCM_D-SCM_C DE=SCM_E-SCM_D # Diff of Diff
ABC=BC-AB BCD=CD-BC CDE=DE-CD # Diff Of Diff Of Diff
#
# BELOW MUST BE SAME AS MAIN CODE.
# Big enough compared to movement immediately either side of scene change
T1 = minim<SCM_C # Not static scene ?
T2 = ABC>0.0 && BCD<0.0 && CDE>0.0 # Scene Change sign OK ?
T3 =(df*SCM_B<SCM_C && df*SCM_D<SCM_C) # sufficiently bigger than either side ?
# ABOVE MUST BE SAME AS MAIN CODE.
#
return (T1 && T2 && T3)
") : SCM_SC
Global SCM_A=SCM_B Global SCM_B=SCM_C Global SCM_C=SCM_D Global SCM_D=SCM_E # Pass the parcel
Global SCM_E=RT_LumaSceneChange(dclip,dclip,n=current_frame+2,n2=current_frame+3,bias=BIAS,gain=GAIN,cont=Cont,
\ rpow=RPOW,spow=SPow,Spmid=SPMid,pord=Pord)
AB=SCM_B-SCM_A BC=SCM_C-SCM_B CD=SCM_D-SCM_C DE=SCM_E-SCM_D # Diff of Diff
ABC=BC-AB BCD=CD-BC CDE=DE-CD # Diff Of Diff Of Diff

# Big enough compared to movement immediately either side of scene change
T1 = minim<SCM_C # Not static scene ?
T2 = ABC>0.0 && BCD<0.0 && CDE>0.0 # Scene Change sign OK ?
T3 =(df*SCM_B<SCM_C && df*SCM_D<SCM_C) # sufficiently bigger than either side ?

psc=(psc||current_frame==0)
Global SCM_SC=((T1 && T2 && T3) || current_frame==FrameCount()-1) # Last frame in scene ?
Last = (psc&&!SCM_SC)?start:(SCM_SC&&!psc)?end:Last
CC=(T1&&T2&&T3)?33:45 # Hilite or Default color ?
LFACT=(SCM_C<=SCM_B || !T1 || !T2)?0.0:(SCM_B<0.000001)?999999.9:(SCM_C-0.000001)/SCM_B
RFACT=(SCM_C<=SCM_D || !T1 || !T2)?0.0:(SCM_D<0.000001)?999999.9:(SCM_C-0.000001)/SCM_D
(Show) ? RT_Subtitle(Fmt,current_frame,TrimN,CC,T1,CC,T2,CC,T3,LFACT,RFACT,SCM_A,CC,SCM_B,SCM_C,SCM_D,SCM_E,AB,BC,CD,DE,CC,ABC,BCD,CDE,
\ df,minim,bias,gain,cont,rpow,spow,spmid,pord) : NOP
Global TrimN = (SCM_SC) ? TrimN+1 : TrimN # Incr for next time
Global SCM_Prev=current_frame
Return Last
""",args="start,end,dclip,myName,Show,df,minim,Bias,Gain,Cont,RPow,SPow,SPMid,Pord,Fmt") # Needs Grunt for args
return Last
}


Client script just to show subtitled results

Import("Dynacrop.avs")

Avisource("D:\avs\test.avi")

DClip=Last
Start=Last
End=Last
Motion=Last

SC_SceneSelect(dclip,start,end,motion,show=true,dfact=4.0,minim=4.0,Bias=0.0,gain=1.0,cont=1.0,rpow=1.0,spow=1.0,SPMid=0.5,pord=false)

[Forensic]

Increasing a video's gamma or histogram profile can brighten a seemingly black nighttime surveillance video. However, with only 256 shades of YUV illumination, the results can fail to be forensically useful. It seems to me that a your RT-Stats tool in RGB space could provide a much better alternative. My idea is to select the upper left most pixel and read its RGB values along with those of the preceding and postceding "N" frames. Average those temporal values and then subtract that average from the current frame's R G and B pixel values. If I now multiply that difference by "M", and add it to the original RGB pixel values, the resulting RGB values will be an exaggeration of the color deviation from average at that one pixel. If I repeat this process for every pixel of every frame, I should end up with greater detail of any changes that are occurring, even if the scene appears almost completely dark. I realize that the resulting scene will look weird, and poison noise will become amplified, but the goal is to detect movement, amplify signage (logos, license plates, etc...), and other forensic stuff.

I tried this myself using "RgbChanMedian" to read each pixel and "Overlay" to write the new pixel data. Not only was the process painfully slow (several minutes per frame) but it also caused out of memory issues due to all the looping required (thus I now have no solution). Although my work is mostly with low quality surveillance video, such a feature would encourage new uses. For example, the exaggerated color shifts would allow anyone to see the normally imperceptible temperature changes as machinery heated up (and thus red incremented by one in RGB space).

EDIT: This comment led to a new thread that contains my original code, and improvements by others, all located HERE (http://forum.doom9.org/showthread.php?t=168293)

[StainlessS]

RT_Stats v1.21 update. See 1st post.

New GUI Filselector functions.


RT_FSelOpen(string "title"="Open",string "dir"="",string "filt",string "fn="",bool "multi"=false,bool "debug"=false)

Function to select EXISTING filename using GUI FileSelector.

Title = Title bar text.
Dir = Directory, "" = Current
Filt = Lots, eg "All Files (*.*)|*.*"
[Displayed text | wildcard] [| more pairs of Displayed text and wildcard, in pairs ONLY].
first one is default.
fn = Initially presented filename (if any).
multi = Multiply Select filenames. Allows selection of more than one filename at once.
debug = Send error info to DebugView window.

Returns
int, 0, user CANCELLED.
int, non zero is error (error sent to DebugView window).
String, Filename selected, Chr(10) separated multiline string if MULTI==true (and multiple files selected).

Example, to prompt for an AVI file and play it.
avi=RT_FSelOpen("I MUST have an AVI",filt="Avi files|*.avi")
Assert(avi.IsString,"RT_FSelOpen: Error="+String(avi))
AviSourcE(avi)

***
***
***

Function RT_FSelSaveAs(string "title"="Open",string "dir"="",string "filt",string "fn="",bool "debug"=false)

Function to select filename for Save using GUI.

Title = Title bar text.
Dir = Directory, "" = Current
Filt = Lots, eg "All Files (*.*)|*.*"
[Displayed text | wildcard] [| more pairs of Displayed text and wildcard, in pairs ONLY].
first one is default.
fn = Initially presented filename (if any).
debug = send errors to DebugView window.

Returns
int, 0, user CANCELLED.
int, non zero is error (error sent to DebugView window).
String, Filename selected.

***
***
***

Function RT_FSelFolder(string "title"="",string "dir"=".",bool "debug"=false)

Function to select Folder using GUI.

Title = UNDERNEATH the title bar text, for instructions.
Dir = Directory, Default "." = Current, ""=Root.
debug = Send errors to DebugView window.

Returns
int, 0, user CANCELLED.
int, non zero is error (ie -1, error sent to DebugView window, usually selecting non Folder object eg 'My Computer').
String, Folder selected (minus trailing BackSlash).



Also one of the added FSel_Demo AVS files, for scrolling text demo (5 modes).

avi = RT_FSelOpen("Please select an AVI file",Filt="*.AVI|*.AVI")
Assert(avi.IsString,"RT_FSelOpen: Error="+String(avi))
AVISource(AVI)
txt = RT_FSelOpen("And now select a Text file",Filt="*.txt|*.txt")
Assert(txt.IsString,"RT_FSelOpen: Error="+String(txt))
Txt=RT_ReadTxtFromFile(txt)
Lines=RT_TxtQueryLines(Txt)
# config
DELAY=100
ALIGN=5 # As Numeric KeyPad
SCROLL=0 # 0 = Upwards : 1 = Downwards : 2 = Right to Left : 3 = Left to right : 4 = Karaoke
#
ORG=Last
Last=(SCROLL==4)? ORG.Blankclip(height=64) : Last
CMD_0 = """RT_Subtitle("%s",Txt,align=ALIGN,y=height+DELAY-current_frame,expx=true,expy=true)"""
CMD_1 = """RT_Subtitle("%s",Txt,align=ALIGN,y=-(Lines*20+DELAY) + current_frame,expx=true,expy=true)"""
CMD_2 = """RT_Subtitle("%s",Txt,align=ALIGN,x=width+DELAY-current_frame,expx=true,vcent=true)"""
CMD_3 = """RT_Subtitle("%s",Txt,align=ALIGN,x=-(width+DELAY)+current_frame,expx=true,vcent=true)"""
CMD_4 = """RT_Subtitle("%s",Txt,align=ALIGN,y=height+DELAY-current_frame,expx=true,expy=true)"""
CMD_5 = """RT_Subtitle("BAD SCROLL COMMAND (0->4)")"""
CMD = (SCROLL<0 || SCROLL>4) ? CMD_5 : Select(Scroll,CMD_0,CMD_1,CMD_2,CMD_3,CMD_4)
ScriptClip(CMD)
Return (SCROLL==4)? StackVertical(ORG,Last) : Last



EDIT: RT_FselOpen() prompts for EXISTING filename, RT_FselSAveAs() will prompt for overwrite if already existing.

[StainlessS]

RT_Stats v1.22, update.

v1.22, Fixed RT_RGBChanXXXX channels. Changed RT_YStdev() to Standard Deviation from Sample Standard Deviation.

See 1st post.