RT_Stats, Compile-time/Runtime Functions v1.43 - 08 Oct 2014

[Forensic]

To me "Worked like a charm" referred to the fact I could finally build working AviSynth DLLs, even if they were significantly larger. Obviously, I prefer to make my builds fully compliant and appreciate any guidance. My environmental paths start with:
C:\Program Files\Microsoft SDKs\Windows\v7.1A\Include;C:\Program Files\Microsoft Visual Studio 11.0\VC\bin;
When I try to use MT (versus MTd) I get two errors {I used StainlessS's RGBamplifier source code for the test since I understand its behavior and can easily test the results}
Error 1 error LNK2019: unresolved external symbol __CrtDbgReportW referenced in function "public: class AVSValue const & __thiscall AVSValue:: operator[](int)const " (??AAVSValue@@QBEABV0@H@Z) c:\...\RgbAmplifier.obj RGBamplifier
Error 2 error LNK1120: 1 unresolved externals c:\...\Debug\RGBamplifier.dll 1 1 RGBamplifier
NOTE: The (...) are just to not annoy you with the full directory paths

[Groucho2004]

To me "Worked like a charm" referred to the fact I could finally build working AviSynth DLLs, even if they were significantly larger. Obviously, I prefer to make my builds fully compliant and appreciate any guidance.
For full compliance you would have to use VC6 unless you're using Avisynth+.
Looking at your error message it seems that you configured your project file to output a Unicode binary which is not supported by Avisynth.

[Forensic]

For full compliance you would have to use VC6 unless you're using Avisynth+.
Looking at your error message it seems that you configured your project file to output a Unicode binary which is not supported by Avisynth.
I recognize that the MT build does not work or even build (you identified the cause as Unicode), but the MTd version does both. I would happily use VC6, but can not find a copy. Do you know of an active download link? If I cannot get access to VC6, then I have no choice but to use what I have.

[Groucho2004]

I recognize that the MT build does not work or even build (you identified the cause as Unicode), but the MTd version does both. I would happily use VC6, but can not find a copy. Do you know of an active download link? If I cannot get access to VC6, then I have no choice but to use what I have.
The compatibility issues are mostly about different exception handling introduced in newer compilers. For example, it might just crash instead of throwing a nice error message before gracefully terminating.
There's a good chance that you'll never have any problems using VC11 for your plugin.
Please have a look at this (http://www.avisynth.nl/index.php/Filter_SDK/Compiling_instructions) page, it should give you enough info to build your DLL. The instructions for VS2008 should be more or less valid for VS2010/2012.

[TurboPascal7]

I would happily use VC6, but can not find a copy.
Don't. Just don't. Visual studio 2012 is a nice and correct way of building avisynth plugins as long as you handle all exceptions yourself and don't do dumb stuff no one is doing anyway. Don't even think about using VC6 in almost 2014.

[StainlessS]

we luddites would suggest using least modern that you can get away with :)

[Forensic]

Build->Configuration Manager, select "Release" in the left drop-down
Configuration Properties, General, Character Set "Multi-Byte Character Set"
Configuration Properties, General, Configuration Type "Dynamic Library (.dll)"

Now my builds are MT, work, and are very very small. :) Thank you.

[jmac698]

ok I can't get planetcrop to work :(
http://www.sendspace.com/file/h3x698

It just says planet can't be found, but it's very distinct, about 20 pixels across, luma from 30 to 160, and the background is under 20.

Show the detect clip it's very clear, the first frame is from x=380 to x=401

edit: I think the problem is that a few frames have no planet. Can you make it so that those frames are ignored?

There was another problem, a few frames were very large and made the whole sequence large, I'd like those ignored also. So ideally, find the most consistent size and use that.

[StainlessS]

Well I dont know what I was supposed to do with that bunch of pics.
The 3rd one does not look like it was taken on same night nor location in the sky, and has not just 1 but at least 3
luminous bodies (non of them being the body in the other pics) spanning almost entire frame width.
The forth pic looks like it could be the same planet as in 1st two pics, but definitely not in time lapse sequence.
You pic these pics out at random ?
If you delete/remove the 3rd pic (from another time/place) and just process the remaining 3 it works quite fine.
For the first few frames that have no planet (just black sky, not included in zip), suggest you delete them :)

EDIT: Actually, I'm quite amazed at how well it works when you remove the unrelated frame.

Curious, what was it (planet) ?

EDIT:
Jmac, I dont actually expect you to [rule 4], but you never did acknowledge receipt of previous requested
mods to PlanetCrop, nor to eg ShowChannels where I spent about a month or more banging my head against a wall
and despite pointing out the existence of mods got zero response. In that situation, all I can think is I wasted
my time (and a lot of it, not to mention the bruises). All you have to do to placate me is say yes, I saw the post
then I am at least aware that you have seen it (whether you thought it well accepted, or not).
I am quite frankly not in the mood to spend time especially when I end up not knowing whether it was completely
wasted or not ("I saw it, but it was shite." would be not ideal but appreciated).

P.S., if you did wanna [rule 4]

[jmac698]

Yep you're right, if I manually remove bad frames it works quite awesome, and I must say good job because in an animation, it looks virtually steady. They are all in the same sequence, but here's what happens: I bump the tripod, making it appear in a different location, or it moves out of frame after a few minutes. As I try to re-center it, I might catch a bit of something else like street lights.
I think most of these can be easily removed if I look through the beginning and end, so it's not a big problem after all.

I managed to process 85 frames to a noise-free image, unfortunately it was just a smooth blob. It's supposed to be Jupiter, but I'm thinking I pointed at a bright star by mistake, as there's no detail.

I'll try it on a quarter moon and see if it can center that.

It's quite amazing really and thanks for making it :)

[StainlessS]

I also made an edit Jmac, forgive me, bit pissed off at the moment.

[wiseant]

@StainlessS

Hi StainlessS

I said it before and I'll say it again:

You have to be one of the more underappreciated members here [how ironic underappreciated is deemed to be misspelled or not a recognized word].

What I see is someone who so generously gives their time to anyone that asks for your assistance. I wish that I had more time to look at all the neat stuff that you have created. I hope that you don't get discouraged and that you will keep on keeping on. The amount of information you have on MediaFire since the start of this year is amazing.

[vampiredom]

I agree 100%. Thank you, StainlessS.

[Forensic]

Your contributions have allowed my forensic peers to have customizable video enhancement resources beyond the limitations of industry specific software. And, as a mentor, you have helped me advance from scripting to DLL coding, so that I can pay it forward. You help anyone in need and never ask for anything in return. I believe that is the definition of a hero. Thank you.

[StainlessS]

Way too many sycophants on this site, fortunately Jmac aint one of them :)

Will not be active until mods to RT_ Array + DBase completed.

[wiseant]

Hmm . . . surely you jest - you get three supportive comments - not insincere nor excessive and certainly not self-serving

However, if you can, StainlessS, please try to find another place to upload your files - the pop-ups are horrendous

[StainlessS]

v1.30, 19 Dec 2013, Added DBase Functions, Increased Array attributes to 256, improved error messages.

See mediafire in sig

Script ReDAR_DGIT.avs, may be of some use:-


/*
ReDAR_DGIT.avs v0.1 - DGIndex AVS Template - By StainlessS.
Requires GSCript, RT_Stats v1.30+, RoboCrop, TIVTC, FFT3DFilter, NicAudio plugins.
Requires MediaInfo CLI version.

Progressive, or NTSC FILM with soft pulldown.
Purpose:- Crop letterboxing, with additional crop/resize to LESS wide screen DAR, CANNOT make more wide (DAR = Display Aspect Ratio).
Useful to crop to eg 16:9 on 16:9 hand held device.
Can also resize to SAR 1:1 [Sample (pixel) Aspect Ratio] for devices/players that cannot deal with non 1:1 clips.
Can Also call TFM and TDecimate to remove soft pulldown from NTSC FILM when Honour Pulldown Flags was used in DGIndex.
Signals resultant DAR for MeGUI in Global vars MeGUI_DarX and MeGUI_DarY.
*/

SHOW=False # Show Info on frame.
FRAME_DAR =RT_Undefined # Float, Input DAR of frame before RoboCrop(including letterboxing). RT_Undefined auto interrogates d2v file.
OUT_DAR =2.21/1.0 # Float, 0.0 = Use RoboCrop'ed DAR : Other = Additional crop to output DAR, eg FRAME 16:9, Robocrop'ed 2.4:1, OUT 16:9.
OUT_DAR_IS_LIMIT=True # If False, then OUT_DAR is required output DAR. (Error if OUT_DAR > RoboCrop'ed DAR, Cannot widen aspect ratio)
# If True and OUT_DAR != 0.0(0.0=RoboCrop'ed DAR), then reduce DAR only if RoboCrop'ed DAR is greater than OUT_DAR.
# Allows setting of maximum allowable output DAR(feel less like a peeping Tom for 2.4:1, eg limit to 16:9 on 4:3 device).

OUT_HEIGHT=0 # Int, 0 = Use RoboCrop'ed Height, Otherwise vertical resize to OUT_HEIGHT
OUT_WIDTH =0 # Int, 0 = Use OUT_SAR : Otherwise ignore OUT_SAR and use explicit OUT_WIDTH
OUT_SAR =0.0 # Float, Used if OUT_WIDTH=0 : 0.0 = Original SAR(crop only if OUT_HEIGHT=0) : 1.0 = 1:1 SAR
#
STRENGTH =0.75 # Float, Luma Auto Level strength (0.0 -> 1.0, 0.0=off)
UPCONV=True # Bool, False = no upsample(Progressive Encoded), True=upsample to YUY2(Progressive encoded as Interlaced), Can leave True.
###### Less often Changed
SAMPLES=32 # Int, Frames to sample, used by both RoboCrop and Auto Luma Levelling.
THRESH=-32 # int, -32 = RoboCrop AUTO Thresh
CROPMODE=2 # int, RoboCrop CropMode. 1=CropLess, 2= CropMore(Default)
TFM=True # If True, and FrameRate > 29.0 and MPEG-2 and DGIndex 'Honour Pulldown Flags' then Call TFM and TDecimate.
TFM_ANIME=False # True if Anime, For TDecimate
XMOD=0 # Int, 0=AUTO For crop/resize.
WMOD=4 # Int, 0=AUTO For crop/resize. (4 avoids problems on eg mp4 encodes displayed in MPC-HC)
MEDIAINFO="C:\BIN\MEDIAINFO.EXE" # To Check Aspect Ratio in IFO and compare with VOB : If conflict use IFO_MODE
IFO_MODE= 0 # 0 Error Alert on VOB/IFO DAR mismatch, 1 = Use IFO Frame Aspect Ratio
# List of programs that DONT like WMOD=2 for eg YUV.
# WMOD=0(AUTO) For YV12 will use eg 2 but switch to 4 if Process Name in WMOD4_PROGS list below (can append other names as in Task Manager).
WMOD4_PROGS="
VirtualDubMod.exe # VirtualDub Current OK
AVSEdit.exe # Built in Player
mplayer2.exe
wmplayer.exe
mpc-hc.exe
"
ROBOCROP_DEBUG=False
AUTOLEVEL_DEBUG=False
######################
### Filled in by DGIndex when using as DGIndex Template
VideoFileName ="__vid__"
AudioFileName =LCase("__aud__")
AudioDelay =Value("__del__")
######################
MPEG2Source(VideoFileName,UpConv=(UPCONV)?1:0) # Upconv to convert to YUY2, unmangle chroma if encoded as interlaced.

AudioExt=RT_GetFileExtension(AudioFileName)

Audio= (AudioExt==".ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
\ : (AudioExt==".mpa"||AudioExt=="mp1"||AudioExt==".mp2"||AudioExt==".mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
\ : (AudioExt==".wav") ? RaWavSource(AudioFileName)
\ : (AudioExt==".dts") ? NicDTSSource(AudioFileName)
\ : 0

Assert(!isInt(Audio),"NO AUDIO")

(!isInt(Audio)) ? AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0) : NOP # Trim, chop/add audio to length

(!isInt(Audio) && AudioRate() <> 44100) ? ResampleAudio(44100) : NOP
#(!isInt(Audio) && AudioRate() <> 48000) ? ResampleAudio(48000) : NOP
#(!IsInt(Audio) && (AudioRate() <> 44100 && AudioRate() <> 48000))?ResampleAudio(44100) : NOP
#(!IsInt(Audio))? ConvertToMonO().ResampleAudio(44100) : NOP

######
GSCript("""
myName="ReDAR_DGIT: "
D2VVOBS_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=1)
D2V_START=RT_NumberValue(D2VVOBS_S) + 3
D2V_LINES = 12
SS0=""
if(TFM && FrameRate>29.0) {
MPEG_LINE = RT_FileFindStr(VideoFileName,"MPEG_Type",Sig=False,start=D2V_START,lines=D2V_LINES)
MPEG_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=MPEG_LINE)
MPEG2 = (RT_FindStr(MPEG_S,"=2",Pos=9) >= 0)
if(MPEG2) {
HONOUR_LINE = RT_FileFindStr(VideoFileName,"Field_Operation",Sig=False,start=D2V_START,lines=D2V_LINES)
HONOUR_S=RT_ReadTxtFromFile(VideoFileName,lines=1,Start=HONOUR_LINE)
HONOUR_PULLDOWN_FLAGS=(RT_FindStr(HONOUR_S,"=0",Sig=False,Pos=15) > 0)
if(HONOUR_PULLDOWN_FLAGS) {
FIN_LINE = RT_FileFindStr(VideoFileName,"FINISHED",Sig=False)
Assert(FIN_LINE>=0,myName+"'FINISHED' NOT FOUND in d2v file")
FIN_S = RT_ReadTxtFromFile(VideoFileName,lines=1,Start=FIN_LINE)
IsFILM = (RT_FindStr(FIN_S,"FILM",Sig=False,Pos=9) > 0)
FILM_PERC = RT_NumberValue(FIN_S,Pos=9)
FILM_PERC = (IsFILM) ? FILM_PERC : 100-FILM_PERC
if(FILM_PERC >= 95) {
SS0=RT_String("Calling TFM and TDecimate")
RT_DebugF(SS0,name=myName)
SS0=SS0+Chr(10)
TFM(d2v=VideoFileName)
TDecimate(mode=(TFM_ANIME)?1:0)
}
}
}
}
# Cannot eg Trim/Add frames before here, will interfere with TFM and TDecimate
if(!Defined(FRAME_DAR)) { # Interrogate d2v for FRAME Display Aspect Ratio (incl letterboxing)
FDAR_LINE=RT_FileFindStr(VideoFileName,"Aspect_Ratio=",Sig=False,start=D2V_START,lines=D2V_LINES)
Assert(FDAR_LINE>=0,myName+"'Aspect Ratio' NOT FOUND in d2v file")
FDAR_S=MidStr(RT_ReadTxtFromFile(VideoFileName,lines=1,start=FDAR_LINE),14)
FNDS=RT_String("16:9,625\n16:9,525\n4:3,625\n4:3,525") # DGIndex version of MPEG1 PAR (y/x)
REPS=RT_String("0.7031\n0.8437\n0.9375\n1.1250") # Actual numbers replacing (MPEG1 PAR=y/x)
FDAR_S=RT_StrReplaceMulti(FDAR_S,FNDS,REPS) # Convert MPEG1 625(PAL),525(NTSC) Ratio to number
FDAR_N=Value(FDAR_S) # May be Float
COLON=RT_FindStr(FDAR_S,":") # Ratio ?
FDAR_D=(COLON==0) ? 1 : RT_NumberValue(FDAR_S,Pos=COLON+1)
Assert(FDAR_N>0 && FDAR_D>0,myName+"Error Reading d2v 'Aspect Ratio' (N="+String(FDAR_N)+",D="+String(FDAR_D)+")")
# Get Frame DAR from:- 1:1 SAR, or MPEG2 DAR, or MPEG1 PAR(y/x)
FRAME_DAR=(FDAR_N==1 && FDAR_D==1) ? Float(Width)/Height : (COLON!=0) ? Float(FDAR_N)/FDAR_D : (Float(Width)/Height)/FDAR_N
FRAME_DAR = Round(FRAME_DAR * 1000.0) / 1000.0
If(MEDIAINFO != "" && Exist(MEDIAINFO)) {
MPEG_S = RT_ReadTxtFromFile(VideoFileName,lines=1,start=2)
MPEG_S= RT_TxtGetLine(MPEG_S) # remove newline
M_EXT = RT_GetFileExtension(MPEG_S)
if(M_EXT == ".VOB") {
IFO = RT_FilenameSplit(MPEG_S,7)
IFO = LeftStr(IFO,Strlen(IFO)-1)
IFO = IFO + "0.IFO"
if(Exist(IFO)) {
RT_DebugF("File Is VOB and IFO exists",name=myName)
TMP=RT_GetFullPathName("~"+RT_LocalTimeString + ".txt")
RT_DebugF("Calling MediaInfo on IFO '%s'",IFO,name=myName)
# write Aspect Ratio in form "1.333" to file TMP
result=RT_Call(
\ RT_QuoteStr(MEDIAINFO) +
\ " --LogFile=" +
\ RT_QuoteStr(TMP) +
\ " --Inform=Video;%DisplayAspectRatio% " +
\ RT_Quotestr(IFO)
\ , hide=True,Debug=True
\ )
if(result==0 && Exist(TMP)) {
MI_TXT = RT_TxtGetLine(RT_ReadTxtFromFile(TMP))
RT_FileDelete(TMP)
Val = Value(MI_TXT)
RT_DebugF("MediaInfo Text = %s",MI_TXT,name=myName)
if(Val != FRAME_DAR) {
RT_DebugF("IFO(%f) and d2v(%f) Aspect Ratios DISAGREE, Using IFO AR = %f",Val,FRAME_DAR,Val,name=myName)
Assert(IFO_MODE==1,RT_String("IFO(%f) and d2v(%f) Aspect Ratios DISAGREE\nSet FRAME_DAR manually"
\ ,Val,FRAME_DAR))
FRAME_DAR=Val
} Else {
RT_DebugF("IFO and d2v Aspect Ratios AGREE on %f",FRAME_DAR,name=myName)
}
} Else {
RT_DebugF("MediaInfo Fails, result = %d",result,name=myName)
}
}
}
}
}

PROCESSNAME=RT_GetProcessName()
XMOD = (XMOD!=0) ? XMOD : RT_ColorSpaceXMod()
WMOD = (WMOD!=0) ? WMOD : (IsYUV() && RT_TxtFindStr(WMOD4_PROGS,PROCESSNAME,Sig=False)>=0) ? 4 : RT_ColorSpaceXMod
HMOD = Max(RT_ColorSpaceYMod(Laced=false),2)
RT_DebugF("ProcessName = %s WMOD=%d HMOD=%d",PROCESSNAME,WMOD,HMOD,name=myName)
OUT_DAR =Float(OUT_DAR)
OUT_SAR =Float(OUT_SAR)
SAR=RT_GetSAR(FRAME_DAR) # Sample (pixel) aspect ratio of input frame
SS1=RT_String("INPUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",Width,Height,FRAME_DAR,SAR,Float(Width)/Height)
RT_DebugF(SS1,name=myName)
SS1=SS0+SS1
RoboCrop(samples=SAMPLES,thresh=THRESH,cropmode=CROPMODE,debug=ROBOCROP_DEBUG,hmod=HMOD,Laced=false)
DAR=RT_GetDAR(SAR) # Image DAR after RoboCrop'ed letterboxing
SS2=RT_String("RoboCrop: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",Width,Height,DAR,SAR,Float(Width)/Height)
RT_DebugF(SS2,name=myName)
if(OUT_DAR_IS_LIMIT && OUT_DAR > DAR) {
OUT_DAR = DAR # Do not report Error, we are just limiting DAR
RT_DebugF("Ignoring OUT_DAR, (RoboCrop'ed DAR is Less))",name=myName)
}
Assert(OUT_DAR <= DAR,"OUT_DAR("+String(OUT_DAR,"%.3f") + ") Must be <= DAR("+String(DAR,"%.3f")+")")
OUT_SAR = (OUT_SAR == 0.0) ? SAR : OUT_SAR
OUT_DAR = (OUT_DAR == 0.0) ? DAR : OUT_DAR
OUT_HEIGHT=OUT_HEIGHT/HMOD*HMOD
OUT_WIDTH =OUT_WIDTH /WMOD*WMOD
H = (OUT_HEIGHT == 0) ? Height : OUT_HEIGHT # RoboCrop'ed OR user supplied resize height
# If OUT_WIDTH==0 Then use H(Height or user OUT_HEIGHT) & OUT_SAR(SAR or user OUT_SAR) to calc output width ELSE user OUT_WIDTH.
W = (OUT_WIDTH==0) ? Int(H * OUT_DAR / Float(OUT_SAR) / WMOD) * WMOD : OUT_WIDTH
if(OUT_WIDTH==0 && H==Height && OUT_SAR==SAR) { # Output RoboCrop'ed Width,Height, No resize, non-fractional cropping
CROP_WIDTH = W # Crop Only, maintain SAR, with approx OUT_DAR (occurs when original OUT_WIDTH==0 and OUT_HEIGHT==0 and OUT_SAR==0.0)
CROP_LEFT = Int((Width - CROP_WIDTH) * 0.5 / XMOD + 0.5) * XMOD # Crop centering, CROP_WIDTH and CROP_LEFT are Int
} Else { # Crop/Resize with fractional cropping to OUT_DAR
CROP_WIDTH = Width * (Float(OUT_DAR) / DAR)
CROP_LEFT = (Width - CROP_WIDTH) * 0.5 # Crop centering, CROP_WIDTH and CROP_LEFT are Float
}
# Now decide none, crop only, or crop/resize. CROP_WIDTH and CROP_LEFT may be Int or Float.
if(CROP_WIDTH==Width && CROP_LEFT==0 && W==Width && H==Height) {
SS3=RT_String("NO CROP/RESIZE, %dx%d DAR=%.3f",Width,Height,DAR)
} Else if(Int(CROP_LEFT/XMOD)*XMOD == CROP_LEFT && CROP_WIDTH==W && H==Height) { # Crop Only
CROP_LEFT=Int(CROP_LEFT)
CROP_WIDTH=Int(CROP_WIDTH)
Crop(CROP_LEFT,0,CROP_WIDTH,-0) # Cropping does not alter SAR
DAR=RT_GetDAR(SAR) # Calc resultant DAR after crop
SS3=RT_String("Crop(%.d, %d, %d, %d)",CROP_LEFT,0,CROP_WIDTH,-0)
} Else {
DAR=RT_GetCropDAR(DAR,CROP_LEFT,0,CROP_WIDTH,-0) # Pre-calc resultant DAR of Last clip, after possible fractional cropping
Spline36Resize(W,H, CROP_LEFT,0,CROP_WIDTH,-0)
SAR=RT_GetSAR(DAR) # Calc new SAR due to Resizing
SS3=RT_String("Resize(%d,%d, %.1f, %d, %.1f, %d)",W,H, CROP_LEFT,0,CROP_WIDTH,-0)
}
RT_DebugF(SS3,name=myName)
SS9=""
if(STRENGTH>0.0) { # Auto Luma Levelling
Eval(RT_QueryLumaMinMax(samples=SAMPLES,debug=AUTOLEVEL_DEBUG)) # Get luma min/max over SAMPLES frames
if(IsRGB()) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
SS9=RT_String("Levels(%d,1.0,%d,%d,%d,Coring=False)",ALMin,AlMax,CSMin,CSMax)
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring
}
}
RT_DebugF ("%s\nOUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f ",SS9,Width,Height,DAR,SAR,Float(Width)/Height,name=myName)
(SHOW) ? RT_Subtitle("%s\n%s\n%s\n%s\nOUT: %dx%d DAR=%.3f SAR=%.3f FAR=%.3f",SS1,SS2,SS3,SS9,Width,Height,DAR,SAR,Float(Width)/Height) : NOP
RT_SignalDar(OUT_DAR) # Tell MeGUI the Exact required Display Aspect Ratio rather than approximate.
RT_DebugF("Signalled MeGUI DAR=%f MeGUI_DarX=%d MeGUI_DarY=%d",OUT_DAR,MeGUI_DarX,MeGUI_DarY,name=myName)
ConvertToYV12
# FFT3DFilter(Sigma=1.6,plane=0,Sharpen=0.01)
""")
Return Last


EDITED:

[johnmeyer]

Many thanks for the security cam detection script:

SecurityCamExtractMotion.AVS (http://forum.doom9.org/showthread.php?p=1628675#post1628675)

It works much better than using Scene Detection in MVTools2, or YDiffxxx, etc. In fact, it is just plain great!

Quick question.

I was able to get it to work when using SetMTMode, but I'm still looking for more performance, since my security camera captures are twelve hours each at 30 fps. I've used the cropping parameters to reduce the pixels that must be evaluated, but can you suggest any other things I might do to get even faster performance? I'm able to get 80 fps on 30 fps 1440x1080 AVCHD footage. This is obviously pretty darned good, and I'm very impressed with how fast this works. So, I'm not complaining; I'm just greedy.

Thanks!

[StainlessS]

Hmm . . . surely you jest - you get three supportive comments - not insincere nor excessive and certainly not self-serving

However, if you can, StainlessS, please try to find another place to upload your files - the pop-ups are horrendous

Yes of course I jest (hence the smiley :) )
You should never take anything I say as serious, I dont (even without smiley).

Mediafire does still seem to be preferred file host on D9, I dont usually see much in the way of pop-ups from there (or anywhere else). Suggest that you switch on/install pop-up blocker, ad-blocker and flash-blocker, also FireFox is a good idea and way safer from malware than IE (EDIT: every hackers favorite target).

@John,
In the older script version you linked, I think I tried to keep all
frame in areas of movement even if no movement in particular frames, Forensic seemed to give more importance to getting rid of as many frames as possible, so I think I changed behaviour
to match (in later versions of script in RT_Stats zip).
Do you have any preference, least frames/ keep alll frames in movement area ?

I have been revisiting scripts supplied in RT_Stats.zip and will again look at this one, give me some time, I'm not spending as much time as previously on Avisynth, found other things to occupy myself.

EDIT: I'm not in NTSC land and only added the pulldown removal stuff to script in previous post as a sort of
example for others to expand on. I only have 1 single NTSC VOB (rescued from a DVD that is now almost totally corrupt)
and so have only 1 sample to play with. If anyone does expand on NTSC handling, you could post results here.

[Groucho2004]

Mediafire does still seem to be preferred file host on D9
Mediafire lately seems to have been embracing the dark side. I suggest you move your stuff to dropbox.com, much lighter and no popups. I did. :)

[StainlessS]

Thanks Grouchy, I shall look into that. :)

Here, DBase funcs as finally added to RT_Stats v1.30.


*****************************************************
********** DBASE FUNCTIONS ***********
*****************************************************

RT_Stats DBASE functions allow fast access to a file based DataBase where each record can have up to 256 fields of variable type,
Bool, Int, Float, String, or BIN. BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in the DBase field
as an unsigned 8 bit Int, upper 24 bits ignored.
The maximum possible file size is about 2GB, and the number of records is restricted by ($7FFFFFFF - Header_Size) / Record_Size
where $7FFFFFFF is maximum +ve integer, Header_Size is 4096 and Record_Size is sum of all Field sizes.
Field types Bool and BIN require 1 byte, Int and Float 4 bytes, and String is of fixed maximum length (user chosen) up to
256KB ((256 * 1024), no nul terminating character is stored in a DBase String).
In addition to the DBase records, a DBase file allows up to 256 Attributes of types Int or Float, which can be used for
whatever your heart desires, these attributes are stored in the DBase file header.
You could use the DBase functions to return multiple results from a script function to the caller via a caller supplied DBase
filename. The DBase functions also make it more likely that script functions could be written to eg find where edits between
two clips have been made, or previously impossible/implausible tasks performed.
Currently, the DBase functions are just primitive access functions, it is possible that some future update may have functions
added for eg sorting or searching or selecting records, who knows. :)


***
***
***

RT_DBaseAlloc(String Filename,Int Records,String TypeString,Int "StringLenMax"=256)
Creates a file to hold DataBase records. Each record can have up to 256 fields of varying type, Bool, Int, Float, String, or BIN.
BIN is a BYTE (8 bit) sized Int where only lowest 8 bits of Int are stored in DBase field as an unsigned 8 bit int, upper 24 bits ignored.

Filename, DBASE Filename, no default.

Records, int zero or more. Number of records to create, initalized to zeros (initialized to zeros, Bool=False,Float=0.0,String="").

TypeString, string holding record construction data. One character for each field, valid characters are "bifsn" or Uppercase.
'b' = BOOL, 'i' = INT, 'f' = FLOAT, 's' = STRING, 'n' = BIN.
The 's' STRING construction character can be followed by numeric digits describing fixed maximum string length (excluding
any nul term character), otherwise StringLenMax is used by default instead.
For no particular reason, I have limited the maximum string length to 256K, ie 256 * 1024, throws an error if greater.

StringLenMax Default=256. Default length of String fields if no explicit string length defined following TypeString S constructor char.

Example TypeString:-
"ifffs1024bsn", 1st field Int, 3 Float fields, 1 string field of length=1024, 1 Bool field, 1 string field using length of
StringLenMax, and 1 BIN field. 8 Fields in all (0->7).
Returns number of records created.
All RT_DBaseXXXX() errors produce an error alert.

***
***
***

RT_DBaseRecords(String Filename)
Filename, DBASE Filename, no default.
Returns current number of records in Filename database.

***
***
***

RT_DBaseRecordSize(String Filename)
Filename, DBASE Filename, no default.
Returns record size, ie sum of field sizes.

***
***
***

RT_DBaseRecordsMax(String Filename)
Filename, DBASE Filename, no default.
Returns maximum theoretical number of records that could be stored in file based on (Maximum +ve int - header size) / Record Size.
(where record size is sum of all field sizes).

***
***
***

RT_DBaseFields(String Filename)
Filename, DBASE Filename, no default.
Return number of fields per record.

***
***
***

RT_DBaseFieldType(String Filename,Int Field)
Filename, DBASE Filename, no default.
Field, range 0 -> RT_DBaseFields(Filename) - 1
Return type of Field. 0=Bool, 1=Int, 2 = Float, 3 = String, 4 = BIN.

***
***
***

RT_DBaseFieldSize(String Filename,Int Field)
Filename, DBASE Filename, no default.
Field, range 0 -> RT_DBaseFields(Filename) - 1
Return size of Field in bytes. Bool=1, Int=4, Float=4, String length as set by RT_DBaseAlloc(), BIN=1.

***
***
***

RT_DBaseGetField(String Filename,Int Record,Int Field)
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Field, range 0 -> RT_DBaseFields(Filename) - 1
Returns contents of a field belonging to the particular record.

***
***
***

RT_DBaseSetField(String Filename,Int Record,Int Field, FieldVar)
Sets a single Field in a Record to FieldVar.
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
Field, range 0 -> RT_DBaseFields(Filename) - 1
FieldVar, variable of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseSet(String Filename,Int Record, FieldVar_1, ... , FieldVar_n)
Simultaneously sets all fields of a record. All field variables must be supplied.
Filename, DBASE Filename, no default.
Record, range 0 -> RT_DBaseRecords(Filename) - 1
FieldVar_1, ... , FieldVar_n. A variable for each field of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseExtend(String Filename,Int "Add"=1)
Adds a number of blank records to Filename Database. (initialized to zeros, Bool=False, Float=0.0, String="").
Filename, DBASE Filename, no default.
Add, Int default 1, Number of blank records to add to the DataBase.
Returns the current number of records.

***
***
***

RT_DBaseAppend(String Filename, FieldVar_1, ... , FieldVar_n)
Creates a record from Field Vars and appends it to the end of the DataBase file. All field variables must be supplied.
Filename, DBASE Filename, no default.
FieldVar_1, ... , FieldVar_n. A variable for each field of type as set by RT_DBaseAlloc().
Returns the current number of records.

***
***
***

RT_DBaseGetAttrib(String Filename,Int ix)
There are 256 available attributes in the DataBase file header, that can be used for whatever purpose you like,
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 255.
Returns the value of the user set attribute Int or Float (or type Int 0 if not yet set).

***
***
***

RT_DBaseSetAttrib(String Filename,Int ix, Var)
There are 256 available attributes in the DataBase file header, that can be used for whatever purpose you like,
RT_DBaseAlloc() initializes all Attributes to type Int, 0.
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 255.
Value, Int or Float, The value that Attrib(ix) will be set to.
Returns Value. Error if type not of type Int or type Float.


Also, number of Array attributes increased to 256.

[johnmeyer]

In the older script version you linked, I think I tried to keep all frame in areas of movement even if no movement in particular frames, Forensic seemed to give more importance to getting rid of as many frames as possible, so I think I changed behaviour to match (in later versions of script in RT_Stats zip).

Do you have any preference, least frames/ keep alll frames in movement area ?

I have been revisiting scripts supplied in RT_Stats.zip and will again look at this one, give me some time, I'm not spending as much time as previously on Avisynth, found other things to occupy myself.No need to make any changes. I am using the script to generate frame numbers that I then import into my NLE (Vegas). This gives me markers, and I can then simply go to each marker. I "thin out" the markers in Excel, prior to importing to Vegas, so I don't need any logic in the script itself to only keep the first/middle/last frame during a motion sequence.

So, the script works brilliantly and perfectly; my only goal is to improve the performance, even though it is already very fast. To do this, I'm simply looking for settings that might bypass certain operations. I'm not asking for you to do any programming or changes.

Once again, many thanks!

[StainlessS]

Oooops, sorry John, I meant to come back to this but forgot until I saw you online a little earlier.
I've just looked at the script and only suggestions I can make are to increase block size (should
not be so important for HD content, I think [governs small localized movement]), and an ad hoc fix.
Ad hoc fix being to set a Global var name eg LACED=true/False to control args to all instances
of RT_LumaMovement, and add to each instance of RT_LumaMovement an 'Interlaced=LACED' arg.
Increasing block size should not matter too much for HD and although testing only every other row
of pixels would make a difference to sensitivity, I'm guessin that it will not matter too much.
Reducing EDIT: Should read Increasing block size should reduce overhead (eg checking coords) for each block, and the laced thing
could perhaps yield up to double speed, but you'll have to suck it and see to find out for sure,
let us know how it goes, good luck.

I'll add the laced thing in next version, peace bro.

EDIT: 'yield up to double speed', relates to the script, eg no effect on source filter, reading hard drive etc.
EDIT: Will likely require changes to thresholds (if changing block size).
EDIT: Try ad hoc fix first, and play with block size after that. If you seem to lose sensitivity after LACED thing,
you might even want to decrease block size.
EDIT: I dont use MT, and so have no suggestions concerning that.

[Forensic]

I am trying to detect if a specific file already exists in the same directory as my script and, if not, do stuff. I tried

try { rt_temp = RT_GetFileTime ("test.txt") }
catch (err) { DO STUFF }

Something about that syntax is wrong. Any advice or a better way to test for this?

[StainlessS]

Try built-in Exist().

From Docs:-

Exist | v2.07 | Exist(filename)
Tests if the file specified by filename exists.
Examples:

filename = ...
clp = Exist(filename) ? AviSource(filename) : Assert(false, "file: " + filename + " does not exist")


EDIT: From RT Docs

RT_GetFileTime(string filename,int item)
Returns string, one of three times associated with filename file.
Error if cannot access file. (debugview for error string).
Item, int, (0 -> 2). 0=Creation Time, 1=Last Write/Modified Time 2=Last Accessed Time,
The return string is always of format "YYYY-MM-DD HH:MM:SS.mmm" and so can be used to
compare whether one file time is later than another using string comparison.
The time strings returned are in UTC (Coordinated Universal Time), not local time.
v1.28Beta5, added milliseconds to string.


You were missing compulsory 'Item' arg.

[Forensic]

Thank you.

[StainlessS]

Posted in response to a Reel.Deel post here: http://forum.doom9.org/showthread.php?p=1681509#post1681509


Extracts all filter/function names and arguments contained in an autoloaded dll, writes results to a text file.
Target dll required to be in Plugins dir. Hack of something I did to list RT_Stats funcs.

# Make_FunctionList.AVS by StainlessS, Requires RT_Stats v1.30+, and GScript.
# Extracts names and args of Filters and Functions contained in an autoloaded dll, results written to text file.
# Plugins need to be auto loaded in Plugins Dir.

ORDER=True

HEAD="""
There follows a list of all function names together with CPP style argument specifiers that inform
Avisynth the argument types and optional names. Optional arguments have square brackets surrounding
their name as in [name] and are followed by a type specifier character that gives the type.
Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args,
followed by optional 'arg1' of type bool and optional 'arg2' of type int.

# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
###################################
"""
Assert(RT_FunctionExist("GScript"),"You Need GScript To Run This Script"+Chr(10)+"http://forum.doom9.org/showthread.php?t=147846")
FSEL_TITLE="Select DLL file"
FSEL_DIR=RT_PluginDir()
FSEL_FILT="DLL files|*.dll"
FSEL_MULTI=False
DLLFILE_LIST = RT_FSelOpen(title=FSEL_TITLE,dir=FSEL_DIR,filt=FSEL_FILT,multi=FSEL_MULTI)
Assert(DLLFILE_LIST.IsString,"RT_FSelOpen: Error="+String(DLLFILE_LIST))
DLLFN=RT_TxtGetLine(DLLFILE_LIST,0)
PREFIX = RT_FilenameSplit(DLLFN,4) + "_"
PREFIX_LEN=StrLen(PREFIX)
ALLPLUGS = RT_StrReplace(RT_PluginFunctions," ",Chr(10))
Assert(RT_TxtFindStr(ALLPLUGS,PREFIX,Sig=False,Start=0)>=0,PREFIX+" Not found, Not an AutoLoad plugin(or more than 50 plugs)")
LINES = RT_TxtQueryLines(ALLPLUGS)
PLUGS = ""
PARMS = ""
MaxLen = 0
DQUOT=34 # Double Quote Chr(34)
GSCript("""
for(i=0,LINES-1) {
i=RT_TxtFindStr(ALLPLUGS,PREFIX,Sig=False,Start=i)
if(i >= 0) {
PLG=RT_TxtGetLine(ALLPLUGS,Line=i)
if(RT_FindStr(PLG,PREFIX,Sig=False)==1) { # match at start ?
SHORTPLG=MIDStr(PLG,PREFIX_LEN+1)
MaxLen=Max(MaxLen,StrLen(SHORTPLG))
PLUGS=RT_TxtAddStr(PLUGS,SHORTPLG)
PARMS=RT_TxtAddStr(PARMS,RT_String(" %c%s%c",DQUOT,RT_PluginParam(PLG),DQUOT))
}
} Else {
i = LINES # break, no more to find
}
}
LINES = RT_TxtQueryLines(PLUGS)
PLUGS_AND_PARMS=""
for(i=0,LINES-1) {
PLUGS_AND_PARMS=RT_TxtAddStr(PLUGS_AND_PARMS,RT_StrPad(RT_TxtGetLine(PLUGS,Line=i),MaxLen)+RT_TxtGetLine(PARMS,Line=i))
}
""")
PLUGS_AND_PARMS = (ORDER) ? RT_TxtSort(PLUGS_AND_PARMS) : PLUGS_AND_PARMS # Order Alphabetical
TXT = RT_String("%s%sFunction_List",PREFIX,(ORDER)?"ORDERED_":"")
FN=TXT+".TXT"
S=RT_String("\n %s \n\n%s\n\n%s",TXT,HEAD,PLUGS_AND_PARMS)
RT_Debug(S)
RT_TxtWriteFile(S,FN)
S=RT_TxtAddStr(S,RT_String("\n\n%s\n\nCreated in current directory\n",FN))
W=1280 H=640 LINES=RT_TxtQueryLines(S)
L=LINES*20+H+100
BlankClip(Width=W,Height=H,Length=L).ScriptClip("""RT_Subtitle("%s",S,x=10,y=height+100-current_frame,expx=true,expy=true)""")

EDITED: Throws Error if not an autoloaded plugin.

EDIT: If I remember correctly, Avisynth only autoloads the first 50 plugins it comes across in the plugins dir, so the above script may not work for all autoload plugs if more than 50 in plugins dir.

[Reel.Deel]

Posted in response to a Reel.Deel post here: http://forum.doom9.org/showthread.php?p=1681509#post1681509

Gave it a try and it works like a charm, thanks StainlessS. :)

[StainlessS]

RT_Stats v1.40 Beta 01
Post #1 of #2 (docs too big for 16KB limit)


v1.40Beta01

RT_WriteFile(String FileName, string format,dat1,...,datn,bool "Append"=False)
Writes a formatted string to FileName file. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the
text string that is written, uses C/CPP printf() style formatting.
Format: compulsory string controlling format and describing the datn type args that are expected.
datn: Variable number of data args of any type (excluding clip).
Append, Default False, optional and MUST be supplied with name (eg Append=True) as we do not know where optional data args end,
Appends to file if already exists, else opens new FileName.

printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned (o=octal, x=Hex).
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

Formatting Insertion point is marked with '%' character in the format string (as in Avisynth String function), if you wish to use
a percent character within the output string, it should be inserted twice in the format string ie '%%' will produce a single '%'.
The data datn arg strings do not require a double '%' character.

A Backslash character '\' introduces escape sequences, to insert a backslash character itself, you must supply a double
backslash sequence ie '\\'.
Converts embedded escape character sequences (Case Significant):-
'\\' Converted to '\' Single Backslash
'\a' Converted to Chr(7)
'\b' Converted to Chr(8)
'\t' Converted to Chr(9)
'\n' Converted to Chr(10) NewLine
'\r' [and Chr(13)] Converted to Chr(10) NewLine
'\v' Converted to Chr(11)
'\f' Converted to Chr(12)
'\x', where x is ANY OTHER CHARACTER not included above, will be copied verbatim, ie '\x'.

Escape sequences are replaced only when occurring in the format string, if you need escapes in data string then use RT_String.

Example:
RT_WriteFile(FileName,"Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013,Append=False)

***
***
***

RT_VarType(var)
Returns type of var arg as an int.
Returns,
0 = Type BOOL
1 = Type INT.
2 = Type FLOAT
3 = Type STRING
4 = Type CLIP
-1 = Type UNDEFINED
Avoids multiple tests in script to establish a variable type.
eg, RT_VarType(66.6) returns 1, ie FLOAT.
NOTE, If called without an arg, it will return type of implicit Last (Undefined if no clip implicitly/explicitly assigned to Last).

***
***
***

RT_VarIsSame(var Var1, ...,var Varn)
Accepts two or more variables of any type. NOTE, if only 1 arg supplied, will use implicit Last as first arg.
Returns Bool, False if args are not all of same type AND/OR value.
Return True, if all args same type and value.
Type String, compares strings exactly, same length and casing.
Type Clip, compares ONLY video attributes, frame Width, Height, ColorSpace.
If first clip has no video (HasVideo==False) then all other clips must also have no video (other attributes not tested).
Clip audio, compares Number of Audio Channels, Sampe Rate, Sample Type (int16, Float etc).
If first clip has no audio (HasAudio==False) then all other clips must also have no audio (other attributes not tested).
No other attributes are taken into account, (ie FrameRate, FrameCount etc).

***
***
***

RT_ChanAve(clip c,int "n",string "Prefix"=RCA_")
Sets channel averages for clip c frame n, as local variables, 0.0 -> 255.0. RGB24/32, YUY2, Planar.
Returns the number of channels in clip ie 3 unless Y8 where returns 1.
Default Prefix is "RCA_" so for YV12, would return 3 (3 channels) and set
RCA_0= Luma Ave, RCA_1= U Ave, RCA_2 = V Ave.
For RGB, RCA_0= Red Ave, RCA_1= Grn Ave, RCA_2 = Blue Ave.
For Y8, RCA_2 and RCA_3 will be set at 128, but will return a value of 1 for number of channels.


***
***
***

RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0)
Returns difference between clip c frame n and clip c2 frame n2 (0.0 -> 255.0).
ChromaWeight range 0.0 -> 1.0.
For RGB, returns same as RGBDifference.
For Y8 returns same as LumaDifference.
For non Y8 Planar returns Lumadif + Chromadif with chroma weighted by ChromaWeight arg.
Returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
EXPERIMENTAL: Also currently accepts ChromaWeight -ve (eg -1.0) where will return
Lumadif + (1.0 - Lumadif/255.0) * ((Udif + Vdif)/2.0)
This may well be removed at some future date (although would do no harm to leave in-situ).

***
***
***

RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=current_frame,int "x2"=0,int"y2"=0,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns number of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Note, returns the number of pixels whose difference is greater than Thresh, so ranges 0 -> (Width*Height).
Thresh Default 0, returns number of pixles that are not exactly the same.

***
***
***

RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elased in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only ting that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str="Test Start="+String(s,"%.3f")+" End="+String(e,"%.3f")+" Time="+String(e-s,"%.6")+"secs"
SubTitle(str)

***
***
***

RT_SignalDAR2(int darx,int dary)
As for RT_SignalDAR() except it allows setting of numerator and denominator individually.
Signal DAR to MEGUI. Sets Global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

***
***
***

[StainlessS]

RT_Stats v1.40Beta01
Post #2 of 2


RT_QwikScanCreate(clip c,string DB,string "PrevDB"="",string "NextDB"="",int "matrix"=(Width>1100||Height>600?3:2),bool "Debug"=False)
Creates a DataBase set used to quickly scan a clip for frame matching, DataBases used by RT_QwikScanGetNear.
c, clip used to create database frame fingerprint info.
PrevDB, DBase filename, used to find frames prior to a 'start frame'.
NextDB, DBase filename, used to find frames after a 'start frame'.
Either PrevDB or NextDB (or both) must be given.
Matrix, used to convert RGB to Luma_Y for Fingerprint creation.
Debug, True sends a little info to DebugView window (google).
Maximum PrevDB/NextDB size is restricted to 524283 records(frames).

***
***
***

RT_QwikScanGetNear(clip DBClip,string DB,string PNDB,int StartFrame,int FindFrame, \
clip "FindClip"=NULL,int "Maxdistance"=DBClip.FrameCount-1,bool "Inclusive"=True,Float "LumaTol"=0.0, \
int "Flags"=0, Float "LD"=0.0,Float "LC"=0.0,Float "FD"=0.0,int "PC"=0,int "XP"=1,string "Prefix"="QSGN_",Float "ChromaWeight"=1.0/3.0)

*** EXPERIMENTAL ***

Developer function only, not for the casual scriptor.
Function that scans a pair of database files to find a sequence of candidate frames that could match a target FindFrame.
Returns frame number of possible matching frame, or -1 meaning NOT FOUND.
DBClip, the clip to search, and used as arg to RT_QwikScanCreate to create DB, PrevDB and NextDB database files that hold
"FingerPrint" and frame location data to assist in rapid location of candidate frames.
DB, Filename of the "FingerPrint" Dbase. Created via RT_QwikScanCreate.
PNDB, Filename of the frame locator Dbase. Created via RT_QwikScanCreate as either PrevDB or NextDB.
Use the PrevDB if searching frames prior to StartFrame in DBClip, or the NextDB if search frames after StartFrame.
StartFrame, frame in DBClip where searching will start, see also Inclusive.
FindFrame, Frame number of the frame that is to be matched (in FindClip if supplied or DBClip if not supplied).
FindClip, clip holding frame to match, defaults to DBClip if not supplied.
MaxDistance, maximum number of frames to search, defaults DBClip.FrameCount -1, but silently limited to the maximum possible
distance depending upon where StartFrame is and whether searching forwards or backwards. A MaxDistance of 1 will search only
the StartFrame (if Inclusive) and the frame next to it.
Inclusive. If true (default) then also tests StartFrame.
LumaTol, +- Luma Tolerance (0.0 -> 16.0). Allow for mismatch in luma between DBClip and FindClip. If eg the FindClip pixels can vary
by as much as +- 4 pixel levels, then set LumaTol to 4.0, the bigger this number the slower the function, but it should be still
quite fast with most clips even up to a LumaTol of maximum 16.0, the bigger this number, the more frames will fit the fingerprint
and so there will be more candidate frames to check.
Flags, Bit flags, sets the additional Exit Conditions of the function.
Bit 0 set eg 0x01, Sets the LumaDifference Exit Condition.
Bit 1 set eg 0x02, Sets the LumaCorrelation Exit Condition.
Bit 2 set eg 0x04, Sets the FrameDifference Exit Condition.
Bit 3 set eg 0x08, Sets the PixelsDifferentCount Exit Condition.
LD, LumaDifference Exit Condition Threshold.
LC, LumaCorrelation Exit Condition Threshold.
FD, FrameDifference Exit Condition Threshold.
PC, PixelsDifferentCount Exit Condition Threshold (int).
XP, Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=continue searching while next frame is a better Exit Condition match.
2=Continue searching while next frame is a better or equal Exit Condition match.
Prefix, Default "QSGN_". Prefix for names of local variables returned to give status of exit conditions.
Below only set if Flags non zero.
QSGN_Exit_Flags, bit flag, condition that was satisfied to cause exit (same as Flags arg, only 1 bit will be set).
QSGN_Exit_LD, LumaDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x01.
QSGN_Exit_LC, LumaCorrelation for the exit frame, Only set if QSGN_Exit_Flags == 0x02.
QSGN_Exit_FD, FrameDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x04.
QSGN_Exit_PC, PixelsDifferentCount for the exit frame, Only set if QSGN_Exit_Flags == 0x08.
QSGN_Exit_XP, Count of additional frames traversed if found to be a better match. Only set if both XP arg is non zero
and exit condition occurred ie QSGN_Exit_Flags non zero.
ChromaWeight, Argument to the FD FrameDifference Exit Condition (YUV only).

The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
The PNDB Frame Locator Dbase holds records with 1024 fields per record (frame). The first 256 of these fields give frame numbers
of the next nearest frame (previous[PrevDB] or next[NextDB]) that roughly complies with the first component FingerPrint value.
Eg, field 42 holds the frame number of the nearest frame whose first component FingerPrint value is greater or equal to 42.0
and less than 43.0. Fields 256 to 511 hold frame info for the 2nd FingerPrint component, etc.

This is roughly how the function works without Exit Conditions (Flags=0):-
It first establishes a range of component FingerPrint values for FindFrame, based on LumaTol.
If Inclusive, then checks StartFrame for a FingerPrint match and returns StartFrame if good match.
Next it Checks the Frame Locator Dbase (PNDB) looking for the next nearest frame that roughly complies with the FingerPrint
range. It then checks that candidate frame in the FingerPrint DBase to see if it more exactly matches the FingerPrint, if it
does match, the frame is returned to the client and if not then it moves the 'Cursor' in the Frame Locator DBase up to that
frame and repeats the searching until found or not.
So, without Exit conditions, the function returns a sequence of candidate frame numbers that may or may not be the exact frame
you are looking for, it will likely skip vast swathes of frames and then deliver a stream of frames from the same scene as
your FindFrame, where you would likely want to establish a more exact match using some other function.

With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a result to
the client. If an Exit condition is set, then that exit condition also has to be satisfied, if not satisfied, then the candidate
frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and the Exit
Condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, they are processed in bit number order ie LumaDifference condition first, the first
condition that succeeds will cause exit and no further exit conditions will be processed. If an Exit Condition is satisfied,
and the XP arg is set, the function will continue scanning adjacent frames and step forward until a better match for the Exit
Condition is NOT found. (XP=1 is better than, and XP=2 is better or equal).

Exit Conditions:-

LD, LD is the Threshold for RT_LumaDifference, if the match is less or equal to this then the Exit Condition is satisfied.

LC, LC is the Threshold for RT_LumaCorrelation, if the match is less or equal to this then the Exit Condition is satisfied.
We have 'swizzled' the values returned by RT_LumaCorrelation(), instead of the normal range -1.0 -> 1.0,
we use 0.0 -> 255.0 where 0.0 means exact match. We have done this for 2 reasons. Firstly, to make it the same (roughly)
as the other exit conditions, the closer to zero, the better the match. 2nd reason is, Avisynth Floats have only 6 digits
of decimal precision, so eg 0.999999 is about a good as it gets (closest to 1.0 without actually being 1.0) if we were
setting a threshold for normal range LumaCorrelation (-1.0 -> 1.0), but when we 'swizzle' the numbers to make 0.0 an exact
match, we can supply a more precise threshold which will be used at double precision within the function,
eg 0.0000000000000000000000001 is still a valid float (and in this case it is only 1 significant digit of precision).
RT_LumaCorrelation is slower than RT_LumaDifference but it may be more appropriate if there is a big difference in
luma beteen DBClip and FindClip.

FD, FD is the Threshold for RT_FrameDifference, if the match is less or equal to this then the Exit Condition is satisfied.
The RT_FrameDifference Exit Condition uses the ChromaWeight arg to weight chroma for YUV colorspaces.
For RGB, RT_FrameDifference is equivalent to RGBDifference. For Y8 it is equivalent to LumaDifference.
For non Y8 YUV, returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
EXPERIMENTAL: Also currently accepts ChromaWeight -ve (eg -1.0) where will return
Lumadif + (1.0 - Lumadif/255.0) * ((Udif + Vdif)/2.0)
This may well be removed at some future date (although would do no harm to leave in-situ).

PC, PC is the Threshold for RT_LumaPixelsDifferent, if the match is less or equal to this then the Exit Condition is satisfied.
PC is a pixel count, ranging 0 -> (Width * Height). Allows you to set maximum number of pixels that are in any way different
(RGB converted to Luma-Y).

Usage:-
Using PAL SD Jurassic Park with 182050 frames (UT_Video Compressed YV12) it took 15:06 to create DB and PrevDB FingerPrint
and Locator DB on a 2.13Ghz Core Duo, using this script.

AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
return RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)

And test script:
AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
#RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)
GScript("""
LUMATOL=0.0
End=FrameCount-1
For(i=3000,13000,1000) {
S=RT_TimerHP()
j = RT_QwikScanGetNear(Last,DB,PNDB,End,i,Last,maxdistance=End-i,lumatol=LUMATOL,flags=$04,fd=0.0,xp=2)
T=RT_TimerHP-S
RT_Debugf("%d] Got %d in %f secs",i,j,T)
}
""")
return Last

Results:- LumaTol = 0.0
00000005 398.69018555 RT_DebugF: 3000] Got 3000 in 0.019132 secs
00000006 398.70736694 RT_DebugF: 4000] Got 4000 in 0.017023 secs
00000007 398.73089600 RT_DebugF: 5000] Got 5000 in 0.023335 secs
00000008 398.75195313 RT_DebugF: 6000] Got 6000 in 0.020897 secs
00000009 398.76660156 RT_DebugF: 7000] Got 7000 in 0.014457 secs
00000010 398.78686523 RT_DebugF: 8000] Got 8000 in 0.020073 secs
00000011 398.80563354 RT_DebugF: 9000] Got 9000 in 0.018632 secs
00000012 398.82150269 RT_DebugF: 10000] Got 10000 in 0.015690 secs
00000013 398.85241699 RT_DebugF: 11000] Got 11000 in 0.030706 secs
00000014 398.86682129 RT_DebugF: 12000] Got 12000 in 0.014216 secs
00000015 398.88012695 RT_DebugF: 13000] Got 13000 in 0.013135 secs

Results:- LumaTol = 4.0 (many more candidate frames to test)
00000016 444.04776001 RT_DebugF: 3000] Got 3000 in 0.059216 secs
00000017 444.26177979 RT_DebugF: 4000] Got 4000 in 0.213837 secs
00000018 446.82482910 RT_DebugF: 5000] Got 5000 in 2.562858 secs
00000019 454.65585327 RT_DebugF: 6000] Got 6000 in 7.830853 secs
00000020 456.44543457 RT_DebugF: 7000] Got 7000 in 1.789409 secs
00000021 465.26766968 RT_DebugF: 8000] Got 8000 in 8.822010 secs
00000022 465.99948120 RT_DebugF: 9000] Got 9000 in 0.731642 secs
00000023 466.64910889 RT_DebugF: 10000] Got 10000 in 0.649458 secs
00000024 471.09573364 RT_DebugF: 11000] Got 11000 in 4.446451 secs
00000025 471.81393433 RT_DebugF: 12000] Got 12000 in 0.718018 secs
00000026 475.26538086 RT_DebugF: 13000] Got 13000 in 3.451269 secs


EDIT: Above usage examples scan from last frame to target frame, ie about 180,000 frames to search.
EDIT: RT_QwikScanCreate, takes some time to create the DB Dbase, the PrevDB and NextDB
are almost instantanious (maybe about 5 secs for a 200,000 frame DB that takes about 15 mins
to create, so bi-directional searches are not any kind of problem).
I'm hoping that someone will give this a beta test, works unbelievably well. (Wish I had the cash for a patent).




Get it here:- EDIT: Link Removed

[StainlessS]

RT_Stats v1.40 Beta02,

Main changes,
LumaTol change from Float to Int.
Added Exit Condition RT_LumaPixelsDifferent arg PCThresh.
Added BEST MATCH status when Function returns -1 NOT FOUND and EXIT CONDITION Flags set.

RT_QwikScanGetNear(clip DBClip,string DB,string PNDB,int StartFrame,int FindFrame, \
clip "FindClip"=NULL,int "Maxdistance"=DBClip.FrameCount-1,bool "Inclusive"=True,Int "LumaTol"=0, \
int "Flags"=0, Float "LD"=0.0,Float "LC"=0.0,Float "FD"=0.0,int "PC"=0,int "XP"=1, \
string "Prefix"="QSGN_",Float "ChromaWeight"=1.0/3.0,Int "PCThresh"=0)

*** EXPERIMENTAL ***

Developer function only, not for the casual scriptor.
Function that scans a pair of database files to find a sequence of candidate frames that could match a target FindFrame.
Returns frame number of possible matching frame, or -1 meaning NOT FOUND.
DBClip, the clip to search, and used as arg to RT_QwikScanCreate to create DB, PrevDB and NextDB database files that hold
"FingerPrint" and frame location data to assist in rapid location of candidate frames.
DB, Filename of the "FingerPrint" Dbase. Created via RT_QwikScanCreate.
PNDB, Filename of the frame locator Dbase. Created via RT_QwikScanCreate as either PrevDB or NextDB.
Use the PrevDB if searching frames prior to StartFrame in DBClip, or the NextDB if search frames after StartFrame.
StartFrame, frame in DBClip where searching will start, see also Inclusive.
FindFrame, Frame number of the frame that is to be matched (in FindClip if supplied or DBClip if not supplied).
FindClip, clip holding frame to match, defaults to DBClip if not supplied.
MaxDistance, maximum number of frames to search, defaults DBClip.FrameCount -1, but silently limited to the maximum possible
distance depending upon where StartFrame is and whether searching forwards or backwards. A MaxDistance of 1 will search only
the StartFrame (if Inclusive) and the frame next to it.
Inclusive. If true (default) then also tests StartFrame.
LumaTol, +- Luma Tolerance (0 -> 16). Allow for mismatch in luma between DBClip and FindClip. If eg the FindClip pixels can vary
by as much as +- 4 pixel levels, then set LumaTol to 4, the bigger this number the slower the function, but it should be still
quite fast with most clips even up to a LumaTol of maximum 16, the bigger this number, the more frames will fit the fingerprint
and so there will be more candidate frames to check.
Flags, Bit flags, sets the additional Exit Conditions of the function.
Bit 0 set eg 0x01, Sets the LumaDifference Exit Condition.
Bit 1 set eg 0x02, Sets the LumaCorrelation Exit Condition.
Bit 2 set eg 0x04, Sets the FrameDifference Exit Condition.
Bit 3 set eg 0x08, Sets the PixelsDifferentCount Exit Condition.
LD, LumaDifference Exit Condition Threshold.
LC, LumaCorrelation Exit Condition Threshold.
FD, FrameDifference Exit Condition Threshold.
PC, PixelsDifferentCount Exit Condition Threshold (int). See Also PCThresh.
XP, Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=Continue searching while next frame is a better Exit Condition match.
2=Continue searching while next frame is a better or equal Exit Condition match.

Prefix, Default "QSGN_". Prefix for names of local variables returned to give status of exit conditions.
All Below only set if Flags arg non zero.
QSGN_Exit_Flags, bit flag, condition that was satisfied to cause exit (same as Flags arg, only 1 bit will be set).
Below Only set if QSGN_Exit_Flags != 0 (where function returns a valid candidate frame number).
QSGN_Exit_LD, LumaDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x01.
QSGN_Exit_LC, LumaCorrelation for the exit frame, Only set if QSGN_Exit_Flags == 0x02.
QSGN_Exit_FD, FrameDifference for the exit frame, Only set if QSGN_Exit_Flags == 0x04.
QSGN_Exit_PC, PixelsDifferentCount for the exit frame, Only set if QSGN_Exit_Flags == 0x08.
QSGN_Exit_XP, Count of additional frames traversed if found to be a better match. Only set if both XP arg is non zero
Below Only set if Function returns -1 NOT FOUND, AND QSGN_BM_Flags != 0 (QSGN_Exit_Flags will be 0 if NOT FOUND)
QSGN_BM_LD, LumaDifference of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x01.
QSGN_BM_LD_Frm, LumaDifference BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x01.
QSGN_BM_LC, LumaCorrelation of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x02.
QSGN_BM_LC_Frm, LumaCorrelation BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x02.
QSGN_BM_FD, FrameDifference of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x04.
QSGN_BM_FD_Frm, FrameDifference BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x04.
QSGN_BM_PC, PixelsDifferentCount of the BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x08.
QSGN_BM_PC_Frm, PixelsDifferentCount BEST MATCHING frame, Only set if QSGN_BM_Flags == 0x08.

If the function returns -1 NOT FOUND, but an EXIT condition was set and at least one candidate frame was found but did
not satisfy any EXIT condition, then the BEST MATCH flags will be set (if none set then no candidate frames were found
that were within LumaTol of the target and so no EXIT conditions were tested). If QSGN_BM_Flags is non zero, it should be
exactly the same as Flags arg, with all selected BEST MATCH status variables returned as noted above.
Where DBClip and FindClip are quite different in eg brightness, It may be possible to have some kind of self tuning
routine. Find a single (or multiple) frames with maximum LumaTol, and EXIT conditions that are sure to fail, and use the
BEST MATCH results to tune both LumaTol and the EXIT conditions, for the remainder of the search session.
It might though be just as good to do some kind of Luma levelling on each clip, based on a sample number of frames
and using YPlaneMin and YPlaneMax followed by Levels (With Coring=False, and before DBase creation of course).

ChromaWeight, Argument to the FD FrameDifference Exit Condition (YUV only).

PCThresh, Range 0 -> 254. Argument to the PC PixelsDifferentCount Exit Condition. Difference between individual pixels has to be
greater than this to be counted. (default=0, count all differences).

The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
The PNDB Frame Locator Dbase holds records with 1024 fields per record (frame). The first 256 of these fields give frame numbers
of the next nearest frame (previous[PrevDB] or next[NextDB]) that roughly complies with the first component FingerPrint value.
Eg, field 42 holds the frame number of the nearest frame whose first component FingerPrint value is greater or equal to 42.0
and less than 43.0. Fields 256 to 511 hold frame info for the 2nd FingerPrint component, etc. A value of -1 means that there is
no frame whose fingerprint component complies.

This is roughly how the function works without Exit Conditions (Flags=0):-
It first establishes a range of component FingerPrint values for FindFrame, based on LumaTol.
If Inclusive, then checks StartFrame for a FingerPrint match and returns StartFrame if good match.
Next it Checks the Frame Locator Dbase (PNDB) looking for the next nearest frame that roughly complies with the FingerPrint
range. It then checks that candidate frame in the FingerPrint DBase to see if it more exactly matches the FingerPrint, if it
does match, the frame is returned to the client and if not then it moves the 'Cursor' in the Frame Locator DBase up to that
frame and repeats the searching until found or not.
So, without Exit conditions, the function returns a sequence of candidate frame numbers that may or may not be the exact frame
you are looking for, it will likely skip vast swathes of frames and then deliver a stream of frames from the same scene as
your FindFrame, where you would likely want to establish a more exact match using some other function.

With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a candidate frame
to the client. If an Exit condition is set, then that exit condition also has to be satisfied, if not satisfied, then the candidate
frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and the Exit
Condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, they are processed in bit number order ie LumaDifference condition first, the first
condition that succeeds will cause exit and no further exit conditions will be processed. If an Exit Condition is satisfied,
and the XP arg is set, the function will continue scanning adjacent frames and step forward until a better match for the Exit
Condition is NOT found. (XP=1 is better than, and XP=2 is better or equal).

Exit Conditions:-

LD, LD is the Threshold for RT_LumaDifference, if the match is less or equal to this then the Exit Condition is satisfied.

LC, LC is the Threshold for RT_LumaCorrelation, if the match is less or equal to this then the Exit Condition is satisfied.
We have 'swizzled' the values returned by RT_LumaCorrelation(), instead of the normal range -1.0 -> 1.0,
we use 0.0 -> 255.0 where 0.0 means exact match. We have done this for 2 reasons. Firstly, to make it the same (roughly)
as the other exit conditions, the closer to zero, the better the match. 2nd reason is, Avisynth Floats have only 6 significant
digits of decimal precision, so eg 0.999999 is about a good as it gets (closest to 1.0 without actually being 1.0) if we were
setting a threshold for normal range LumaCorrelation (-1.0 -> 1.0), but when we 'swizzle' the numbers to make 0.0 an exact
match, we can supply a more precise threshold which will be used at double precision within the function,
eg 0.0000000000000000000000000000001 is still a valid float (and in this case it is only 1 significant digit of precision).
RT_LumaCorrelation is slower than RT_LumaDifference but it may be more appropriate if there is a big difference in
luma beteen DBClip and FindClip, it is less perturbed by difference in brightness or contrast.

FD, FD is the Threshold for RT_FrameDifference, if the match is less or equal to this then the Exit Condition is satisfied.
The RT_FrameDifference Exit Condition uses the ChromaWeight arg to weight chroma for YUV colorspaces.
For RGB, RT_FrameDifference is equivalent to RGBDifference. For Y8 it is equivalent to LumaDifference.
For non Y8 YUV, returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
EXPERIMENTAL: Also currently accepts ChromaWeight -ve (eg -1.0) where will return
Lumadif + (1.0 - Lumadif/255.0) * ((Udif + Vdif)/2.0)
This may well be removed at some future date (although would do no harm to leave in-situ).

PC, PC is the Threshold for RT_LumaPixelsDifferent, if the match is less or equal to this then the Exit Condition is satisfied.
PC is a pixel count, ranging 0 -> (Width * Height). Allows you to set maximum number of pixels that are different.
Also takes PCThresh as an argument, difference between individual pixels has to be greater than this to be counted.
(PCThresh default=0, count all differences).
(RGB converted to Luma-Y).

Usage:-
Using PAL SD Jurassic Park with 182050 frames (UT_Video Compressed YV12) it took 15:06 to create DB and PrevDB FingerPrint
and Locator DB on a 2.13Ghz Core Duo, using this script.

AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
return RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)

And test script searching approx 175,000 frames per FindFrame:
AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="PREV_JPARK.DB"
#RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)
GScript("""
LUMATOL=0
End=FrameCount-1
For(i=3000,13000,1000) {
S=RT_TimerHP()
j = RT_QwikScanGetNear(Last,DB,PNDB,End,i,Last,maxdistance=End-i,lumatol=LUMATOL,flags=$04,fd=0.0,xp=2)
T=RT_TimerHP-S
RT_Debugf("%d] Got %d in %f secs",i,j,T)
}
""")
return Last

Results:- LumaTol = 0
00000005 398.69018555 RT_DebugF: 3000] Got 3000 in 0.019132 secs
00000006 398.70736694 RT_DebugF: 4000] Got 4000 in 0.017023 secs
00000007 398.73089600 RT_DebugF: 5000] Got 5000 in 0.023335 secs
00000008 398.75195313 RT_DebugF: 6000] Got 6000 in 0.020897 secs
00000009 398.76660156 RT_DebugF: 7000] Got 7000 in 0.014457 secs
00000010 398.78686523 RT_DebugF: 8000] Got 8000 in 0.020073 secs
00000011 398.80563354 RT_DebugF: 9000] Got 9000 in 0.018632 secs
00000012 398.82150269 RT_DebugF: 10000] Got 10000 in 0.015690 secs
00000013 398.85241699 RT_DebugF: 11000] Got 11000 in 0.030706 secs
00000014 398.86682129 RT_DebugF: 12000] Got 12000 in 0.014216 secs
00000015 398.88012695 RT_DebugF: 13000] Got 13000 in 0.013135 secs

Results:- LumaTol = 4 (many more candidate frames to test)
00000016 444.04776001 RT_DebugF: 3000] Got 3000 in 0.059216 secs
00000017 444.26177979 RT_DebugF: 4000] Got 4000 in 0.213837 secs
00000018 446.82482910 RT_DebugF: 5000] Got 5000 in 2.562858 secs
00000019 454.65585327 RT_DebugF: 6000] Got 6000 in 7.830853 secs
00000020 456.44543457 RT_DebugF: 7000] Got 7000 in 1.789409 secs
00000021 465.26766968 RT_DebugF: 8000] Got 8000 in 8.822010 secs
00000022 465.99948120 RT_DebugF: 9000] Got 9000 in 0.731642 secs
00000023 466.64910889 RT_DebugF: 10000] Got 10000 in 0.649458 secs
00000024 471.09573364 RT_DebugF: 11000] Got 11000 in 4.446451 secs
00000025 471.81393433 RT_DebugF: 12000] Got 12000 in 0.718018 secs
00000026 475.26538086 RT_DebugF: 13000] Got 13000 in 3.451269 secs


Get it here:- http://www.mediafire.com/download/jn3l36pb9zo63cl/RT_Stats_25%2626_dll_v1.40Beta02_20140529.zip

[jmac698]

Hi,
Your RT_Stats has really grown! I'm quite impressed, there's a lot of useful stuff in there. I made good use of it in http://forum.doom9.org/showthread.php?t=170995, but I noticed a few things. Your idea to convert a float to a colour was useful to me, but you didn't complete the idea - you need a way to store/plot the colour into a frame. I came up with a way to plot with layer but it's slow and limited. I've always wanted to operate on images as arrays.

What I would like is something like RT_PutImageAsArrayByte(clip, frame, value, index), RT_GetImageAsArrayByte(clip,frame,value,index), PutImageAsArrayInt(clip, frame, value, index) etc.

These are basically plot functions. Index scans left-to-right and top-to-bottom. This is determined from the clip size. So an 8x8 clip hold 64 values. You could also write a script function to turn it into a (luma) plot. Int would naturally be stored as YU then YV, which is a bit awkward for (writing a function to make a colour) plot, so maybe you could add a real plot as well :)


Another thing I noticed is that the timer doesn't work for the "compile" phase of the script, which is where all my loops are run, so I can't time them. I suppose avstimer is the solution.

[StainlessS]

Already answered (+ve-ley) in your thread.

[StainlessS]

Another thing I noticed is that the timer doesn't work for the "compile" phase of the script, which is where all my loops are run, so I can't time them. I suppose avstimer is the solution.

No idea what you mean, I use it at compile time all of the time.


START=RT_TimerHP() # or RT_Timer(), matched pairs ONLY.
...
END=RT_TimerHP()
TIME_TAKEN = END - START # In seconds


EDIT:

RT_Timer()
Returns time in seconds since start of process.
Usage:
s=RT_Timer()
Test()
e=RT_Timer()
Str="Test Start="+String(s,"%.2f")+" End="+String(e,"%.2f")+" Time="+String(e-s,"%.2f")+"secs"
SubTitle(str)
RT_Debug(str)


RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elapsed in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only thing that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str="Test Start="+String(s,"%.3f")+" End="+String(e,"%.3f")+" Time="+String(e-s,"%.6")+"secs"
SubTitle(str)

[Seedmanc]

Is there any way to measure rendering speed of a script inside of it, using RT? To disable heavy parts of processing dynamically, for example.
I came up with something quick and dirty like this without gscript
pre=0.0
post=0.0
frames=0.0
fps=0.0
scriptclip("
\ post=RT_Timer()
\ time=post-pre
\ pre=(time>=1)?post:pre
\ fps=(time>=1)?current_frame-frames:fps
\ frames=(time>=1)?current_frame:frames
\RT_Subtitle(string(fps))")
It works, but I'm sure there should be a way to make it look less ugly, maybe even an inbuilt function.

Also, another question, how heavy are RT functions? I read somewhere that they are generally slower than similar (if you can even compare) non-runtime functions given their execution for every frame.
The reason I'm asking both questions is that I'm looking for ways of optimizing a somewhat exotic way of using Avisynth - real-time processing of video, captured from a videogame for output on a larger screen. Thus the speed and lag introduced by a script matters. Here's my thread about it, if you wonder: http://forum.doom9.org/showthread.php?t=171105

[StainlessS]

How about this

Avisource("Test.avi")

Start = RT_Timer() # Timing relative to start ie average timing for entire length of clip so far.
CondS = """
T=RT_Timer
RT_Subtitle("%.3f",current_frame/Max(T-Start,0.000001))
Return Last
"""
ScriptClip(CondS)


To disable heavy parts of processing dynamically

What do you mean by that ?

You cannot generally compare runtime functions with filters, they do a different job.
eg, AverageLuma() returns a float, not a clip.
Some filter functions may eg return the same value as AverageLuma would but as a local variable rather than a function return (ShowChannels can do that).
Any standard filter used in eg Scriptclip, has it's constructor called at every frame, and also destroyed at every frame, and so filters that have 'heavy'
start up costs, have them at every frame.

[StainlessS]

New version RT_Stats v1.40.
Post #1 of 3

Functions added.

RT_WriteFile(String FileName, string format,dat1,...,datn,bool "Append"=False)
Writes a formatted string to FileName file. The unnamed 'format' string and optional unnamed 'dat' args are used to construct the
text string that is written, uses C/CPP printf() style formatting.
Format: compulsory string controlling format and describing the datn type args that are expected.
datn: Variable number of data args of any type (excluding clip).
Append, Default False, optional and MUST be supplied with name (eg Append=True) as we do not know where optional data args end,
Appends to file if already exists, else opens new FileName.

printf Format spec here:- http://msdn.microsoft.com/en-us/library/56e442dc%28v=vs.71%29.aspx
NOTE, the only support for printing Bool variables is %s as string, ie prints "True" or "False".
Formatting supported %[flags] [width] [.precision] type
flags, one of "-,+,0, ,#"
width, integer, "*" supported (width supplied via dat arg).
Precision, integer, "*" supported (precision supplied via dat arg).
type,
"c,C,d,i,o,u,x,X", Integer type, c,C=character, d,i=signed, o,u,x,X=unsigned (o=octal, x=Hex).
"e,E,f,g,G", Floating point type
"s,S", String type (also Bool).

Formatting Insertion point is marked with '%' character in the format string (as in Avisynth String function), if you wish to use
a percent character within the output string, it should be inserted twice in the format string ie '%%' will produce a single '%'.
The data datn arg strings do not require a double '%' character.

A Backslash character '\' introduces escape sequences, to insert a backslash character itself, you must supply a double
backslash sequence ie '\\'.
Converts embedded escape character sequences (Case Significant):-
'\\' Converted to '\' Single Backslash
'\a' Converted to Chr(7)
'\b' Converted to Chr(8)
'\t' Converted to Chr(9)
'\n' Converted to Chr(10) NewLine
'\r' [and Chr(13)] Converted to Chr(10) NewLine
'\v' Converted to Chr(11)
'\f' Converted to Chr(12)
'\x', where x is ANY OTHER CHARACTER not included above, will be copied verbatim, ie '\x'.

Escape sequences are replaced only when occurring in the format string, if you need escapes in data string then use RT_String.

Example:
RT_WriteFile(FileName,"Hello there %s and %s.\nGoodbye %d.","Fred","Ted",2013,Append=False)

***
***
***

RT_ChanAve(clip c,int "n"=current_Frame,string "Prefix"=RCA_")
Sets channel averages for clip c frame n, as local variables, 0.0 -> 255.0. RGB24/32, YUY2, Planar.
Returns the number of channels in clip ie 3 unless Y8 where returns 1.
Default Prefix is "RCA_" so for YV12, would return 3 (3 channels) and set
YUV
RCA_Ave_0 = Luma Ave
RCA_Ave_1 = U Ave
RCA_Ave_2 = V Ave.
RGB,
RCA_Ave_0 = Red Ave
RCA_Ave_1 = Grn Ave
RCA_Ave_2 = Blue Ave.
Y8
RCA_Ave_2 = Luma Ave
RCA_Ave_2 = 128
RCA_Ave_3 = 128,
NOTE, No support for alpha channel of RGB32, although could be added if anybody thought it useful.(returns num of channels = 3)

***
***
***

RT_FrameDifference(clip c,clip c2,int "n"=current_frame,int "n2"=current_frame,Float "ChromaWeight"=1.0/3.0)
Returns difference between clip c frame n and clip c2 frame n2 (0.0 -> 255.0).
ChromaWeight range 0.0 -> 1.0.
For RGB, returns same as RGBDifference.
For Y8 returns same as LumaDifference.
For non Y8 Planar returns Lumadif + Chromadif with chroma weighted by ChromaWeight arg.
Returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).

***
***
***

RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=current_frame,int "x2"=0,int"y2"=0,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns amount of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Thresh Default 0, returns number of pixels that are not exactly the same.
Return value is in range 0.0 (meaning none) to 255.0 meaning 100% of pixels.

***
***
***

RT_LumaPixelsDifferentCount(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
int "n2"=current_frame,int "delta2"=current_frame,int "x2"=0,int"y2"=0,bool "interlaced"=false, \
int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns number of pixels
whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
Thresh Default 0, returns number of pixels that are not exactly the same.
Note, returns the number of pixels whose difference is greater than Thresh, so ranges 0 -> (Width*Height).

***
***
***

RT_TimerHP()
Returns a higher precision time than RT_Timer (If available on your system, otherwise uses same routine as RT_Timer).
Note, where RT_Timer returns time elapsed in seconds since start of process, RT_TimerHP is not defined to return anything
in particular, the only thing that is meaningful is the difference between returns from 2 calls to RT_TimerHP.
DO NOT mix times from RT_Timer with RT_TimerHP, results may be meaningless.
Usage:
s=RT_TimerHP() Test() e=RT_TimerHP()
Str="Test Start="+String(s,"%.3f")+" End="+String(e,"%.3f")+" Time="+String(e-s,"%.6")+"secs"
SubTitle(str)

***
***
***

RT_SignalDAR2(int darx,int dary)
As for RT_SignalDAR() except it allows setting of numerator and denominator individually.
Signal DAR to MEGUI. Sets Global vars MeGUI_darX and MeGUI_darY which are read during MEGUI loading of AVS file.

***
***
***

RT_VarType(var)
Returns type of var arg as an int.
Returns,
0 = Type BOOL
1 = Type INT.
2 = Type FLOAT
3 = Type STRING
4 = Type CLIP
-1 = Type UNDEFINED
Avoids multiple tests in script to establish a variable type.
eg, RT_VarType(66.6) returns 1, ie FLOAT.
NOTE, If called without an arg, it will return type of implicit Last (Undefined if no clip implicitly/explicitly assigned to Last).

***
***
***

RT_VarIsSame(var Var1, ...,var Varn,Bool "Sig")
Accepts two or more variables of any type. NOTE, if only 1 arg supplied, will use implicit Last as first arg.
Returns Bool, False if args are not all of same type AND/OR value.
Return True, if all args same type and value.
Sig, Bool Default True. Only used where variables are either String or Clip.
Type String,
Compares strings same length and contents.
Sig==True, case significant comparison ('a' != 'A').
Sig==False, case insignificant comparison ('a' = 'A').
Type Clip,
Compares clip video attributes, frame Width, Height, ColorSpace.
If first clip has no video (HasVideo==False) then all other clips must also have no video (other video attributes not tested).
If Sig==False, Audio Not considered in comparison.
If Sig==True, audio is considered.
If first clip has no audio (HasAudio==False) then all other clips must also have no audio (other audio attributes not tested).
If first clip does have audio, then all other clips must have audio and same Number of Audio Channels, Sample Rate,
and Sample Type (int16, Float etc).
No other attributes are taken into account, (ie FrameRate, FrameCount etc).
*** NOTE ***, as we cannot tell where variable number of Varn args ends, you must use optional 'sig' arg with the name ie 'sig=True'.

***
***
***

RT_DBaseGetStrAttrib(String Filename,Int ix)
There are 10 (currently) available String attributes in the DataBase file header, that can be used for whatever purpose you like, they are
1024 (maximum) characters in length excluding nul terminating character (nul not stored in file).
Filename, DBASE Filename, no default.
Ix, String Attribute array element index range 0 -> 9.
Returns the value of the user set string attribute.
NOTE, if supplied ix is -1, then will return the number (Int) of available string attributes (currently 10).
NOTE, if supplied ix is -2, then will return the maximum string length allowed (Int) of string attributes (currently 1024).

***
***
***

RT_DBaseSetStrAttrib(String Filename,Int ix, String s)
There are 10 (currently) available string attributes in the DataBase file header, that can be used for whatever purpose you like,
RT_DBaseAlloc() initializes all String Attributes to "", ie zero length.
Filename, DBASE Filename, no default.
Ix, Attribute array element index range 0 -> 9.
s, The string that String_Attrib(ix) will be set to.
Returns 0. Error if type not of type String.

***
***
***

RT_DBaseTypeName(Int Type)
Type, Int (0 -> 5), as value returned by RT_DBaseFieldType(Filename,Field), returns the type as a string.
Returns 0="Bool", 1="Int", 2 = "Float", 3 = "String", 4 = "Bin", 5 = "Double".
NOTE, type 5 "Double" is an RT_Stats Private type, ie used internally by RT_Stats to store floating point values as double,
if used from within Avisynth script, will get and set values as Type Float (but stored in DB as double, 8 bytes instead of 4).
There is little point to using type double in avisynth script, will just make files bigger and take a little longer to access.

***
***
***

RT_ArrayGetStrAttrib(String Filename,Int ix)
There are 10 (currently) available String attributes in the Array file header, that can be used for whatever purpose you like, they are
1024 (maximum) characters in length excluding nul terminating character (nul not stored in file).
Filename, Array Filename, no default.
Ix, String Attribute array element index range 0 -> 9.
Returns the value of the user set string attribute.
NOTE, if supplied ix is -1, then will return the number (Int) of available string attributes (currently 10).
NOTE, if supplied ix is -2, then will return the maximum string length allowed (Int) of string attributes (currently 1024).

***
***
***

RT_ArraySetStrAttrib(String Filename,Int ix, String s)
There are 10 (currently) available string attributes in the Array file header, that can be used for whatever purpose you like,
RT_ArrayAlloc() initializes all String Attributes to "", ie zero length.
Filename, ARRAY Filename, no default.
Ix, Attribute array element index range 0 -> 9.
s, The string that String_Attrib(ix) will be set to.
Returns 0. Error if type not of type String.

***
***
***

RT_ArrayTypeName(Int Type)
Type, Int (0 -> 5), as value returned by RT_DBaseFieldType(Filename,Field), returns the type as a string.
Returns 0="Bool", 1="Int", 2 = "Float", 3 = "String", 4 = "Bin", 5 = "Double".
NOTE, type 5 "Double" is an RT_Stats Private type, ie used internally by RT_Stats to store floating point values as double,
if used from within Avisynth script, will get and set values as Type Float (but stored in DB as double, 8 bytes instead of 4).
There is little point to using type double in avisynth script, will just make files bigger and take a little longer to access.

***
***
***

RT_FileRename(String OldFilename,String NewFileName)
Renames a file from OldFilename to NewFilename.
This function renames the file or directory specified by OldFilename to the name given by NewFilename.
The OldFilename must be the path of an existing file or directory.
The NewFilename must not be the name of an existing file or directory.
You can use the function to move a file from one directory or device to another by giving a different path in the NewFileName argument.
However, you cannot use the function to move a directory. Directories can be renamed, but not moved.

Return 0 on success.
On Error, returns
-1, Access denied or read only, or NewFileName already exists or could not be created (invalid path),
or oldname is a directory and newname specifies a different path.
-2, OldFilename File or Path not found.
-3, NewFileName contains invalid characters.
-4, Unknown error.

***
***
***

RT_QwikScanCreate(clip c,string DB,string "PrevDB"="",string "NextDB"="",int "matrix"=(Width>1100||Height>600?3:2),bool "Debug"=False)
Creates a DataBase set used to quickly scan a clip for frame matching, DataBases used by RT_QwikScan.
c, clip used to create database frame fingerprint info.
PrevDB, DBase filename, used to find frames prior to a 'start frame'.
NextDB, DBase filename, used to find frames after a 'start frame'.
Either PrevDB or NextDB (or both) must be given.
Matrix, used to convert RGB to Luma_Y for Fingerprint creation.
Debug, True sends a little info to DebugView window (google).
Maximum PrevDB/NextDB size is restricted to 524283 records(frames).

***
***
***

[StainlessS]

New version RT_Stats v1.40.
Post #2 of 3

RT_QwikScan(clip SearchClip,Int SeachStart,clip FindClip,Int FindFrame,string DB,string PNDB,
\ Int "LumaTol"=1,int "Flags"=1,
\ Float "LC"=1.0,Float "LD"=1.0,Float "FD"=1.0,Float "PD"=1.0,int "PC"=(SearchClip.Width*SearchClip.Height+128)/255, int "XP"=1,
\ Float "ChromaWeight"=1.0/3.0,Int "PDThresh"=0
\ int "Maxdistance"=SearchClip.FrameCount-1,bool "Inclusive"=True,string "Prefix"="QWKS_")

Developer function only, not for the casual scriptor.
Function that scans a clip using pair of database files to find a sequence of candidate frames that could match a target FindFrame.
Returns frame number of possible matching frame, or -1 meaning NOT FOUND.

ARGS:-

SearchClip, the clip to search, and used as arg to RT_QwikScanCreate to create DB, PrevDB and NextDB database files that hold
"FingerPrint" and frame location data to assist in rapid location of candidate frames.

SearchStart, frame number in SearchClip where searching will start, see also Inclusive.

FindClip, clip holding FindFrame to match.

FindFrame, Frame number of the FindClip frame that is to be matched.

DB, Filename of the 'FingerPrint' Dbase. Created via RT_QwikScanCreate.

PNDB, Filename of the frame locator Dbase. Created via RT_QwikScanCreate as either PrevDB or NextDB.
Use the PrevDB if searching frames prior to SearchStart in SearchClip, or the NextDB if search frames after SearchStart.
See also 'Inclusive'.

OPTIONAL ARGS:

LumaTol (default 1), +- Luma Tolerance (0 -> 16). Allow for mismatch in luma between SearchClip and FindClip. If eg the FindClip
pixels can vary by as much as +- 4 pixel levels, then set LumaTol to 4, the bigger this number the slower the function, but it
should be still quite fast with most clips even up to a LumaTol of maximum 16, the bigger this number, the more frames will fit
the fingerprint and so there will be more candidate frames to check.

Flags (Int, default=1 ie LumaCorrelation Exit Condition), Bit flags, sets the additional Exit Conditions of the function.

Bit 0 set eg $01, Sets the LumaCorrelation Exit Condition.
Bit 1 set eg $02, Sets the LumaDifference Exit Condition.
Bit 2 set eg $04, Sets the FrameDifference Exit Condition.
Bit 3 set eg $08, Sets the PixelsDifferent Exit Condition.
Bit 4 set eg $10, Sets the PixelsDifferentCount Exit Condition.

To eg set Exit conditions LumaCorrelation + FrameDifference + PixelsDifferent, then add 1 + 4 + 8 (1+4+8=13=$0D).

If exit conditions set, then there may be info generated in the Exit and Best Match returned local variables.

LC, LumaCorrelation Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 0 of Flags is set.
LC is the Threshold for RT_LumaCorrelation, if the match is less or equal to this then the Exit Condition is satisfied.
We have 'swizzled' the values returned by RT_LumaCorrelation() here to something akin to half range "Pearson's Distance".
See Pearson’s Distance here: http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
where Pearson's Distance is equivalent to:

PD(x,y) = 1.0 - Correlation(x,y)

and our Swizzled half range version using equivalent to:-

SwizzledLC(x,y) = Min( (1.0 - RT_LumaCorrelation(x,y)) * 255.0, 255.0 )

Correlation Pearsons_Distance Swizzled_LC
1.0 0.0 0.0 Total +ve Correlation
0.0 1.0 255.0 UnCorrelated
-1.0 2.0 255.0 Total -ve Correlation

so instead of the RT_LumaCorrelation normal range -1.0 -> 1.0, we use 0.0 -> 255.0 where 0.0 means exact match and 255.0 means
unmatched, Swizzled RT_LumaCorrelation results that were -ve are clipped at 255.0 so we consider Total -ve Correlation to be no
more distant than UnCorrelated, ie photo invert of find frame cannot be more distant than 'totally unmatched'.
We have done this for 2 reasons. Firstly, to make it the same (roughly) as the other exit conditions, the closer to zero, the
better the match. 2nd reason is, Avisynth Floats have only 6 significant digits of decimal precision. So eg 0.999999 is about as
good as it gets (closest to 1.0 without actually being 1.0). When we 'swizzle' the numbers to make 0.0 an exact match, we can
supply a more precise threshold which will be used at double precision within the function.
Eg 0.0000000000000000000000000000001 is still a valid float (and in this case it is only 1 significant digit of precision).
Despite the swizzling, we will keep the name LC LumaCorrelation (PD already taken).
LumaCorrelation is slower than LumaDifference but it may be more appropriate if there is a big difference in
luma beteen SearchClip and FindClip, it is less perturbed by difference in brightness or contrast.
See RT_LumaCorrelation.

LD, LumaDifference Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 1 of Flags is set.
LD is the Threshold for RT_LumaDifference, if the match is less or equal to this then the Exit Condition is satisfied.
See RT_LumaDifference.

FD, FrameDifference Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 2 of Flags is set.
FD is the Threshold for RT_FrameDifference, if the match is less or equal to this then the Exit Condition is satisfied.
The RT_FrameDifference Exit Condition uses the ChromaWeight arg to weight chroma for YUV colorspaces.
For RGB, RT_FrameDifference is equivalent to RGBDifference. For Y8 it is equivalent to LumaDifference.
For non Y8 YUV, returns:- (1.0 - ChromaWeight) * Lumadif + ChromaWeight * ((Udif + Vdif)/2.0).
See RT_FrameDifference.

PD, PixelsDifferent Exit Condition Threshold (Float, default 1.0, Range 0.0 -> 255.0).
Selected when bit 3 of Flags is set.
PD is the Threshold for RT_LumaPixelsDifferent, if the match is less or equal to this then the Exit Condition is satisfied.
PD is amount of pixels, that are different with 0.0 meaning none and 255.0 meaning 100%.
Also takes PDThresh as an argument, difference between individual pixels has to be greater than this to be counted.
PixelsDifferent is similar to below PixelsDifferentCount, but using a frame size relative way of specifiying the exit condition.
(PDThresh default=0, count all differences). (RGB converted to Luma-Y).
See RT_LumaPixelsDifferent and also PDThresh.

PC, PixelsDifferentCount Exit Condition Threshold (int, default=(SearchClip.Width*SearchClip.Height+127)/255,
Range 0 -> SearchClip.Width*SearchClip.Height).
Selected when bit 4 of Flags is set.
PC is the Threshold for RT_LumaPixelsDifferentCount, if the match is less or equal to this then the Exit Condition is satisfied.
PC is a pixel count, ranging 0 -> (Width * Height). Allows you to set maximum number of pixels that are different.
Also takes PDThresh as an argument, difference between individual pixels has to be greater than this to be counted.
(PDThresh default=0, count all differences). (RGB converted to Luma-Y).
See RT_LumaPixelsDifferentCount and also PDThresh.

XP, (Default 1), Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=Continue searching while next frame is a better Exit Condition match ('next frame' meaning dependant upon search direction).
2=Continue searching while next frame is a better or equal Exit Condition match ('next frame' meaning dependant upon search direction).
When an exit condition occurs eg LumaDifference, then QWKS_BM_LD_XP will be set to the count of additional frames traversed due
to finding a match that is better or equal (depending upon whether XP==1 or XP==2) to the initial frame that caused the exit condition,
In such cases, both QWKS_BM_LD and QWKS_BM_LD_FRM will also be set to (in this case) LumaDifference and the final frame (with XP offset)
that was found to be a better or equal match.

ChromaWeight (Float, default 1.0/3.0, Range 0.0 -> 1.0), Argument to the FD FrameDifference Exit Condition (YUV only).

PDThresh, (Int, Default 0, Range 0 -> 255). Argument to the PD PixelsDifferent and PC PixelsDifferentCount Exit Conditions.
Difference between individual pixels has to be greater than this to be counted. (default=0, count all differences).

MaxDistance (Int, default SearchClip.FrameCount-1). Maximum number of frames to search, defaults SearchClip.FrameCount-1, but silently
limited to the maximum possible distance depending upon where SearchStart is and whether searching forwards or backwards.
A MaxDistance of 1 will search only the SearchStart (if Inclusive) and the frame next to it.

Inclusive, (Bool Default True). If true (default) then also tests SearchStart.

Prefix, (String Default "QWKS_"). Prefix for names of local variables returned to give status of exit conditions and Best Match.

-------------------------------------------------------------------------

RETURNS:-
Returns frame number of CANDIDATE (possible matching frame), or -1 meaning NOT FOUND.

LOCAL VARIABLES returned where using default Prefix of "QWKS_":

QWKS_EXIT_FLAGS,Int, bit flags, (always exists). If Exit condition occurred then there will be one of more bits set for the conditions
that caused exit, bit numbers same as described for Flags arg. QWKS_EXIT_FLAGS will always be 0 when flags arg == 0 (no exit
conditions required) OR, function returned -1, ie frame NOT FOUND.
If exit condition occurred and XP arg is 0, then the function return frame and QWKS_BM_xx_FRM (described below) will be the same,
if XP arg is non zero, then XP processing is tried on all successful exit conditions and the QWKS_BM_xx_FRM frame numbers
(and concerned QWKS_BM_xx differences) may all be different from the function return frame. So, when XP processing, the
function return frame is only the initial frame that caused exit, XP results are in QWKS_BM_xx_FRM and QWKS_BM_xx.

QWKS_EXIT_COUNT, Int (always exists), Count of set bits in QWKS_EXIT_FLAGS (ie number of exit conditions that succeeded).

QWKS_BM_FLAGS,Int, bit flags, (always exists). Returns flag bits of valid BEST MATCHES found for requested exit conditions, bit numbers
are as described for Flags arg. If Flags arg == 0, then QWKS_BM_FLAGS will always be 0, ie no exit conditions were tried.
If one or more exit conditions caused exit then QWKS_BM_FLAGS will have the same bits set as QWKS_EXIT_FLAGS, and possibly other bits
set giving info on best matches found for other exit conditions that did not successfully cause exit.

QWKS_BM_COUNT, Int (always exists), Count of set bits in QWKS_BM_FLAGS. (ie number of Best Match vars set)

Below Only exist/updated if associated QWKS_BM_FLAGS bit is set.
NOTE, the below frame numbers are the frames after XP processing if XP arg non zero. The associated difference is also for the
frame after XP processing is done. If XP arg == 0, then all frame numbers (and associated differences) are for the function result
frame. (The function result frame is the first frame that was encountered that had a successful exit condition and does not have
XP processing applied, the valid frame numbers QWKS_BM_xx_FRM can all be different from the function return frame if better or equal
frames were found via XP processing).

QWKS_BM_LC,(Float) LumaCorrelation of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 0 is set.
QWKS_BM_LD,(Float) LumaDifference of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 1 is set.
QWKS_BM_FD,(Float) FrameDifference of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 2 is set.
QWKS_BM_PD,(Float) PixelsDifferent of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 3 is set.
QWKS_BM_PC,(Int) PixelsDifferentCount of the BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 4 is set.

QWKS_BM_LC_FRM,(Int) LumaCorrelation BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 0 is set.
QWKS_BM_LD_FRM,(Int) LumaDifference BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 1 is set.
QWKS_BM_FD_FRM,(Int) FrameDifference BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 2 is set.
QWKS_BM_PD_FRM,(Int) PixelsDifferent BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 3 is set.
QWKS_BM_PC_FRM,(Int) PixelsDifferentCount BEST MATCHING frame, Only valid if QWKS_BM_FLAGS bit 4 is set.

QWKS_BM_LC_XP,Int. Number of additional frames traversed because a better or equal match was found for LumaCorrelation exit condition.
QWKS_BM_LD_XP,Int. Number of additional frames traversed because a better or equal match was found for LumaDifference exit condition.
QWKS_BM_FD_XP,Int. Number of additional frames traversed because a better or equal match was found for FrameDifference exit condition.
QWKS_BM_PD_XP,Int. Number of additional frames traversed because a better or equal match was found for PixelsDifferent exit condition.
QWKS_BM_PC_XP,Int. Number of additional frames traversed because a better or equal match was found for PixelsDifferentCount exit condition.
Above only exist/updated for conditions that have a bit set in QWKS_BM_FLAGS otherwise may not exist. They will exist
whether or not the associated exit condition succeded or whether XP arg is zero or non zero. In cases where XP arg was 0 or where
associated exit condition failed, or where XP processing failed to find a better or equal match, then the QWKS_BM_xx_XP
variables will contain 0 (will be non zero only where associated exit condition succeded and found a better/equal XP match).

[StainlessS]

New version RT_Stats v1.40 (see 1st post).
Post #3 of 3

RT_QwikScan continued:


If the function returns a valid frame number (greater or equal to 0) and exit conditions were set in Flags arg, then QWKS_EXIT_FLAGS
will be non zero and have 1 bit set for each condition that caused exit, and the best match status variables
will exist/be updated, the other status variables not flagged in QWKS_EXIT_FLAGS but which are flagged in QWKS_BM_FLAGS will also
exist and be valid, but are BEST MATCHES ONLY.

If the function returns -1 NOT FOUND, but an EXIT condition was set in Flags and QWKS_BM_FLAGS is non zero,
then all associated BEST MATCH status variables will exist and be set (associated to the QWKS_BM_FLAGS set bits).

-------------------------------------------------------------------------


FINGERPRINT & LUMATOL:
The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
Each FingerPint component is derived from a pixel count for a portion of the possible 0 -> 255 luma pixel value range.
Lets say that one of the fingerprint components is derived from the 128->191 luma pixel value range, the fingerpint component
is computed for every frame in the SearchClip eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height},
this result is in range 0.0 -> 255.0, with 255.0 being equal to 100%. [There would be another three fingerprint components computed,
in different ranges. We do not actually use range 128->191, and are reluctant to give any real used ranges as they may change, if you
want to see the real current ranges used, then see the current source code]. The 4 fingerprint components are stored in the
FingerPrint DB and are EXACT fingerprint component values. For the FindFrame, we use a FingerPrint component Range with LumaTol
tolerance. For FindClip(FindFrame), we might compute a fingerprint component range for [128->191]:
FingerPrintMin = {luma PixelCount in range [128+LumaTol] to [191-LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
FingerPrintMax = {luma PixelCount in range [128-LumaTol] to [191+LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
So using computed FingerPrintMin and FingerPrintMax for four different ranges as upper and lower limits, if any frame in the
SeachClip (DBase) has all four EXACT FingerPrint components within all four fingerprint component limits, then it is a candidate
frame and eligible for exit condition testing if exit conditions were set, or returned to the client as a result CANDIDATE frame if
exit conditions were not set.

The PNDB Frame Locator Dbase holds records with 1024 fields per record (frame). The first 256 of these fields give frame numbers
of the next nearest frame (previous[PrevDB] or next[NextDB]) that roughly complies with the first component FingerPrint value.
Eg, field 42 holds the frame number of the nearest frame whose first component FingerPrint value is greater or equal to 42.0
and less than 43.0. Fields 256 to 511 hold frame info for the 2nd FingerPrint component, etc. A value of -1 means that there is
no frame whose fingerprint component complies.

Without Exit Conditions:
This is roughly how the function works without Exit Conditions (Flags=0):-
It first establishes a range of component FingerPrint values for FindFrame, based on LumaTol.
If Inclusive, then checks SearchStart for a FingerPrint match and returns SearchStart if good match.
Next it Checks the Frame Locator Dbase (PNDB) looking for the next nearest frame that could comply with the FingerPrint
range. It then checks that candidate frame in the FingerPrint DBase to see if it more exactly matches the FingerPrint, if it
does match, the frame is returned to the client and if not then it moves the 'Cursor' in the Frame Locator DBase up to that
frame and repeats the searching until found or not.
So, without Exit conditions, the function returns a sequence of candidate frame numbers that may or may not be the exact frame
you are looking for, it will likely skip vast swathes of frames and then deliver a stream of frames from the same scene as
your FindFrame, where you would likely want to establish a more exact match using some other function.
Without Exit conditions, QWKS_EXIT_FLAGS and QWKS_BM_FLAGS will both be 0 and no associated exit condition nor Best Match
local variables will be set (may not even exist).

With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a candidate frame
to the client. If Exit condition are set, then at least one of those exit conditions also has to be satisfied, if not satisfied, then
the candidate frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and an Exit
condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, if any (or all) of the exit conditions succeed then there will be a +ve frame number
returned by the function, QWKS_EXIT_FLAGS and QWKS_BM_FLAGS bits will be set, the returned frame number will be the first frame
that satisfied an exit condition, and does NOT include any XP arg processing.
If any exit condition is satisfied, and the XP arg is set, the function will continue scanning adjacent frames to the frame
where an exit condition was satisfied and step onward until a better match for the exit condition is NOT found.
(XP=1 is better than, and XP=2 is better or equal). XP Processing occurs on satisfied exit conditions and the XP processed frame
numbers and differences are returned via QWKS_BM_xx and QWKS_BM_xx_FRM local vars.
When using exit conditions, use the return frame number only as an indicator of success, use the returned local vars irrespective of
whether or not XP processing is done (although if XP==0, then returned frame will be same as QWKS_BM_xx_FRM).

Tuning:
Where there is significant difference between SearchClip and FindClip, it could be problematic to find a suitable value for LumaTol.
If LumaTol is set too high, then each and every search could take much longer to complete, if set too low then will skip over possible
matches as it will see them as not fitting the LumaTol fingerprint range (and so not even included in Best Matches).
Where both matching clip frames are identical (NOT processed/re-encoded) then can use LumaTol=0 (default is 1).
If you eg apply a ColorYUV(Off_y = 4) on exact copy of frame to find in SearchClip, then you would need at least a LumaTol of 4 for
the FingerPrint locator to include the frame as a CANDIDATE frame, (in this case, every single pixel luma value would have had 4 added
to it), ColorYUV(Off_y = -4) would also be detected by LumaTol = 4, and in both cases where using LumaDifference exit condition, you
would need at least a LD of 4.0 to produce a satisfied exit condition, an LD of eg 1.0 would not produce an exit condition but would
give a BEST match for the correct frame, if using an LC exit condition on same clip then result diff would be about 0.0.
We did a test search on SD PAL Jurassic Park frame 173,000, a scene at end of movie where helicopter is flying away into the distance
across the ocean with caption "Directed by Steven Spielberg".
Where find frame had Blur(1.58,1.58) applied (ie maximum Blur) it was found that a LumaTol of 5 was required to enable the Fingerprint
locator to work at all. Even though the original and blurred frame looked reasonably similar, there were obvious differences eg the the
blurred one looked more dull with indistinct text. Tests showed that there were 2 pixels that were different by 75 luma pixel levels
and there were no doubt many more that were different to a lesser degree. So, where processing akin to a maximum blur is suspected, a
rough guess might be about LumaTol=5. Results below where Flags=1, LC=0.0 and StartFrame = 0 (search from frame 0)
(The below FindFrm->FoundFrm lumadiff is carried out on the found frame, not returned from RT_QwikScan).

FindFrm->FoundFrm
LumaTol SECONDS EXIT_FLAGS BM_FLAGS RESULT_FRAME BM_LC RT_LumaDifference
0 0.011211 0 0 -1 --- ---
1 0.593617 0 1 173087 46.869492 12.616102
2 0.411454 0 1 173005 19.927631 7.802255
3 2.059347 0 1 173004 17.460129 7.176220
4 3.024366 0 1 173004 17.460129 7.176220
5 5.267460 0 1 173000 2.454145 2.577775 <---
6 9.544320 0 1 173000 2.454145 2.577775
8 18.132370 0 1 173000 2.454145 2.577775
12 55.960625 0 1 173000 2.454145 2.577775
16 129.069290 0 1 173000 2.454145 2.577775

It may be possible to write a script function to tune for LumaTol, if a search fails (and you know that the frame is present),
it could search on LumaCorrelation LC=0.0 (LC less perturbed by brightness/contrast changes) and step through LumaTol from 0 upwards
until BM_LC no longer changes and QWKS_BM_LC is below some threshold (maybe eg 2.55 ie 1.0%). Also note above that BM_LC is smaller
than RT_LumaDiff(FindFrm->FoundFrm) when LumaTol is at lowest active setting (5).
When acceptable LumaTol is found then might test using eg RT_LumaDifference between FindFrame and FoundFrame and after adding
a little leeway get an approximate value for doing an LD LumaDifference search (if that were what was required exit condition).
NOTE above, even when LumaTol at maximum 16 with timimg about 2 Mins, it is still considerably quicker than approx 18 Mins to
to find Frame 173000 (1hr:55:20) doing a standard LumaDifference search (using MatchFrames.avs), although would also need to
create the DBase set too (only once if multiple frames).
There is much room for research into how best to use this function.

Usage:-

Eg, send to debug window if BEST MATCH for LumaDifference was established (bit 1).
(RT_BitTst(QWKS_BM_FLAGS,1)) ? RT_DebugF("Best Match for LumaDifference = %f @ frame Number %d",QWKS_BM_FD,QWKS_BM_FD_FRM) : NOP
Eg, send to debug window if LumaDifference caused exit(bit 1).
(RT_BitTst(QWKS_EXIT_FLAGS,1)) ? RT_DebugF("LumaDifference Caused EXIT: Diff = %f @ frame Number %d",QWKS_BM_FD,QWKS_BM_FD_FRM) : NOP

Using PAL SD Jurassic Park with 182050 frames (UT_Video Compressed YV12) it took 15:06 to create DB FingerPrint DB and PrevDB
Locator DB on a 2.13Ghz Core Duo, using this script. (NOTE, The time taken to create all three DataBases, is not much different
to the time it would take to scan the clip to find a single frame using usual methods.
EDIT: Test for single frame match using MatchFrames.avs took ~18:20 to locate a single frame, scanning nearly entire clip using
RT_FrameDifference as locator, so if searching for more than one frame it is a small price to pay, and perhaps {as in this case}
even quicker than a standard single frame search (it is quicker to create a DB FingerPrint for one frame than to compare two frames).

AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="JPARK_PREV.DB"
return RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)

# And test script searching approx 175,000 frames per FindFrame:
AVISource("D:\AVS\JurassicPark.avi")
DB ="JPARK.DB"
PNDB="JPARK_PREV.DB"
#RT_QwikScanCreate(Last,DB,prevdb=PNDB,nextdb="",debug=true)
GScript("""
LUMATOL=0
SearchStart=FrameCount-1 # Search direction from end to beginning, using PrevDB.
For(FindFrame=3000,13000,1000) {
S=RT_TimerHP()
j = RT_QwikScan(Last,SearchStart,Last,FindFrame,DB,PNDB,lumatol=LUMATOL,Flags=1,LD=0.0,maxdistance=SearchStart-FindFrame)
T=RT_TimerHP-S
RT_Debugf("%d] Got %d in %f secs",i,j,T)
}
""")
return Last

Results:- LumaTol = 0
00000005 398.69018555 RT_DebugF: 3000] Got 3000 in 0.019132 secs
00000006 398.70736694 RT_DebugF: 4000] Got 4000 in 0.017023 secs
00000007 398.73089600 RT_DebugF: 5000] Got 5000 in 0.023335 secs
00000008 398.75195313 RT_DebugF: 6000] Got 6000 in 0.020897 secs
00000009 398.76660156 RT_DebugF: 7000] Got 7000 in 0.014457 secs
00000010 398.78686523 RT_DebugF: 8000] Got 8000 in 0.020073 secs
00000011 398.80563354 RT_DebugF: 9000] Got 9000 in 0.018632 secs
00000012 398.82150269 RT_DebugF: 10000] Got 10000 in 0.015690 secs
00000013 398.85241699 RT_DebugF: 11000] Got 11000 in 0.030706 secs
00000014 398.86682129 RT_DebugF: 12000] Got 12000 in 0.014216 secs
00000015 398.88012695 RT_DebugF: 13000] Got 13000 in 0.013135 secs

Results:- LumaTol = 4 (many more candidate frames to test)
00000016 444.04776001 RT_DebugF: 3000] Got 3000 in 0.059216 secs
00000017 444.26177979 RT_DebugF: 4000] Got 4000 in 0.213837 secs
00000018 446.82482910 RT_DebugF: 5000] Got 5000 in 2.562858 secs
00000019 454.65585327 RT_DebugF: 6000] Got 6000 in 7.830853 secs
00000020 456.44543457 RT_DebugF: 7000] Got 7000 in 1.789409 secs
00000021 465.26766968 RT_DebugF: 8000] Got 8000 in 8.822010 secs
00000022 465.99948120 RT_DebugF: 9000] Got 9000 in 0.731642 secs
00000023 466.64910889 RT_DebugF: 10000] Got 10000 in 0.649458 secs
00000024 471.09573364 RT_DebugF: 11000] Got 11000 in 4.446451 secs
00000025 471.81393433 RT_DebugF: 12000] Got 12000 in 0.718018 secs
00000026 475.26538086 RT_DebugF: 13000] Got 13000 in 3.451269 secs

[jmac698]

Amazing work, from what I can tell, it's a database to quickly match frames in a large range, even approximately. That could help find the different edits in laser/vhs/dvd copies of a movie, or on a smaller scale, find the dropped/duped frames that always show in multiple passes of captures.

Just an idea for autotuning things in general I had though, I used strong matches to get statistics to tune other matches. For example in an experiment with matching by luma correlation on extremely noisy frames, I found a strong indication of where one frame was different from others in the 5 copies, but somewhat similar to itself in the 5 copies, and between the two thresholds I could find other frames. This then helped where there was less motion and less of a strong indicator that the frame was unique compared to surrounding, but then relied more on the indicator that the same frames correlated strongly.

I mean,
p1a p1b p1c
p2a p2b p2c
p3a p3b p3c
I find that p1b<<>>p1a,p1c and p2b<<>>p2a,p2c and p3b<<>>p3a,p3c,
then p1b=p2b=p3b
In frames d,e,f,
p1e?p1d,p1f but p1e==p2e==p3e
<<>> means strongly not equivalent, = means weakly equivalent, == means strongly equivalent, ? means indeterminate
Maybe this could help somehow.

[StainlessS]

Well have a go Jmac, you can make both PrevDB and NextDB and do bi-directional searches (one at a time), function intended to be quite flexible.
In the no-exit-condition case, it returns a stream of candidate frames where you can do your own comparisons/matching, the
exit conditions are there to save a little time when it can be done automatically, I intend using mixtures of both.
In no-exit-condition mode, you might set up a for/next loop with eg 'j' as loop counter, give j as StartFrame and assign result back to j,
thereby skipping all intervening non-candidate frames.
Could make possible/plausible lots of differing tasks, edits detection, cut and spice where two scenes of separate clips overlap, + lots more
probs I'm sure.

I have not done the histogram thing yet (well I have a version that I hacked up for myself) but have not forgotten, it will come eventually.
I had to get this out of the way so I could finally try to resolve a problem here:- http://forum.doom9.org/showthread.php?t=170332

Its a pig of a prob, and thread that prompted the function.

gotta go, peace bro.

[Seedmanc]

@StainlessS:
It looks like your fps measuring script calculates the running average fps, so the "true" fps is only achieved at the end of the video, while I'm going for "current" fps, measured over a short timespan (1 second, for example).
I formatted my script a bit better to demonstrate what I mean:
timespan=1
pre=0.0
frames=0.0
fps=0.0
FrameEvaluate("""
post = RT_Timer()
tick = (post - pre) >= timespan
pre = tick?
\ post:
\ pre
fps = tick?
\ current_frame - frames:
\ fps
frames = tick?
\ current_frame:
\ frames
""")
scriptclip("rt_subtitle( string( fps / timespan))")

Unfortunately I can't make this script into a function that would return a numerical value of fps calculated over a timespan supplied as a parameter. I think it's because I can't get rid of having to declare those variables before frameevaluate, but without declaring them, I can't use conditional assignment later. Maybe there's any way to rewrite the assignments differently? Basically I only need it happen when the "tick" is true, which means the needed timespan of seconds has passed already. After that I calculate the amount of frames processed over that time and thus get the fps.

As for what I mean by dynamically disabling heavy filters, it's something like this:
last=(MeasureFPS(timespan)<30)?last:SlowFilter()
As I said, I'm working on realtime live processing of video so I need to take speed into consideration to avoid framedrops. Of course something like this wouldn't make any sense in the usual "offline" encoding of video.
Dynamical switching already works for me, it's just I can't make the fps measurement nicely into a function and it makes fps rather irregular in its current implementation.

[StainlessS]

How about this (I'm not sure that its correct)


GSCript("""
Function MyTimer(clip clp,Int HalfFrameRes) {
clp
CurrTime = RT_TimerHP
if(current_frame == 0 || !RT_VarExist("Prev") || Prev!=current_frame-1) {
Global StartFrame = current_frame Global StartFrame2 = current_frame
Global StartTime = CurrTime Global StartTime2 = CurrTime
Global Prev=current_frame
}
FrameCnt = current_frame-StartFrame
TimeExpired = Max(CurrTime-StartTime,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
if(ToggleOnce) {
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
} Else if(Toggle2) {
Global StartFrame = StartFrame2
Global StartTime = StartTime2
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
}
Global Prev=current_frame
Return FrameCnt/TimeExpired
}
""")

Avisource("Test.avi")

ScriptClip("""RT_Subtitle("%.3f",MyTimer(25))""")


And multi Instance version putting sub on frame

Function MyTimer(clip clp,Int "HalfFrameRes") {
myName="MyTimer: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
HalfFrameRes=Default(HalfFrameRes,25)
Assert(HalfFrameRes>0,myName+"HalfFrameRes Must be greater than 0")
FuncS="""
Function MyTimer@@@(clip clp,Int HalfFrameRes) {
clp
CurrTime = RT_TimerHP
if(current_frame==0 || !RT_VarExist("Prev@@@") || Prev@@@!=current_frame-1) {
# Start/Reset timer within script rather than in Multi-instance Global vars init
Global StartFrame@@@ = current_frame Global StartFrame2@@@ = current_frame
Global StartTime@@@ = CurrTime Global StartTime2@@@ = StartTime@@@
Global Prev@@@=current_frame
}
FrameCnt = current_frame-StartFrame@@@
TimeExpired = Max(CurrTime-StartTime@@@,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
# Comment out below Lines
RT_Subtitle("%.3f",FrameCnt/TimeExpired)
RT_Subtitle("%d] %.3f ToggleOnce=%s Toggle2=%s\nStartFrame =%d StartTime =%f\nStartFrame2=%d StartTime2=%f\nFrameCnt=%d TimeExpired=%f",
\ current_frame,FrameCnt/TimeExpired,ToggleOnce,Toggle2,
\ StartFrame@@@,StartTime@@@,StartFrame2@@@,StartTime2@@@,FrameCnt,TimeExpired,align=1)
if(ToggleOnce) {
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
} Else if(Toggle2) {
Global StartFrame@@@ = StartFrame2@@@
Global StartTime@@@ = StartTime2@@@
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
}
Global Prev@@@=current_frame
# EDIT: Uncomment below line if commenting out RT_Subtitles to give Global MeasuredFPS
# First Multi-Instance Variable name will be MeasuredFPS_MTIM_1
# Global MeasuredFPS@@@ = FrameCnt/TimeExpired
Return Last
}
ARGS = "HalfFrameRes"
clp.GScriptClip("MyTimer@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
RT_IncrGlobal("MTIM_InstanceNumber")
InstS = RT_StrReplace(FuncS,"@@@","_MTIM_"+String(MTIM_InstanceNumber))
# RT_TxtWriteFile(InstS,"DEBUG_MTIM_"+String(MTIM_InstanceNumber)+".TXT")
Return GScript(InstS)
}

Avisource("Test.avi")
MyTimer()



EDITED: RT_TimerHP requires RT_Stats v1.40, Can use RT_Timer instead.

[StainlessS]

Dont know if anyone has done a similar function, this seems to yield ApparentFPS {after dupe removal}.
(gives static scene as 0.0 FPS)
Perhaps saves counting duplicates (always a fun pastime).


GSCript("""
Function ApparentFPS(clip clp,Int "HalfFrameRes",Float "DupeThresh",Float "FrameRate",Bool "MaxFPS",Float "ChromaWeight") {
myName="ApparentFPS: "
clp
HalfFrameRes=Default(HalfFrameRes,25)
DupeThresh=Default(DupeThresh,1.0)
FR=Float(Default(FrameRate,clp.FrameRate))
MaxFPS=Default(MaxFPS,False)
ChromaWeight = Float(Default(ChromaWeight,1.0/3.0))
Assert(HalfFrameRes>0,myName+"HalfFrameRes Must be greater than 0")
Assert(DupeThresh>0.0,myName+"DupeThresh Must be greater than 0.0")
Assert(ChromaWeight>=0.0 && ChromaWeight<=1.0,myName+"ChromaWeight 0.0 -> 1.0")
CurrTime = current_frame / FR
if(current_frame==0 || !RT_VarExist("Prev") || Prev!=current_frame-1) {
Global StartFrame = current_frame Global StartFrame2 = current_frame
Global StartTime = CurrTime Global StartTime2 = CurrTime
Global DupeCnt = 0 Global DupeCnt2 = 0
Global MaxApparentFPS = (RT_VarExist("MaxApparentFPS")) ? MaxApparentFPS : 0.0
Global Prev = current_frame
}
Diff = RT_FrameDifference(clp,clp,n2=current_frame-1,ChromaWeight=ChromaWeight)
if(Diff < DupeThresh && current_frame > 0) {
Global DupeCnt = DupeCnt + 1
Global DupeCnt2 = DupeCnt2 + 1
}
FrameCnt = current_frame-StartFrame
UniqueFrameCnt = FrameCnt - DupeCnt
TimeExpired = Max(CurrTime-StartTime,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
ApparentFPS = UniqueFrameCnt/TimeExpired
Global MaxApparentFPS = (ApparentFPS>MaxApparentFPS) ? ApparentFPS : MaxApparentFPS

if(ToggleOnce) {
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
Global DupeCnt2 = 0
} Else if(Toggle2) {
Global StartFrame = StartFrame2
Global StartTime = StartTime2
Global DupeCnt = DupeCnt2
Global StartFrame2 = current_frame
Global StartTime2 = CurrTime
Global DupeCnt2 = 0
}
Global Prev = current_frame
Return (MaxFPS) ? MaxApparentFPS : ApparentFPS
}
""")

#Avisource("Test.avi")
Avisource("CAP1.avi")
MAXFPS=False
ScriptClip("""RT_Subtitle("%.3f",ApparentFPS(maxfps=MaxFPS))""")


Requires RT_Stats v1.40

EDIT: And Multi-Instance version


Function ApparentFPS(clip clp,Int "HalfFrameRes",Float "DupeThresh",Float "FrameRate",Float "ChromaWeight") {
myName="ApparentFPS: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
HalfFrameRes=Default(HalfFrameRes,25)
DupeThresh=Default(DupeThresh,1.0)
FR=Float(Default(FrameRate,clp.FrameRate))
ChromaWeight=Float(Default(ChromaWeight,1.0/3.0))
Assert(HalfFrameRes>0,myName+"HalfFrameRes Must be greater than 0")
Assert(DupeThresh>0.0,myName+"DupeThresh Must be greater than 0.0")
Assert(ChromaWeight>=0.0 && ChromaWeight<=1.0,myName+"ChromaWeight 0.0 -> 1.0")
FuncS="""
Function ApparentFPS@@@(clip clp,Int HalfFrameRes,DupeThresh,Float FR,Float ChromaWeight) {
clp
CurrTime = current_frame / FR
if(current_frame==0 || !RT_VarExist("Prev@@@") || Prev@@@!=current_frame-1) {
# Start/Reset timer within script rather than in Multi-instance Global vars init
Global StartFrame@@@ = current_frame Global StartFrame2@@@ = current_frame
Global StartTime@@@ = CurrTime Global StartTime2@@@ = StartTime@@@
Global Dropped@@@ = 0
Global DupeCnt@@@ = 0 Global DupeCnt2@@@ = 0
Global MaxApparentFPS@@@ = (RT_VarExist("MaxApparentFPS@@@")) ? MaxApparentFPS@@@ : 0.0
Global Prev@@@ = current_frame
}
Diff = RT_FrameDifference(clp,clp,n2=current_frame-1,ChromaWeight=ChromaWeight)
if(Diff < DupeThresh && current_frame > 0) {
Global Dropped@@@ = Dropped@@@ + 1
Global DupeCnt@@@ = DupeCnt@@@ + 1
Global DupeCnt2@@@ = DupeCnt2@@@ + 1
}
FrameCnt = current_frame-StartFrame@@@
UniqueFrameCnt = FrameCnt - DupeCnt@@@
TimeExpired = Max(CurrTime-StartTime@@@,0.000001)
ToggleOnce = (FrameCnt==HalfFrameRes)
Toggle2 = (FrameCnt>=HalfFrameRes*2)
# EDIT: Below lines give Global return values
# First Multi-Instance ApparentFPS@@@ Variable name will be ApparentFPS_AFR_1
# First Multi-Instance MaxApparentFPS@@@ Variable name will be MaxApparentFPS_AFR_1
Global ApparentFPS@@@ = UniqueFrameCnt/TimeExpired
Global MaxApparentFPS@@@ = (ApparentFPS@@@>MaxApparentFPS@@@) ? ApparentFPS@@@ : MaxApparentFPS@@@
# Comment out below Lines
RT_Subtitle("%.3f (Max=%.3f)",UniqueFrameCnt/TimeExpired,MaxApparentFPS@@@)
RT_Subtitle("%d] FrameRate=%.3f ApparentFrameRate=%.3f (Max=%.3f)" +
\ "\nToggleOnce=%s Toggle2=%s\nStartFrame =%d StartTime =%f\nStartFrame2=%d StartTime2=%f" +
\ "\nFrameCnt=%d TimeExpired=%f\nDropped=%d DupeCnt=%d DupeCnt2=%d UniqueFrameCnt=%d",
\ current_frame,FrameCnt/TimeExpired,UniqueFrameCnt/TimeExpired,MaxApparentFPS@@@,ToggleOnce,Toggle2,
\ StartFrame@@@,StartTime@@@,StartFrame2@@@,StartTime2@@@,FrameCnt,TimeExpired,
\ Dropped@@@,DupeCnt@@@,DupeCnt2@@@,UniqueFrameCnt,align=1)
if(ToggleOnce) {
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
Global DupeCnt2@@@ = 0
} Else if(Toggle2) {
Global StartFrame@@@ = StartFrame2@@@
Global StartTime@@@ = StartTime2@@@
Global DupeCnt@@@ = DupeCnt2@@@
Global StartFrame2@@@ = current_frame
Global StartTime2@@@ = CurrTime
Global DupeCnt2@@@ = 0
}
Global Prev@@@ = current_frame
Return Last
}

ARGS = "HalfFrameRes,Dupethresh,FR,ChromaWeight"
clp.GScriptClip("ApparentFPS@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
RT_IncrGlobal("AFR_InstanceNumber")
InstS = RT_StrReplace(FuncS,"@@@","_AFR_"+String(AFR_InstanceNumber))
RT_TxtWriteFile(InstS,"DEBUG_AFR_"+String(AFR_InstanceNumber)+".TXT")
Return GScript(InstS)
}

#Avisource("TEST.avi")
Avisource("CAP1.avi")
ApparentFPS()


EDITED:

[StainlessS]

Ooops, posted RT_Stats v1.40 in wrong folder on MediaFire (was not available, was available via SendSpace), Fixed.

[StainlessS]

New Version RT_Stats v1.41, See 1st post.


v1.41, 15 Sept 2014, Fixed borked v1.40, (released wrong source + binary), RT_QwikScan PC, PD related stuff.
Changed RT_QwikScan LumaTol arg to float.



RT_QwikScan(clip SearchClip,Int SeachStart,clip FindClip,Int FindFrame,string DB,string PNDB,
\ Float "LumaTol"=1.0,int "Flags"=1,
\ Float "LC"=1.0,Float "LD"=1.0,Float "FD"=1.0,Float "PD"=1.0,int "PC"=(SearchClip.Width*SearchClip.Height+127)/255, int "XP"=1,
\ Float "ChromaWeight"=1.0/3.0,Int "PDThresh"=0
\ int "Maxdistance"=SearchClip.FrameCount-1,bool "Inclusive"=True,string "Prefix"="QWKS_")



FINGERPRINT & LUMATOL:
The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
Each FingerPint component is derived from a pixel count for a portion of the possible 0 -> 255 luma pixel value range.
Lets say that one of the fingerprint components is derived from the 128->191 luma pixel value range, the fingerpint component
is computed for every frame in the SearchClip eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height},
this result is in range 0.0 -> 255.0, with 255.0 being equal to 100%. [There would be another three fingerprint components computed,
in different ranges. We do not actually use range 128->191, and are reluctant to give any real used ranges as they may change, if you
want to see the real current ranges used, then see the current source code]. The 4 fingerprint components are stored in the
FingerPrint DB and are EXACT fingerprint component values. For the FindFrame, we use a FingerPrint component Range with LumaTol
tolerance. For FindClip(FindFrame), we might compute a fingerprint component range for [128->191]:
FingerPrintMin = {luma PixelCount in range [128+LumaTol] to [191-LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
FingerPrintMax = {luma PixelCount in range [128-LumaTol] to [191+LumaTol} * 255.0 / {SearchClip.Width*SearchClip.Height}
So using computed FingerPrintMin and FingerPrintMax for four different ranges as upper and lower limits, if any frame in the
SeachClip (DBase) has all four EXACT FingerPrint components within all four fingerprint component limits, then it is a candidate
frame and eligible for exit condition testing if exit conditions were set, or returned to the client as a result CANDIDATE frame if
exit conditions were not set.

v1.41, Have changed LumaTol arg from type Int to type Float, default 1.0. (Will still accept type Int so will not break script).
Whole number values of LumaTol will behave as previously described.
Where a non integer value of Lumatol is given, then LumaTol is split into integer and fractional parts, the integer part is
processed normally where the FingerPrint value is derived from a pixel count for the particular range. After being converted
into a float via eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height}, then the fractional
part of LumaTol is added to the result and limited to range 0.0->255.0. So, the fractional part is a way of giving an additional
margin of error to LumaTol in a frame size relative way.


EDIT:
Change to FingerPrint source

void __cdecl RT_Fingerprint_Lo(const AVSValue &std,const AVSValue &xtra,FINGER &finger,unsigned int *hist,IScriptEnvironment* env,double LTol) {
// NOTE, LTol optional default 0.0
char *myName="RT_Fingerprint_Lo: ";
LTol = max(min(LTol,16.0),0.0);
MYLO mylo;
RT_MYstats_Lo(RTHIST_F,std,xtra,mylo,myName,env,hist);
const int rng[4][2]={{0 ,85},{86,170},{171,255},{64,191}};
const int Pixels = mylo.pixels;
double frac,whole;
frac=modf(LTol,&whole);
int Tol = int(whole);
int i,j;
if(frac == 0.0) {
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0];
const int e = rng[i][1];
int s2 = s+Tol;
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.lolim[i] = sm * 255.0 / Pixels;
if(Tol==0) {
finger.hilim[i]=finger.lolim[i];
} else {
sm=0;
s2 = max(s-Tol,0);
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.hilim[i] = sm * 255.0 / Pixels;
}
}
} else {
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0];
const int e = rng[i][1];
int s2 = s+Tol;
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.lolim[i] = max( (sm * 255.0 / Pixels - frac) ,0.0);
sm=0;
s2 = max(s-Tol,0);
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.hilim[i] = min( (sm * 255.0 / Pixels + frac) ,255.0);
}
}
}

[StainlessS]

@ Forensic,

Are lurking about out there, not online since April.

You only gave two lines of documentation for AvgLumaDif and I think that you never said finally whether the function was as required,
so I've been waiting to add to RT_Stats, anyway, this is what I've knocked up as docs, is it to your satisfaction, will integrate in next version (v1.42).



RT_AvgLumaDif(clip clp, int "n"=current_frame,Bool "Slices"=False,Int "Matrix"=(Width>1100||Height>600?3:2) )

The function, returns the average of pixel pair differentials for a given frame n, ignoring frame perimeter (1 pixel
perimeter around frame edge), range 0.0 -> 255.0.

Args:
n, Int, Default current_frame.
Slices, Bool Default False.
False, Analyze every adjacent pixel pair in the frame with the exception of frame perimeter.
True, Analyze adjacent pixel pairs in the frame, all 4x4 pixel block boundaries are treated as perimeter pixels
and omitted from processing along with the frame perimeter pixels.
Matrix, Int, Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709,

The function finds the average luminosity differential by analyzing every adjacent pixel pair within the video frame,
(both horizontal and vertical, not diagonal) with an option to only process every other 2x2 horizontal and vertical block
to ensure any slice (e.g. h.264, h.265) boundaries are excluded. By calling this function before and again after a given
filter, this function quantifies a filter's affect on luminosity contrast.
NOTE: The perimeter pixels are intentionally ignored as they often incur higher quantization errors.

'x'= Process, '.'=Don't Process

Slices=false Slices=true
........................ ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
........................ ........................

[StainlessS]

Warning of change to RT_Stats v1.42 RT_QwikScan, XP arg (time to give views if any).

Currently implemented:

XP, (Default 1), Sets eXtra Paranoia for EXIT Conditions.
0=Off
1=Continue searching while next frame is a better Exit Condition match ('next frame' meaning dependant upon search direction).
2=Continue searching while next frame is a better or equal Exit Condition match ('next frame' meaning dependant upon search direction).
When an exit condition occurs eg LumaDifference, then QWKS_BM_LD_XP will be set to the count of additional frames traversed due
to finding a match that is better or equal (depending upon whether XP==1 or XP==2) to the initial frame that caused the exit condition,
In such cases, both QWKS_BM_LD and QWKS_BM_LD_FRM will also be set to (in this case) LumaDifference and the final frame (with XP offset)
that was found to be a better or equal match.


Intend to change XP = 1 mode. ('Better Than' mode)

Instead of terminating scan when next frame is not better (even if equal) will do instead:


0 ) Will search for continually better matching frames, until
Worse match or MaxDistance reached,
then returns best so far.

OR

Equal Match, will drop anchor (remember where it got to) and continue scanning until
1) Worse match or MaxDistance reached,
it will return the remembered frame where it dropped anchor.
2) Better match,
then exits the equal match scanning and updates best so far scan position and continues as for step 0).

The last frame returned will be better than all previous matches, but some adjacent previous matches may have been equal to each other.

I could add another XP Mode 3=Strictly Better Match, that did not scan equal match paths if anyone thought it had functionality.

EDIT: I have just encountered a float match condition that did indeed provide better result with above amendment (Hence the change).

EDIT: Will add XP mode 3 = Strictly Better Match (have almost done it).

EDIT: You can treat RT_QwikScan as permanently EXPERIMENTAL, LumaTol Float fractional part stuff may change, as may the
way the fingerprint is computed. The function works just fine at it is but it would be nice if LumaTol could be made purely ornamental.
Currently, the fractional LumaTol thing is really intended for use where integer part is 0.

EDIT: Have now changed slightly the fractional part functionality of LumaTol a bit, works much better and more
targeted at specific frames, speeds it up a bit.

[StainlessS]

RT_Stats v1.42, new version see 1st post.


v1.40, 9 Sept 2014, Fixed, RT_DBaseGetAttrib returned float instead of Int (for int attrib).
Added args Start and End to RT_QueryBorderCrop and RT_QueryLumaMinMax. Added RT_WriteFile + other funcs.
Now 1024 Array & DBase Attributes. Array and DBase now supports type double as private type for RT_Stats
internal routines, set or get as type Float from Avisynth script.
DBase max number of fields increased to 1024. Added DBase/Array String Attributes.
Added RT_QwikScan + other stuff.
v1.41, 15 Sept 2014, Fixed borked v1.40, (released wrong source + binary), RT_QwikScan PC, PD related stuff.
Changed RT_QwikScan LumaTol arg to float.
v1.42, 19 Sept 2014. Modified Float LumaTol imlementation. Added additional XP mode to RT_QwikScan, and affects
BEST MATCH results. Added RT_AvgLumaDif.


RT_QwikScan Doc mods.

XP, (Default 1), Sets eXtra Paranoia for EXIT Conditions.
0 = Off
1 = LOOSELY BETTER THAN (v1.42 changed STRICTLY BETTER THAN to mode 3, and using this one instead).
Continue searching while next frame is a loosely better Exit Condition match ('next frame' meaning dependant upon search direction).
Differs from STRICTLY BETTER THAN in that it scans along equal matches and if a better match found along that equal matches
path, then will return that match, if no better match occurs along an equal match path, then returns the previous frame number
that was better than previous matches. So, the last frame returned will be better than all previous matches, but some adjacent
previous matches may have been equal to each other.
2 = BETTER THAN OR EQUAL
Continue searching while next frame is a better or equal Exit Condition match ('next frame' meaning dependant upon search direction).
3 = STRICTLY BETTER THAN
Continue searching while next frame is a better Exit Condition match ('next frame' meaning dependant upon search direction).
Does not scan along equal match paths.

When an exit condition occurs eg LumaDifference, then QWKS_BM_LD_XP will be set to the count of additional frames traversed due
to finding a match that is better or equal (depending upon whether XP==1 or XP==2, or XP==3) to the initial frame that caused the exit
condition. In such cases, both QWKS_BM_LD and QWKS_BM_LD_FRM will also be set to (in this case) LumaDifference and the final frame
(with XP offset) that was found to be a better or equal match.
v1.42, BEST MATCHES also now comply with XP = 2 (BETTER OR EQUAL) setting, Where XP != 2, then returns the first encountered
frame position where matches are equal, XP == 2 will returns the last frame position encountered where matches are equal.



FINGERPRINT & LUMATOL:
The frame FingerPrint consists of 4 floating point component values (0.0 -> 255.0), and stored in the DB FingerPrint DBase.
Each FingerPint component is derived from a pixel count for a portion of the possible 0 -> 255 luma pixel value range.
Lets say that one of the fingerprint components is derived from the 128->191 luma pixel value range, the fingerprint component
is computed for every frame in the SearchClip eg {luma PixelCount in range 128 to 191} * 255.0 / {SearchClip.Width*SearchClip.Height},
this result is in range 0.0 -> 255.0, with 255.0 being equal to 100%. [There would be another three fingerprint components computed,
in different ranges. We do not actually use range 128->191, and are reluctant to give any real used ranges as they may change, if you
want to see the real current ranges used, then see the current source code]. The 4 fingerprint components are stored in the
FingerPrint DB and are EXACT fingerprint component values. For the FindFrame, we use a FingerPrint component Range with LumaTol
tolerance. For FindClip(FindFrame), we might compute a fingerprint component range for [128->191]:

FingerPrintMin = {luma PixelCount in range [128+LumaTol] to [191-LumaTol]} * 255.0 / {SearchClip.Width*SearchClip.Height}
FingerPrintMax = {luma PixelCount in range [128-LumaTol] to [191+LumaTol]} * 255.0 / {SearchClip.Width*SearchClip.Height}

So using computed FingerPrintMin and FingerPrintMax for four different ranges as upper and lower limits, if any frame in the
SeachClip (DBase) has all four EXACT FingerPrint components within all four fingerprint component limits, then it is a candidate
frame and eligible for exit condition testing if exit conditions were set, or returned to the client as a result CANDIDATE frame if
exit conditions were not set.

v1.42, Have changed LumaTol arg from type Int to type Float, default 1.0. (Will still accept type Int so will not break script).
Whole number values of LumaTol will behave as previously described.
Where a non integer value of Lumatol is given, then LumaTol is split into integer and fractional parts, and processed as described
below.

PixelCountMin_INT = PixCount[128+LumaTol.Int] to PixCount[191-LumaTol.Int] # Sum of
PixelCountMin_FRAC = Int((PixCount[128+LumaTol.Int] + PixCount[191-LumaTol.Int]) * LumaTol.Frac) # Fractional count of extremities
FingerPrintMin = (PixCountMin_INT - PixCountMin_FRAC) * 255.0 / {SearchClip.Width*SearchClip.Height}

PixelCountMax_INT = PixelCount[128-LumaTol.Int] to PixelCount[191+LumaTol.Int] # Sum of
PixelCountMax_FRAC = Int((PixelCount[128-LumaTol.Int-1] + PixelCount[191+LumaTol.Int+1]) * LumaTol.Frac) # Frac count expanded
FingerPrintMax = (PixelCountMin_INT + PixelCountMin_FRAC) * 255.0 / {SearchClip.Width*SearchClip.Height}

Fractional LumaTol as implemented in v1.42 is a big improvement on integer only Lumatol, and intended primarily for extending
LumaTol=0 to have a little extra tolerance. As the fractional part of LumaTol is applied to the actual existing pixel count of a frame,
so it is to some degree self tuning, if eg a count for PixelCount[128-LumaTol.Int-1] is zero, then there will be no additional speed
overhead for that particular FindFrame FingerPrint component match, if the count is high, then more candidate frames will be searched,
this is contrary to v1.41 Float LumaTol where we just added the fractional part where it would incur a speed penalty on all frames.



With Exit Conditions:
Exit conditions are processed if Flags bits are set, this occurs where the above paragraph description would return a candidate frame
to the client. If Exit condition are set, then at least one of those exit conditions also has to be satisfied, if not satisfied, then
the candidate frame will NOT be returned to the client, it will instead keep searching until both a FingerPrint match occurs and an Exit
condition is satisfied, if not, -1 NOT FOUND is returned.
You can set one or more Exit Conditions, if any (or all) of the exit conditions succeed then there will be a +ve frame number
returned by the function, QWKS_EXIT_FLAGS and QWKS_BM_FLAGS bits will be set, the returned frame number will be the first frame
that satisfied an exit condition, and does NOT include any XP arg processing.
If any exit condition is satisfied, and the XP arg is set, the function will continue scanning adjacent frames to the frame
where an exit condition was satisfied and step onward until a better match for the exit condition is NOT found.
(XP=1 is LOOSELY BETTER THAN, and XP=2 is BETTER OR EQUAL, XP=2 is STRICTLY BETTER THAN, see XP). XP Processing occurs on satisfied
exit conditions and the XP processed frame numbers and differences are returned via QWKS_BM_xx and QWKS_BM_xx_FRM local vars.
When using exit conditions, use the return frame number only as an indicator of success, use the returned local vars irrespective of
whether or not XP processing is done (although if XP==0, then returned frame will be same as QWKS_BM_xx_FRM).
v1.42, BEST MATCHES also now comply with XP = 2 (BETTER OR EQUAL) setting, Where XP != 2, then returns the first encountered
frame position where matches are equal, XP == 2 will returns the last frame position encountered where matches are equal.


New Function:


RT_AvgLumaDif(clip clp, int "n"=current_frame,Bool "Slices"=False,Int "Matrix"=(Width>1100||Height>600?3:2) )

The function, returns the average of pixel pair differentials for a given frame n, ignoring frame perimeter (1 pixel
perimeter around frame edge), range 0.0 -> 255.0.

Args:
n, Int, Default current_frame.
Slices, Bool Default False.
False, Analyze every adjacent pixel pair in the frame with the exception of frame perimeter.
True, Analyze adjacent pixel pairs in the frame, all 4x4 pixel block boundaries are treated as perimeter pixels
and omitted from processing along with the frame perimeter pixels.
Matrix, Int, Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used
Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709,

The function finds the average luminosity differential by analyzing every adjacent pixel pair within the video frame,
(both horizontal and vertical, not diagonal) with an option to only process every other 2x2 horizontal and vertical block
to ensure any slice (e.g. h.264, h.265) boundaries are excluded. By calling this function before and again after a given
filter, this function quantifies a filter's affect on luminosity contrast.
NOTE: The perimeter pixels are intentionally ignored as they often incur higher quantization errors.

'x'= Process, '.'=Don't Process

Slices=false Slices=true
........................ ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. ........................
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
.xxxxxxxxxxxxxxxxxxxxxx. .xx..xx..xx..xx..xx..xx.
........................ ........................


EDIT: Modified Float LumaTol now implemented as so:

void __cdecl RT_Fingerprint_Lo(const AVSValue &std,const AVSValue &xtra,FINGER &finger,unsigned int *hist,IScriptEnvironment* env,double LTol) {
// NOTE, LTol optional default 0.0
char *myName="RT_Fingerprint_Lo: ";
LTol = max(min(LTol,16.0),0.0);
MYLO mylo;
RT_MYstats_Lo(RTHIST_F,std,xtra,mylo,myName,env,hist); // Get luma histogram
const int rng[4][2]={{0 ,85},{86,170},{171,255},{64,191}}; // FingerPrint Component Histogram Ranges
const int Pixels = mylo.pixels;
double frac,whole;
frac=modf(LTol,&whole);
int Tol = int(whole);
int i,j;
if(frac == 0.0) { // Integer LumaTol, no fractional part
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0]; // Start of range
const int e = rng[i][1]; // End of range
int s2 = s+Tol; // contract lolim range by LumaTol Int
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
const double smd = sm * 255.0 / Pixels; // cannot be bigger than 255.0
finger.lolim[i] = smd;
if(Tol==0) {
finger.hilim[i]=smd; // EXACT FINGERPRINT components, search identical frames OR DB FINGER
} else {
sm=0;
s2 = max(s-Tol,0); // expand hilim range by LumaTol Int
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
finger.hilim[i] = sm * 255.0 / Pixels; // cannot be bigger than 255.0
}
}
} else {
for(i=4;--i>=0;) {
unsigned int sm=0;
const int s = rng[i][0];
const int e = rng[i][1];
int s2 = s+Tol; // contract lolim range by LumaTol Int
int e2 = e-Tol;
for(j=s2;j<=e2;++j)
sm += hist[j];
sm -= int((hist[s2] + hist[e2]) * frac + 0.5); // Fractional reduction of extremities
finger.lolim[i] = sm * 255.0 / Pixels;
sm=0;
s2 = max(s-Tol,0); // expand hilim range by LumaTol Int
e2 = min(e+Tol,255);
for(j=s2;j<=e2;++j)
sm += hist[j];
if(s2 > 0) sm += int(hist[s2-1] * frac + 0.5); // Add frac parts if possible
if(e2 < 255) sm += int(hist[e2+1] * frac + 0.5);
finger.hilim[i] = sm * 255.0 / Pixels;
}
}
}

[StainlessS]

Ah, hem. I dont know what to say here other than I have a recommendation, if you dont want it, dont get it, I am in no way affiliated
with said device other than that I bought one.

Not the best place to post this, but I just got one and am so goddam impressed (my PC is core Duo 2.4, so I'm easlily impressed),
anyway, it does have my sincere recommendation from a company that I've been using (well a few years ago{EDIT: couple} bought out by another internet company)
since about 1984 (originally on Tottenham Court Rd and New Oxford Street), always a good experience. Brilliant tablet PC, if you dont like it, then tell
somebody else, (well you can call me names if you like but I'm used to that). Dont know why I'm posting here other that I think it is magic, If moderator
wants to spank me, then at least put in some other forum, reccomendation is with good intent. Well good buy.
http://www.morgancomputers.co.uk/product_detail/14895/Advent-Vega-Tegra-Note-7-Quad-Core-1-8GHz-1GB-16GB-Android-4-3-Tablet/#

PS, Comes with Kit-Kat. (As Avisynth now seems all but dead, me guesses that this will soon become my next fav devolopment thingy).

@IanB, if you are reading this, would be nice if you could say one way or another if you have deserted the project,
I'm sure that you would understand that some would be interested in a such statement, either way.