DBSC v0.051, Scene Change Detection - Work-in-Progress

[StainlessS]

Post #1 of Many.

Updated: v0.051
http://www.mediafire.com/download/c9vuw52mhvj8cvy/DBSC-v0.051.zip


Basic Info

/*
DBSC.avs v0.04, by StainlessS @ Doom9

A Scene Change Detection Tool set/kit/box. (Work In Progress - guinea pigs required)

Currently Implemented: (to some degree):-

Scenechange detection:-
DBSC_CreateInfo(), creates fast scan RT_Stats INFO DBase.
For each frame stores RT_YPlaneMin, RT_YPlaneMax, RT_YPlaneMinMaxDifference, RT_YPlaneMedian, RT_AverageLuma, RT_YPlaneStdev
and RT_FrameDifference(n<->n+1). Could also be useful for other purposes.

DBSC_Scan(), Using INFO DBase, creates a second SCAN DBase containing Scene Change detection data.
Single Flash frame tolerant.
Fade-To-Black Scene changes detected.

DBSC_Override(), Using SCAN DBase and Overrides.txt, allows user to set additional Scene Change or remove erroneous ones
from the SCAN DBase. Additional overrides set/unset without re-creating ScanDB.

Scenechange Utility:-
DBSC_WriteFiles(), Using SCAN DBase, Writes Start, End, Mid, And/Or Range Files for Scene Changes.
Also writes a ClipClop command file for Start (ClipClop clip 1) and End (clip 2) frames, used by DBSC_Select().

DBSC_SubsClip(), Returns clip optionally subtitled (metrics And/Or START OF SCENE / END OF SCENE / SINGLE FRAME SCENE subtitles)

DBSC_StackSceneChanges(), Returns clip, End frame of a scene together with Stacked Start frame of following scene.
(Check detections).

DBSC_Select(), Similar-ish to SCSelect.

Dynamically Changing Borders:-
DBSC_DynaCrop(), Detect Dyanamically changing borders using ScanDB and create a SpliceDB for DBSC_DynaSplice().

DBSC_DynaSplice(), Trim, Crop, Resize and Splice, dynamically changing borders. Option to call user defined function
on each trim, after it has been cropped and resized, and before splicing back together again.

DBSC_Levels(), Function to DEMO a User Function as called by DBSC_DynaSplice().

TODO:-
Improve detection.
Blended Scene Change Detection (Well I can dream cant I, anybody any ideas ? )
+ whatver else I (or you) can think of.

Required:-
Frame Accurate seekable source clip.
RT_Stats, http://forum.doom9.org/showthread.php?t=165479
GSCript, http://forum.doom9.org/showthread.php?t=147846
Grunt, http://forum.doom9.org/showthread.php?t=139337
FrameSel, http://forum.doom9.org/showthread.php?t=167971
ClipClop, http://forum.doom9.org/showthread.php?t=162266
RoboCrop http://forum.doom9.org/showthread.php?t=168053
DebugView http://technet.microsoft.com/en-gb/sysinternals/bb896647.aspx (viewing Debug Output in real time)
*/


EDIT: Spell check and auto link insertion broken on site again :( (both worked fine a few days ago)

[StainlessS]

Post #2 of many

Internal Use Functions

####################################################
### INTERNAL USE ONLY FUNCTIONS ***
####################################################

Function DBSC_Str2ID(String IDS) {
Assert(Strlen(IDS)==4,RT_String("%sMust be 4 character string('%s')","DBSC_Str2ID: ",IDS))
Return(((RT_Ord(IDS,4)*256+RT_Ord(IDS,3))*256 + RT_Ord(IDS,2))*256) +RT_Ord(IDS,1)
}

Function DBSC_ColorSpaceIx(clip c) {
cs=c.IsRGB24?0:c.IsRGB32?1:c.IsYUY2?2:c.IsYV12?3:-1
return(cs>=0||VersionNumber<2.6)?cs:c.IsY8?4:c.IsYV16?5:c.IsYV24?6:c.IsYV411?7:-1
}


Function DBSC_InfoValidate(String InfoDB,Bool "MustExist",String "name") {
/*
DBSC_InfoValidate(), Returns InfoDB valid status True/False.
True return signifies that the InfoDB exists and is of type InfoDB.
If InfoDB does not exist and MustExist==True(Default), then throws error alert.
Name, allows display of client function name instead of this function name in error alerts.
*/
myName=Default(name,"DBSC_InfoValidate: ")
Assert(InfoDB!="",RT_String("%sMust provide an InfoDB filename",myName))
Valid = Exist(InfoDB)
GSCript("""
if(Valid) {
Valid = RT_DBaseFields(InfoDB)==7 && RT_VarIsSame(RT_DBaseGetID(InfoDB,0),DBSC_Str2ID("DBSC")) &&
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,1),DBSC_Str2ID("INFO"))
if(!Valid) {
RT_DebugF("InfoDB failed Basic validation",name=myName)
!Valid?Assert(RT_DBaseGetID(InfoDB,1)!=DBSC_Str2ID("SCAN"),RT_String("%sWe Require the InfoDB, not the ScanDB",myName)):NOP
}
} Else {Assert(!Default(MustExist,True),RT_String("%sInfoDB does NOT exist\n'%s'",myName,InfoDB))}
""")
Return Valid
}


Function DBSC_ScanValidate(String ScanDB,Bool "MustExist",String "name") {
/*
DBSC_ScanValidate(), Returns ScanDB valid status True/False.
True return signifies that the ScanDB exists and is of type ScanDB.
If ScanDB does not exist and MustExist==True(Default), then throws error alert.
Name, allows display of client function name instead of this function name in error alerts.
No Check on whether ScanDB scene changes have been user overridden.
*/
myName=Default(name,"DBSC_ScanValidate: ")
Assert(ScanDB!="",RT_String("%sMust provide a ScanDB filename",myName))
Valid = Exist(ScanDB)
GSCript("""
if(Valid) {
Valid = RT_DBaseFields(ScanDB)==5 && RT_VarIsSame(RT_DBaseGetID(ScanDB,0),DBSC_Str2ID("DBSC")) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,1),DBSC_Str2ID("SCAN"))
if(!Valid) {
RT_DebugF("ScanDB failed Basic validation",name=myName)
!Valid?Assert(RT_DBaseGetID(ScanDB,1)!=DBSC_Str2ID("INFO"),RT_String("%sWe Require the ScanDB, not the InfoDB",myName)):NOP
}
} Else {Assert(!Default(MustExist,True),RT_String("%sScanDB does NOT exist\n'%s'",myName,ScanDB))}
""")
Return Valid
}


Function DBSC_QueryOveridden(String ScanDB) {
/*
DBSC_QueryOveridden(), Returns TOTAL number of Overides set on a ScanDB.
Uses third last RT_DBase attribute to store the number of user set Overrides (User attribute count currently 1024).
*/
myName="DBSC_QueryOveridden: "
Assert(DBSC_ScanValidate(ScanDB,MustExist=True,name=myName),RT_String("%sScanDB Invalid\n'%s'",myName,ScanDB))
nAttribs=RT_DBaseGetAttrib(ScanDB,-1)
Return RT_DBaseGetAttrib(ScanDB,nAttribs-4)
}

[StainlessS]

Post #3 of many

User Functions:
Create INFO DBase


####################################################
### USER FUNCTIONS ***
####################################################

Function DBSC_CreateInfo(clip c,String InfoDB,int "Matrix",Float "ChromaWeight",Float "Thresh",Bool "RecycleDB",Bool "DEBUG") {
/*
DBSC_CreateInfo()
Create DBase to assist in scanning clip to extract various data.
c source clip.
InfoDB, DBase Filename to create.
Matrix: Conversion matrix for conversion of RGB to YUV-Y Luma. 0=REC601 : 1=REC709 : 2 = PC601 : 3 = PC709,
Default = (Width > 1100 OR Height>600) then 3(PC709) else 2(PC601). YUV not used
ChromaWeight, Default 0.333333 ChromaWeight range 0.0->1.0. Default gives twice as much weight to luma as combined chroma. (YUV Only).
Thresh, Default 0.2.
RecycleDB, Default True. False, recreate DBase even if already exists. True, Try to re-use existing DBase where possible.
*/
c
myName="DBSC_CreateInfo: "
START_TIME = RT_TimerHP
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
Matrix = Default(Matrix,(Width>1100||Height>600)?3:2)
ChromaWeight = Float(Default(ChromaWeight,1.0/3.0)) # Weighting for combined U + V Channels. Suggest about 1.0/3.0 -> 1.0/2.0
Thresh = Float(Default(Thresh,0.2))
RecycleDB=Default(RecycleDB,True)
DEBUG=Default(DEBUG,True)
Assert(Matrix>=0&&Matrix<=3,RT_String("%sMatrix 0->3",myName))
Assert(ChromaWeight>=0.0 && ChromaWeight<=1.0,RT_String("%sChromaWeight 0.0->1.0",myName))
Assert(Thresh>=0.0 && Thresh<=1.0,RT_String("%sThresh 0.0->1.0",myName))
InfoDB = RT_GetFullPathName(InfoDB) # FullPath name, convert '/' to '\'
FC = FrameCount
#
GScript("""
InfoValid = DBSC_InfoValidate(InfoDB,MustExist=False,name=myName) ? RecycleDB : False
if(InfoValid&&RecycleDB) {
InfoValid =
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,2),Width) && RT_VarIsSame(RT_DBaseGetID(InfoDB,3),Height) &&
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,4),DBSC_ColorSpaceIx(c)) &&
\ RT_VarIsSame(RT_DBaseRecords(InfoDB),FrameCount)
if(!InfoValid) {
(DEBUG) ? RT_DebugF("InfoDB failed Match To Clip validation\n'%s'",InfoDB,name=myName) : NOP
} Else {
InfoValid = RT_VarIsSame(RT_DBaseGetID(InfoDB,5),Matrix) &&
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,6),ChromaWeight) &&
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,7),Thresh)
if(!InfoValid) {
(DEBUG) ? RT_DebugF("InfoDB failed Match Matrix/ChromaWeight/Thresh validation\n'%s'",InfoDB,name=myName) : NOP
}
}
}
if(!RecycleDB||!InfoValid) {
(DEBUG) ? RT_DebugF("Initializing INFO DBase %s",InfoDB,name=myName) : NOP
RT_DBaseAlloc(InfoDB,FC,"iiiifff")
# Info for Frame Number n ie record number
# INT YStats Flags Full name Local Var Range
# 0 1($01) RT_YPlaneMin() "yMin" (0->255)
# 1 2($02) RT_YPlaneMax() "yMax" (0->255)
# 2 4($04) RT_YPlaneMinMaxDifference() "yMinMaxDiff" (0->255)
# 3 8($08) RT_YPlaneMedian() "yMed" (0->255)
# FLOAT
# 4 16($10) RT_AverageLuma() "yAve" (0.0->255.0)
# 5 32($20) RT_YPlaneStdev() "yStdev" (0.0->255.0)
# -
# 6 RT_FrameDifference(n <-> n+1) (0.0->255.0)
pcf = FC / 20.0
nxtn = Round(pcf)
FMT="%d ] %.0f%% Complete"
for(n=0,FC-1) {
# Flgs = YPlaneMin=1,YPlaneMax=2,YPlaneMinMaxDifference=4,YPlaneMedian=8,AverageLuma=16,YPlaneStdev=32
RT_Ystats(Last,n,flgs=$3F,Threshold=Thresh,Matrix=Matrix)
FD=RT_FrameDifference(Last,Last,n,n+1,ChromaWeight)
RT_DBaseSet(InfoDB,n,YS_yMin,YS_yMax,YS_yMinMaxDiff,YS_yMed,YS_yAve,YS_yStdev,FD)

if(DEBUG && n>=nxtn) {
RT_DebugF(FMT,n,Round((n+1)*100.0/FC),name=myName)
nxtn=Round(n+pcf)
}
}
RT_DBaseSetID(InfoDB,0,DBSC_Str2ID("DBSC"))
RT_DBaseSetID(InfoDB,1,DBSC_Str2ID("INFO"))
RT_DBaseSetID(InfoDB,2,Width)
RT_DBaseSetID(InfoDB,3,Height)
RT_DBaseSetID(InfoDB,4,DBSC_ColorSpaceIx(c))
RT_DBaseSetID(InfoDB,5,Matrix)
RT_DBaseSetID(InfoDB,6,ChromaWeight)
RT_DBaseSetID(InfoDB,7,Thresh)
} Else {
(DEBUG) ? RT_DebugF("INFO DBase Recycled '%s'",InfoDB,name=myName) : NOP
}
T = RT_TimerHP - START_TIME
(DEBUG) ? RT_DebugF("Total Time taken = %.2f Seconds (%.2f Mins)",T,T/60.0,name=myName) : NOP
""")
return RecycleDB
}

[StainlessS]

Post #4 of many

Create SCAN DBase {PART_1, (too big)}


Function DBSC_Scan(clip c,String InfoDB,String ScanDB,Float "dFact",Float "dMinim",Int "dMinLen",Float "dCorr",Float "DupTh",Float "POV",
\ Int "dFlashFrames", Bool "RecycleDB",Bool "DEBUG") {
/*
DBSC_Scan().
InfoDB, EXISTING INFO DBSC_CreateInfo created DBase
ScanDB, Name of Scene Change SCAN DBase, created by this function.
dFact, Default 1.5 (greater than 1.0)
dMinim, Default 8.0, (greater than 0.1) dFact and dMinim , Control how much bigger difference between Curr->Next has to be compared
to Prev->Curr and Next->Next+1, to detect scene change.
Curr->Next has to be greater than (dFact * 'Prev->Curr' + dMinim), and also greater than (dFact * 'Next->Next+1' + dMinim)
Dfact=1.4, works pretty good but is on a hair trigger ie very sensitive, dMinim=8.0 reduces/controls that sensitivity.
If clip is anime, then is quite likely that some frame differences are 0.0, so even a small difference next to 0.0 diff frames
will be infinitely greater than 0.0, in this situation dMinim will be the sole govenor of scene detection and so should be of
reasonable value, maybe about 8.0 -> 12.0 would be good choices.
dMinLen, Default 12. Minimum frame length of current scene that qualifies for scene change detection.
DupTh, Default 0.5.
POV, Default 60.0.
RecycleDB, Default True. False, recreate DBase even if already exists. True, Try to re-use existing DBase where possible.

Created ScanDB SCAN DBase Fields
0 ) SOSix, Start Of Scene frame(record)
1 ) EOSix, End Of Scene frame(record)
2 ) Type(1=SOS,2=EOS,3=SOS and EOS(single frame scene),0=none)
3 ) SceneNo(Zero Rel)
4 ) FLGS, Detection Reason code
*/
c
myName="DBSC_Scan: "
START_TIME = RT_TimerHP
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
dFact = Float(Default(dFact,1.25)) # Lower more false +ve's, Higher fewer scene changes (Dfact=1.5 dMinim=8.0 works pretty good)
dMinim = Float(Default(dMinim,6.0)) # Scene Change, avoid silly numbers (very small) on static scenes causing false +ve
dMinLen = Default(dMinLen,6) # Minimum length of scene for scene change detection to be valid (ignore short scenes)
dCorr = Float(Default(dCorr,0.75))
DupTh = Float(Default(DupTh,0.5)) #
POV = Float(Default(POV,60.0)) #
dFlashFrames = Default(dFlashFrames,dMinLen-1)
RecycleDB=Default(RecycleDB,True)
DEBUG=Default(DEBUG,True)
Assert(dFact>1.0 ,RT_String("%sdFact MUST be greater than 1.0",myName))
Assert(dMinim>=1.0,RT_String("%sdMinim MUST be greater or equal to 1.0",myName))
Assert(dMinLen>=2,RT_String("%sdMinLen MUST be greater or equal to 2",myName)) # Cannot detect single frame scene.
Assert(dCorr>=0.6,RT_String("%sdCorr 0.6 -> 1.0",myName))
Assert(DupTh>0.0&&DupTh<=1.0,RT_String("%sdDupTh 0.0 -> 1.0",myName))
Assert(POV>=0.0&&POV<=100.0,RT_String("%sPOV 0.0 -> 100.0",myName))
Assert(dFlashFrames>=1&&dFlashFrames<dMinLen,RT_String("%sdFlashFrames 1 -> dMinLen-1 (%d)",myName,dMinLen-1))
InfoDB = RT_GetFullPathName(InfoDB) # FullPath name, convert '/' to '\'
ScanDB = RT_GetFullPathName(ScanDB)
FC = FrameCount
#
GScript("""
InfoValid = DBSC_InfoValidate(InfoDB,MustExist=True,name=myName)
Assert(InfoValid,RT_String("%sInfoDB failed Basic validation\n'%s'",myName,InfoDB))
InfoValid =
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,2),Width) && RT_VarIsSame(RT_DBaseGetID(InfoDB,3),Height) &&
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,4),DBSC_ColorSpaceIx(c)) &&
\ RT_VarIsSame(RT_DBaseRecords(InfoDB),FC)
Assert(InfoValid,RT_String("%sInfoDB failed Match To Clip validation\n'%s'",myName,InfoDB))
Matrix = RT_DBaseGetID(InfoDB,5)
ChromaWeight = RT_DBaseGetID(InfoDB,6)
Thresh = RT_DBaseGetID(InfoDB,7)
#
ScanValid = DBSC_ScanValidate(ScanDB,MustExist=False,name=myName) ? RecycleDB : False
if(ScanValid&&RecycleDB) {
ScanValid =
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,2),Width) && RT_VarIsSame(RT_DBaseGetID(ScanDB,3),Height) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,4),DBSC_ColorSpaceIx(c)) &&
\ RT_VarIsSame(RT_DBaseRecords(ScanDB),FC)
if(!ScanValid) {
(DEBUG) ? RT_DebugF("ScanDB failed Match To Clip validation\n'%s'",ScanDB,name=myName) : NOP
} Else {
nAttribs=RT_DBaseGetAttrib(ScanDB,-1)
ScanValid = RT_VarIsSame(RT_DBaseGetAttrib(ScanDB,nAttribs-1),Matrix) &&
\ RT_VarIsSame(RT_DBaseGetAttrib(ScanDB,nAttribs-2),ChromaWeight) &&
\ RT_VarIsSame(RT_DBaseGetAttrib(ScanDB,nAttribs-3),Thresh)
if(!ScanValid) {
(DEBUG) ? RT_DebugF("ScanDB failed Match Matrix/ChromaWeight/Thresh to InfoDB\n'%s'",ScanDB,name=myName) : NOP
} else {
ScanValid =
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,5),dFact) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,6),dMinim) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,7),dMinLen) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,8),dCorr) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,9),DupTh) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,10),POV) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,11),dFlashFrames)
if(!ScanValid) {
(DEBUG) ? RT_DebugF("ScanDB failed Match Argument validation\n'%s'",ScanDB,name=myName) : NOP
}
}
}
}
if(!RecycleDB||!ScanValid) {
(DEBUG) ? RT_DebugF("Initializing Scenechange Scan DBase %s",ScanDB,name=myName) : NOP
RT_DBaseAlloc(ScanDB,FC,"iiiii") # SOSix, EOSix, Type(1=SOS,2=EOS,0=non), SceneNo(Zero Rel), FLGS
(DEBUG) ? RT_DebugF("Creating Scan DBase",name=myName) : NOP
Start=0 B=0.0 C=0.0
# Find scene changes, marking EOSix(End Of Scene) of scene start frame(record). (ie set EOSix at SOS record)
# Also set FLGS field at EOS (End frame record).
# A=Diff(n-1<->n) B=Diff(n<->n+1), C=Diff(n+1<->n+2)
# We are detecting scene change between current and next frames, ie when n at End Of Scene
# (@ EOS we already know current SOS, and also next scene SOS is EOS + 1).
#
# For Anime, if Diff(n-1,n) < DupTh, we assume duplicate frame and instead use n-2 in comparisons [same for Diff(n+1,n+2)]
#
dPOV = POV *255.0 / 100.0 # Convert range [0.0,100.0] to [0.0,255.0]
C = RT_DBaseGetField(InfoDB,-1 + 1, 6) # Prep

NextPossEnd = dMinLen - 1
for(n=0,FC-2) {
A=B B=C C=RT_DBaseGetField(InfoDB,Min(n+1,FC-1),6) # Re-use B and C from previous iteration, new C = Diff(n+1 <-> n+2)
if(n >= NextPossEnd) {
Cx = (C<DupTh)?Max(RT_FrameDifference(Last,Last,n+1,n+3,ChromaWeight),C): C # If Diff(n+1,n+2) too low, use Diff(n+1,n+3)
if((dFact*Cx+dMinim) < B) { # sufficiently bigger than 'Next->Next+1' (RHS) ?
# Here, RHS STANDARD DETECT SCENE CHANGE: n+1 CANNOT be a single flash frame
# If fewer than dPOV luma pixels changed by at least 1 luma level, then NOT a scene change
PxDf=RT_LumaPixelsDifferent(Last,Last,n=n,n2=n+1,Thresh=1)
if(PxDf >= dPOV) { # Here, Possible RHS scene change
PxDf2 = RT_LumaPixelsDifferent(Last,Last,n=n,n2=n-1,Thresh=1)
if(PxDf2 < PxDf) { # WOULD LHS be Pixlcount Overridden ? (at previous frame)
C_DR = RT_DBaseGetField(InfoDB,n,2) # MinMaxDifY(n)
N_DR = RT_DBaseGetField(InfoDB,Min(n+1,FC-1),2) # MinMaxDifY(n+1) : ie FROM NEXT FRAME/RECORD
if(Abs(N_DR-C_DR)>4) {
# Look for Flash frames, if found disable detect until reached flash frame,
# can then redetect incl flash frames again from that point.
FlashLim = Min(n+dFlashFrames,FC-2) # MUST HALT on or before last frame where valid n+1 scene is dMinLen
for(i=FlashLim,n+2,-1) {
mfpdf=RT_LumaPixelsDifferent(Last,Last,n=n,n2=i,Thresh=16)
mfcorr=RT_LumaCorrelation(Last,Last,n=n,n2=i)
if(mfpdf<64.0 && mfcorr>dCorr) {
(DEBUG) ? RT_DebugF("%d ] MULTIFLASH: @ %d LumaPixdifCnt=%.2f%% LumaCorr=%.2f",
\ n,i,mfpdf*100.0/255.0,mfcorr,name=myName) : NOP
NextPossEnd = i # No more detections until detected flash frame
RT_DBaseSetField(ScanDB,n,4,$20001)# FLGS reason to Standard detect Override by Multiflash
i = -1 # BREAK
}
}
}
if(n >= NextPossEnd) {
RT_DBaseSetField(ScanDB,n,4,1) # FLGS reason to Standard detect
RT_DBaseSetField(ScanDB,Start,1,n) # Set Start of scene frame/Record EOSix(End OF Scene) to this frame.
NextPossEnd = n + dMinLen
Start = n + 1 # Next frame, SOS follows this EOS
}
} else { # HERE, decide not detected, OR Reason PixelCount overriden
# sufficiently bigger than 'Prev->Curr' (LHS) ? : # If Diff(n,n-1) too low, use Diff(n,n-2)
Ax = (A<DupTh)?Max(RT_FrameDifference(Last,Last,n,n-2,ChromaWeight),A): A
if((dFact*Ax+dMinim) < B) {
RT_DBaseSetField(ScanDB,n,4,$10001) # FLGS reason to Standard detect Overridden by Pixel count
(DEBUG) ? RT_DebugF("%d ] n-1->n Luma Pixel Diff Count OVERRIDE(too low) = %.2f%%",
\ n,PxDf2*100/255.0,name=myName) : NOP
}
}
} else {
RT_DBaseSetField(ScanDB,n,4,$10001) # FLGS reason to Standard detect Overridden by Pixel count
(DEBUG) ? RT_DebugF("%d ] n->n+1 Luma Pixel Diff Count OVERRIDE(too low) = %.2f%%",
\ n,PxDf*100/255.0,name=myName) : NOP
}
}
}
}
# Do the last frame (n = FC-1, moved out of above loop to avoid conditional branch
RT_DBaseSetField(ScanDB,Start,1,FC-1) # Set Start of scene frame/Record EOSix to this last frame.
#

[StainlessS]

Post #5 of many

Create SCAN DBase {PART_2, (too big)}

# Rescan to do Fadeout->FadeIn Scene Changes : Set only EOSix @ Start frame/record of Scene Change, and ALSO FLGS
(DEBUG) ? RT_DebugF("ReScan for Fade-to-Black SceneChanges:",name=myName) : NOP
for(n=0,FC-1) {
Start = n # SOSix for this scene
End = RT_DBaseGetField(ScanDB,Start,1) # EOSix for this scene
Len = End-Start+1
if(Len >= Max(dMinLen*2,24)) {
# Start and End with YPlaneMinMaxDifference >= 16
if(RT_DBaseGetField(InfoDB,Start,2) >= 16 && RT_DBaseGetField(InfoDB,End,2) >= 16) {
for(j=Start+dMinLen,End - dMinLen) {
mnmxdf=RT_DBaseGetField(InfoDB,j,2)
if(mnmxdf <= 6-2) { # 4 or below FADEOUT, 4->6 no mans land (still faded)
je=j
for(k=j+1,End-dMinlen) { # Hunt fade end, No mans land(4->6), remains faded
mnmxdf=RT_DBaseGetField(InfoDB,k,2) # YMinMaxDiff
if(mnmxdf <= 6-0) {
je=k # still fade
} else {
k=End # Not faded any more
}
}
m = (j + je) / 2 # middle faded frame (approx)
if(m < End - dMinLen) {
RT_DebugF("%d ] FADE SCENECHANGE @ %d",n,M,name=myName)
FLGS=RT_DBaseGetField(ScanDB,m-1,4)
RT_DBaseSetField(ScanDB,m-1,4,RT_BitOR(FLGS,$00008000)) # EOSix of prev scene FLGS set to Reason=Fade-To-Black
RT_DBaseSetField(ScanDB,Start,1,m-1) # Adjust EOSix @ Start Frame
Start=m # New Start frame
RT_DBaseSetField(ScanDB,Start,1,End) # Adjust End @ NEW Start Frame
}
# Retry for another FADE SceneChange in original scene
je = Max(m+dMinLen,je)
for(k=je+1,End-dMinLen) { # Scan till MinMaxDif(je+1) is above 16+Leeway
mnmxdf=RT_DBaseGetField(InfoDB,k,2)
if(mnmxdf >=16+8) {
(DEBUG) ? RT_DebugF("%d ] FADE SCENECHANGE RETRY SCAN @ %d",n,je+1,name=myName) : NOP
k=End
} else {
je=k
}
}
j = je # j = je + 1 at next iteration, next frame RETRY scan
}
}
}
}
n = End # End + 1 at next iteration
}

# At this point only EOSix at start of scene frame set, AND ALSO reason FLGS for Last frame of scene
(DEBUG) ? RT_DebugF("Writing Full SceneChange DBase Data",name=myName) : NOP
SceneNo=-1 # Write Scene Number (Zero Relative)
FLGS = 0
for(n=0,FC-1) {
Start = n
End = RT_DBaseGetField(ScanDB,Start,1)
SceneNo = SceneNo + 1
(DEBUG) ? RT_DebugF("%d] START=%d End=%d SceneNo=%d",n,Start,End,SceneNo,name=myName) : NOP
# SOSix, EOSix, Type(1=SOS,2=EOS,3=SOS+EOS,0=non), SceneNo(Zero Rel), FLGS
for(j=Start,End) { # Leave Reason FLGS intact
RT_DBaseSetField(ScanDB,j,0,Start)
RT_DBaseSetField(ScanDB,j,1,End)
RT_DBaseSetField(ScanDB,j,2,0)
RT_DBaseSetField(ScanDB,j,3,SceneNo)
}
RT_DBaseSetField(ScanDB,Start,4,FLGS) # Rewrite FLGS from previous EOS
RT_DBaseSetField(ScanDB,Start,2,1) # Rewrite Type, Start Of Scene
Type = (Start==End) ? 3 : 2 # Single frame scene at last frame ? (SOS & EOS)
RT_DBaseSetField(ScanDB,End ,2,Type) # ReWrite Type, End Of Scene
FLGS = RT_DBaseGetField(ScanDB,End,4) # Current scene FLGS, next time Prev scene FLGS
n = End # Next iteration n = End + 1
}
RT_DBaseSetID(ScanDB,0,DBSC_Str2ID("DBSC"))
RT_DBaseSetID(ScanDB,1,DBSC_Str2ID("SCAN"))
RT_DBaseSetID(ScanDB,2,Width)
RT_DBaseSetID(ScanDB,3,Height)
RT_DBaseSetID(ScanDB,4,DBSC_ColorSpaceIx())
nAttribs=RT_DBaseGetAttrib(ScanDB,-1)
RT_DBaseSetAttrib(ScanDB,nAttribs-1,Matrix)
RT_DBaseSetAttrib(ScanDB,nAttribs-2,ChromaWeight)
RT_DBaseSetAttrib(ScanDB,nAttribs-3,Thresh)
RT_DBaseSetID(ScanDB,5,dFact)
RT_DBaseSetID(ScanDB,6,dMinim)
RT_DBaseSetID(ScanDB,7,dMinLen)
RT_DBaseSetID(ScanDB,8,dCorr)
RT_DBaseSetID(ScanDB,9,DupTh)
RT_DBaseSetID(ScanDB,10,POV)
RT_DBaseSetID(ScanDB,11,dFlashFrames)
} Else {
(DEBUG) ? RT_DebugF("SCScan DBase Recycled '%s'",ScanDB,name=myName) : NOP
}
T = RT_TimerHP - START_TIME
(DEBUG) ? RT_DebugF("Total Time taken = %.2f Seconds (%.2f Mins)",T,T/60.0,name=myName) : NOP
""")
}

[StainlessS]

Post #6 of many

Scene Change Utility funcs, Part 1

##################
# UTILITY
##################

Function DBSC_Override(String ScanDB,String Override_Fn,Bool "DEBUG") {
/*
DBSC_Override(), Using SCAN DBase and Overrides.txt, allows user to set additional Scene Change or remove erroneous ones
from the SCAN DBase.

ScanDB, INFO DBase created by DBSC_Scan.
Override_fn filename: If first character == '*' then uses text in the Override_fn string and does not load file.
When using Override_fn as string, then each override should be on separate line (same format as file).
FORMAT: One frame numer command on each line.
Any +ve frame number sets START Of Scene Change frame.
Any -ve frame number REMOVES a scene change at that frame (START or END).
*/
myName="DBSC_Override: "
START_TIME = RT_TimerHP
DEBUG=Default(DEBUG,True)
Assert(Override_Fn!="",RT_String("%sMust provide a Override filename",myName))
GSCript("""
Assert(DBSC_ScanValidate(ScanDB,MustExist=True,name=myName),RT_String("%sScanDB Invalid\n'%s'",myName,ScanDB))
if(RT_Ord(Override_Fn)!=RT_Ord("*")) {
Assert(Exist(Override_Fn),RT_String("%sOverride_Fn Does NOT exist\n'%s'",myName,Override_Fn))
(DEBUG) ? RT_DebugF("Reading Override File '%s'",Override_Fn,name=myName) : NOP
OVS = RT_ReadTxtFromFile(Override_Fn)
} Else {
OVS = MidStr(Override_Fn,2)
}
OVS=(RT_Ord(OVS)==0) ? "#" : OVS # Avoid error in RT_StrReplaceMulti for Empty string ""
Fnd_S=RT_String("\t\n \n") # Replace TAB and double SPACE with Single SPACE
Rep_S=RT_String(" \n \n")
OVS=RT_StrReplaceMulti(OVS,Fnd_S,Rep_S)
NLINES = RT_TxtQueryLines(OVS) # Lines of text in OVS
(DEBUG) ? RT_DebugF("Scanning Overrides",name=myName) : NOP
FC=RT_DBaseRecords(ScanDB)
INSERTED=0
DELETED =0
RT_DBaseSetID(ScanDB,1,-1) # Disable ScanDB validity until finished without error
for(i=0,NLINES-1) {
STR = RT_TxtGetLine(OVS,i) # Without trailing n/l
While(RT_Ord(STR)==32) { STR=MidStr(STR,2) } # Eat White
S=STR
if(RT_Ord(S)!=RT_Ord("#")&&RT_Ord(S)!=0) { # Ignore comment ONLY or Empty lines
Neg=False
n=-1
if(RT_Ord(S)==RT_Ord("-")) {
Neg=True
S=MidStr(S,2)
While(RT_Ord(S)==32) { S=MidStr(S,2) } # Eat White
} Else if(RT_Ord(S)==RT_Ord("+")) {
S=MidStr(S,2)
While(RT_Ord(S)==32) { S=MidStr(S,2) } # Eat White
}
if(RT_Ord(S)>=RT_Ord("0") && RT_Ord(S)<=RT_Ord("9")) {
n=RT_NumberValue(S)
While(RT_Ord(S)>=RT_Ord("0") && RT_Ord(S)<=RT_Ord("9")) { S=MidStr(S,2) } # Eat Digits
While(RT_Ord(S)==32) { S=MidStr(S,2) } # Eat White
if(RT_Ord(S)==RT_Ord("#")) {S=""} # Eat remaining Comment
}
Assert(S=="",RT_String("%sLINE %d : Non Parse @'%s'\n'%s'",myName,i+1,S,STR))
Assert(n>=0 && n<FC,RT_String("%sLINE %d : Frame %d Does Not Exist @'%s'\n'%s'",myName,i+1,n,S,STR))
Assert(n!=0 && n!=FC-1,RT_String("%sLINE %d : Cannot Override First or Last frame %d @'%s'\n'%s'",myName,i+1,n,S,STR))
# ScanDB Fields: SOSix, EOSix, Type(1=SOS,2=EOS,0=non), SceneNo(Zero Rel), FLGS
Start=RT_DBaseGetField(ScanDB,n,0)
End =RT_DBaseGetField(ScanDB,n,1)
Type =RT_DBaseGetField(ScanDB,n,2)
Single=(Start==End && Start == n && Type==3)
if(Neg) {
if(n!=Start && n != End) {
(DEBUG||Type!=0) ? RT_DebugF("LINE %d : DEL SC, Frame %d Neither Start nor End Of Scene, Ignoring",i+1,n,name=myName) : NOP
Assert(Type==0,RT_String("%sLINE %d : INTERNAL ERROR, Frame %d Type non-zero(%d)",myName,i+1,n,Type))
} Else {
if(Single) {
# Single Frame Scene is melded with previous scene
(DEBUG) ? RT_DebugF("LINE %d : DEL SC, Frame %d Removing Single Frame Scene",i+1,n,name=myName) : NOP
NewStart = RT_DBaseGetField(ScanDB,n-1,0) # SOSix from previous scene
NewEnd = End # EOSix from current scene (ie n)
NewType=(NewStart==n-1)?1:0 # Previous is single frame scene
RT_DBaseSetField(ScanDB,n-1,2,NewType) # Change n-1 to Type NewType (SOS OR none)
FLGS=RT_DBaseGetField(ScanDB,n-1,4) # n-1 FLGS
RT_DBaseSetField(ScanDB,n-1,4,RT_BitOR(FLGS,$40000000)) # FLGS User Deleted
FLGS=RT_DBaseGetField(ScanDB,n,4) # n FLGS
RT_DBaseSetField(ScanDB,n,4,RT_BitOR(FLGS,$40000000)) # FLGS User Deleted
} Else if(Start==n) {
(DEBUG||Type!=1) ? RT_DebugF("LINE %d : DEL SC, Frame %d Removing Start Of Scene",i+1,n,name=myName) : NOP
Assert(Type==1,RT_String("%sLINE %d : INTERNAL ERROR, Frame %d Type non-1(%d)",myName,i+1,n,Type))
NewStart = RT_DBaseGetField(ScanDB,n-1,0) # SOSix from previous scene
NewEnd = End # EOSix from current scene
RT_DBaseSetField(ScanDB,n-1,2,0) # Change n-1 to Type=None
FLGS=RT_DBaseGetField(ScanDB,n-1,4) # n-1 FLGS
RT_DBaseSetField(ScanDB,n-1,4,RT_BitOR(FLGS,$40000000)) # FLGS User Deleted
FLGS=RT_DBaseGetField(ScanDB,n,4) # n FLGS
RT_DBaseSetField(ScanDB,n,4,RT_BitOR(FLGS,$40000000)) # FLGS User Deleted
} Else {
(DEBUG||Type!=2) ? RT_DebugF("LINE %d : DEL SC, Frame %d Removing End Of Scene",i+1,n,name=myName) : NOP
Assert(Type==2,RT_String("%sLINE %d : INTERNAL ERROR, Frame %d Type non-2(%d)",myName,i+1,n,Type))
NewStart = Start # SOSis from current scene
NewEnd = RT_DBaseGetField(ScanDB,n+1,1) # EOSix from next scene
RT_DBaseSetField(ScanDB,n+1,2,0) # Change n+1 to Type=None
FLGS=RT_DBaseGetField(ScanDB,n,4) # n FLGS
RT_DBaseSetField(ScanDB,n,4,RT_BitOR(FLGS,$40000000)) # FLGS User Deleted
FLGS=RT_DBaseGetField(ScanDB,n+1,4) # n+1 FLGS
RT_DBaseSetField(ScanDB,n+1,4,RT_BitOR(FLGS,$40000000)) # FLGS User Deleted
}
for(j=NewStart,NewEnd) {
RT_DBaseSetField(ScanDB,j,0,NewStart) # Change all of scene SOSix to new Start
RT_DBaseSetField(ScanDB,j,1,NewEnd) # Change all of scene EOSix to new End
}
NewType=(Single)?2:0
RT_DBaseSetField(ScanDB,n,2,NewType) # Change n to Type=None OR EOS if Single Frame Scene
DELETED = DELETED + 1 # Removed a scene
}
} Else {
If(Single) {
(DEBUG) ? RT_DebugF("LINE %d : INS SC, Frame %d already Single Frame Scene, Ignoring",i+1,n,name=myName) : NOP
} Else if(Start==n) {
(DEBUG||Type!=1) ? RT_DebugF("LINE %d : INS SC, Frame %d already Start Of Scene, Ignoring",i+1,n,name=myName) : NOP
Assert(Type==1,RT_String("%sLINE %d : INTERNAL ERROR, Frame %d Type non-1(%d)",myName,i+1,n,Type))
} Else {
if(End==n) {
(DEBUG||Type!=2) ? RT_DebugF("LINE %d : INS SC, Frame %d Changing End Of Scene to Single Frame Scene",i+1,n,name=myName) : NOP
Assert(Type==2,RT_String("%sLINE %d : INTERNAL ERROR, Frame %d Type non-2(%d)",myName,i+1,n,Type))
} Else {
(DEBUG||Type!=0) ? RT_DebugF("LINE %d : INS SC, Frame %d Changing Mid Scene to Start Of Scene",i+1,n,name=myName) : NOP
Assert(Type==0,RT_String("%sLINE %d : INTERNAL ERROR, Frame %d Type non-0(%d)",myName,i+1,n,Type))
}
for(j=Start,n-1) {
RT_DBaseSetField(ScanDB,j,1,n-1) # Change all of scene EOSix to new end
}
NewType=(Start==n-1)?3:2
RT_DBaseSetField(ScanDB,n-1,2,NewType) # Change n-1 to Type=Single Frame Scene OR End Of Scene
for(j=n,End) {
RT_DBaseSetField(ScanDB,j,0,n) # Change all of scene SOSix to new start
}
NewType=(End==n)?3:1
RT_DBaseSetField(ScanDB,n,2,NewType) # Change n to Type= Single Frame Scene OR Start Of Scene
FLGS=RT_DBaseGetField(ScanDB,n,4) # n FLGS
RT_DBaseSetField(ScanDB,n,4,RT_BitOR(FLGS,$80000000)) # FLGS User INSERTED START
FLGS=RT_DBaseGetField(ScanDB,n-1,4) # n-1 FLGS
RT_DBaseSetField(ScanDB,n-1,4,RT_BitOR(FLGS,$80000000)) # FLGS User INSERTED END (or Single Frame Scene)
INSERTED = INSERTED + 1 # Added new scene
}
}
}
}
if(INSERTED!=0 || DELETED!=0) {
(DEBUG) ? RT_DebugF("%d Overrides Processed",INSERTED+DELETED,name=myName) : NOP
(DEBUG) ? RT_DebugF("ReScan to update SCAN DBase SceneNo",name=myName) : NOP
SceneNo = -1
for(n=0,FC-1) {
S=n
SceneNo = SceneNo + 1
Start=RT_DBaseGetField(ScanDB,n,0)
End =RT_DBaseGetField(ScanDB,n,1)
Type =RT_DBaseGetField(ScanDB,n,2)
Single=(Start==End && Start == n && Type==3)
Assert(S==Start,RT_String("%s%d ] UPDATE INTERNAL ERROR : Start SOSix Mismatch(%d,%d)",myName,n,S,Start))
(!Single)?Assert(Type==1 ,RT_String("%s%d ] UPDATE INTERNAL ERROR : Start Type non-1(%d)",myName,n,Type)):NOP
EStart=RT_DBaseGetField(ScanDB,End,0)
EEnd =RT_DBaseGetField(ScanDB,End,1)
EType =RT_DBaseGetField(ScanDB,End,2)
Assert(S==EStart,RT_String("%s%d ] UPDATE INTERNAL ERROR : End SOSix Mismatch(%d,%d)",myName,n,S,EStart))
Assert(End==EEnd,RT_String("%s%d ] UPDATE INTERNAL ERROR : End MisMatch(%d,%d)",myName,n,End,EEnd))
(!Single)?Assert(EType==2 ,RT_String("%s%d ] UPDATE INTERNAL ERROR : End Type non-2(%d)",myName,n,EType)):NOP
(Single) ?Assert(EType==3 ,RT_String("%s%d ] UPDATE INTERNAL ERROR : End Type non-3(%d)",myName,n,EType)):NOP
RT_DBaseSetField(ScanDB,Start,3,SceneNo)
RT_DBaseSetField(ScanDB,End ,3,SceneNo)
for(i=Start+1,End-1) {
SOSix=RT_DBaseGetField(ScanDB,i,0)
EOSix=RT_DBaseGetField(ScanDB,i,1)
Type =RT_DBaseGetField(ScanDB,i,2)
Assert(Start==SOSix,RT_String("%s%d ] UPDATE INTERNAL ERROR : Start SOSix Mismatch(%d,%d)",myName,n,Start,SOSix))
Assert(End==EOSix ,RT_String("%s%d ] UPDATE INTERNAL ERROR : End MisMatch(%d,%d)",myName,n,End,EOSix))
Assert(Type==0 ,RT_String("%s%d ] UPDATE INTERNAL ERROR : Start Type non-1(%d)",myName,n,Type))
RT_DBaseSetField(ScanDB,i,3,SceneNo)
}
n = End # Next iteration n = End + 1
}
nAttribs=RT_DBaseGetAttrib(ScanDB,-1)
OVERRIDDEN = RT_DBaseGetAttrib(ScanDB,nAttribs-4)
TotOVR = OVERRIDDEN+INSERTED+DELETED
RT_DBaseSetAttrib(ScanDB,nAttribs-4,TotOVR) # Count of Overridden
(DEBUG) ? RT_DebugF("Total Accumulated Overrides = %d",TotOVR,name=myName) : NOP
}
RT_DBaseSetID(ScanDB,1,DBSC_Str2ID("SCAN")) # Re-enable ScanDB validity
T = RT_TimerHP - START_TIME
(DEBUG) ? RT_DebugF("Total Time taken = %.2f Seconds (%.2f Mins)",T,T/60.0,name=myName) : NOP
""")
Return 0
}

[StainlessS]

Post #7 of many

Scene Change Utility funcs, Part 2


Function DBSC_WriteFiles(String ScanDB,String "Start_Fn",String "End_Fn",String "Range_Fn",String "Mid_Fn",String "ClipClopCmd_Fn",
\ Bool "Comment",Bool "Comma",String "VideoFileName") {
/*
Write Scene info to output files:- Start, End, Mid Frame and Range.
Also writes a ClipClop command file for Start (clip 1) and End (clip 2) frames, used by DBSC_Select().
ScanDB, SCAN DBase file created via DBSC_Scan.
Comment, Default True writes additional Hash (#) separated info to file. False, omits extra info.
Comma, Default True, use Comma (,) range separator, Else False uses SPACE range separator.
VideoFilename, can supply clip name, added to Comment Header.
*/
myName="DBSC_WriteFiles: "
START_TIME = RT_TimerHP
Start_Fn=Default(Start_Fn,"") End_Fn=Default(End_Fn,"") Range_Fn=Default(Range_Fn,"")
Mid_Fn=Default(Mid_Fn,"") ClipClopCmd_Fn=Default(ClipClopCmd_Fn,"")

Comment=Default(Comment,True)
Comma=Default(Comma,True)
VideoFileName=Default(VideoFileName,"")
VideoFileName=(VideoFileName!="") ? RT_FilenameSplit(VideoFileName,12) : ""
SEP = RT_Ord((Comma) ? "," : " ")
GSCript("""
DBSC_ScanValidate(ScanDB,MustExist=True,name=myName)
(Start_Fn !="" && Exist(Start_Fn)) ? RT_FileDelete(Start_Fn) : NOP
(End_Fn !="" && Exist(End_Fn)) ? RT_FileDelete(End_Fn) : NOP
(Range_Fn !="" && Exist(Range_Fn)) ? RT_FileDelete(Range_Fn) : NOP
(Mid_Fn !="" && Exist(Mid_Fn)) ? RT_FileDelete(Mid_Fn) : NOP
(ClipClopCmd_Fn!="" && Exist(ClipClopCmd_Fn)) ? RT_FileDelete(ClipClopCmd_Fn) : NOP
(Start_Fn !="") ? RT_DebugF("Writing Start_Fn File '%s'",Start_Fn,name=myName) : NOP
(End_Fn !="") ? RT_DebugF("Writing End_Fn File '%s'", End_Fn, name=myName) : NOP
(Range_Fn !="") ? RT_DebugF("Writing Range_Fn File '%s'",Range_Fn,name=myName) : NOP
(Mid_Fn !="") ? RT_DebugF("Writing Mid_Fn File '%s'", Mid_Fn,name=myName) : NOP
(ClipClopCmd_Fn!="") ? RT_DebugF("Writing ClipClopCmd_Fn File '%s'", ClipClopCmd_Fn,name=myName) : NOP
if(Comment) {
nAttribs = RT_DBaseGetAttrib(ScanDB,-1)
Matrix = RT_DBaseGetAttrib(ScanDB,nAttribs-1)
ChromaWeight= RT_DBaseGetAttrib(ScanDB,nAttribs-2)
Thresh = RT_DBaseGetAttrib(ScanDB,nAttribs-3)
OVrCnt = RT_DBaseGetAttrib(ScanDB,nAttribs-4)
dFact = RT_DBaseGetID(ScanDB,5)
dMinim = RT_DBaseGetID(ScanDB,6)
dMinLen = RT_DBaseGetID(ScanDB,7)
dCorr = RT_DBaseGetID(ScanDB,8)
DupTh = RT_DBaseGetID(ScanDB,9)
POV = RT_DBaseGetID(ScanDB,10)
dFlashFrames= RT_DBaseGetID(ScanDB,11)
BAR =RT_StrPad("",86,"#")+Chr(10)
HEAD=RT_String("#\n# DBSC: Scene Change @@@ Frames file : @ %s\n#\n",RT_LocalTimeString(File=False))
VSTR=(VideoFileName=="") ? "" : RT_String("# SourceFile:- '%s'\n#\n",VideoFileName)
ARG =RT_String("# dFact =%.2f : dMinim =%.2f : dMinLen =%d : dCorr = %.2f\n# DupTh=%.2f : POV=%.2f : dFlashFrames=%d\n# Matrix=%d : ChromaW=%.2f : Thresh = %.2f : Overides=%d\n#\n",
\ dFact,dMinim,dMinLen,dCorr,DupTh,POV,dFlashFrames,Matrix,ChromaWeight,Thresh,OvrCnt)
LAB=RT_StrPad("# # SceneNo : Start : Mid : End : Length",85)+"#"
Z=RT_String("%s%s%s%s%s%s",BAR,HEAD,VSTR,ARG,BAR,LAB)
Z=RT_StrReplace(Z,"@@@","%s")
(Start_Fn !="") ? RT_WriteFile(Start_Fn, Z,"START" ,Append=True) : NOP
(End_Fn !="") ? RT_WriteFile(End_Fn, Z,"END" ,Append=True) : NOP
(Range_Fn !="") ? RT_WriteFile(Range_Fn, Z,"RANGE" ,Append=True) : NOP
(Mid_Fn !="") ? RT_WriteFile(Mid_Fn, Z,"MID" ,Append=True) : NOP
(ClipClopCmd_Fn!="") ? RT_WriteFile(ClipClopCmd_Fn,Z,"CLIPCLOP_CMD",Append=True) : NOP
}
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start=RT_DBaseGetField(ScanDB,n,0) End =RT_DBaseGetField(ScanDB,n,1) SceneNo=RT_DBaseGetField(ScanDB,n,3)+1
Mid = (Start + End)/ 2 Len = End-Start + 1
if(!Comment) {
(Start_Fn!="") ? RT_WriteFile(Start_Fn,"%d" ,Start ,Append=True) : NOP
(End_Fn!="") ? RT_WriteFile(End_Fn, "%d" ,End ,Append=True) : NOP
(Range_Fn!="") ? RT_WriteFile(Range_Fn,"%d%c%d" ,Start,SEP,End,Append=True) : NOP
(Mid_Fn!="") ? RT_WriteFile(Mid_Fn, "%d" ,Mid ,Append=True) : NOP
if(ClipClopCmd_Fn!="") {
RT_WriteFile(ClipClopCmd_Fn, "%d%c%d",(Start==End)?3:1,SEP,Start,Append=True)
(Start!=End) ? RT_WriteFile(ClipClopCmd_Fn, "2%c%d",SEP,End,Append=True) : NOP
}
} Else {
Com = RT_String(" # %6d : %6d : %6d : %6d : %6d",SceneNo,Start,Mid,End,Len)
(Start_Fn!="") ? RT_WriteFile(Start_Fn,"%-13d%s", Start,Com,Append=True) : NOP
(End_Fn!="") ? RT_WriteFile(End_Fn ,"%-13d%s", End, Com,Append=True) : NOP
(Range_Fn!="") ? RT_WriteFile(Range_Fn,"%s%s",RT_StrPad(RT_String("%d%c%d",Start,SEP,End),13),Com,Append=True) : NOP
(Mid_Fn!="") ? RT_WriteFile(Mid_Fn ,"%-13d%s", Mid, Com,Append=True) : NOP
if(ClipClopCmd_Fn!="") {
RT_WriteFile(ClipClopCmd_Fn,RT_String("%s%s",RT_StrPad(RT_String("%d%c%d",(Start==End)?3:1,SEP,Start),13),com),Append=True)
(Start!=End)?RT_WriteFile(ClipClopCmd_Fn,RT_String("%s%s",RT_StrPad(RT_String("%d%c%d",2,SEP,End),13),com),Append=True):NOP
}
}
n = End # Next iteration n = End + 1, ie Start of Next Scene
}
T = RT_TimerHP - START_TIME
RT_DebugF("Total Time taken = %.2f Seconds (%.2f Mins)",T,T/60.0,name=myName)
""")
}

[StainlessS]

Post #8 of many

Clip Display funcs


Function DBSC_SubsClip(clip c,String InfoDB,String ScanDB,Bool "Subs",Bool "Metrics",Bool "Metrics2",String "name") {
/*
DBSC_SubsClip(), Return clip optionally subtitled (metrics And/Or START OF SCENE : END OF SCENE Subtitles)
InfoDB, INFO DBase file created via DBSC_CreateInfo().
ScanDB, SCAN DBase as created via DBSC_Scan().
Metrics, Default True. Write RT_Subtitle Metrics to frame.
Metrics2, Default True. Write RT_Subtitle Metrics2 to frame.
Subs, Default True. Write Subtitles to frame, "START OF SCENE" and "END OF SCENE".
*/
myName=Default(name,"DBSC_SubsClip: ")
c
FC=FrameCount
Subs=Default(Subs,True)
Metrics=Default(Metrics,True)
Metrics2=Default(Metrics2,True)
#
InfoValid = DBSC_InfoValidate(InfoDB,MustExist=True,name=myName)
Assert(InfoValid,RT_String("%sInfoDB failed Basic validation\n'%s'",myName,InfoDB))
InfoValid =
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,2),Width) && RT_VarIsSame(RT_DBaseGetID(InfoDB,3),Height) &&
\ RT_VarIsSame(RT_DBaseGetID(InfoDB,4),DBSC_ColorSpaceIx(c)) &&
\ RT_VarIsSame(RT_DBaseRecords(InfoDB),FC)
Assert(InfoValid,RT_String("%sInfoDB failed Match To Clip validation\n'%s'",myName,InfoDB))
Matrix = RT_DBaseGetID(InfoDB,5)
ChromaWeight= RT_DBaseGetID(InfoDB,6)
Thresh = RT_DBaseGetID(InfoDB,7)
#
ScanValid = DBSC_ScanValidate(ScanDB,MustExist=True,name=myName)
Assert(ScanValid,RT_String("%sScanDB failed Basic validation\n'%s'",myName,ScanDB))
ScanValid =
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,2),Width) && RT_VarIsSame(RT_DBaseGetID(ScanDB,3),Height) &&
\ RT_VarIsSame(RT_DBaseGetID(ScanDB,4),DBSC_ColorSpaceIx(c)) &&
\ RT_VarIsSame(RT_DBaseRecords(ScanDB),FC)
Assert(ScanValid,RT_String("%sScanDB failed Match To Clip validation\n'%s'",myName,ScanDB))
nAttribs=RT_DBaseGetAttrib(ScanDB,-1)
ScanValid = RT_VarIsSame(RT_DBaseGetAttrib(ScanDB,nAttribs-1),Matrix) &&
\ RT_VarIsSame(RT_DBaseGetAttrib(ScanDB,nAttribs-2),ChromaWeight) &&
\ RT_VarIsSame(RT_DBaseGetAttrib(ScanDB,nAttribs-3),Thresh)
Assert(ScanValid,RT_String("%sScanDB failed Match Matrix/ChromaWeight/Thresh to InfoDB\n'%s'",myName,ScanDB))
RT_Script = """
n = current_frame
SC = RT_DBaseGetField(ScanDB,n,2)
@@
@@@
@@@@
@@@@@
@@@@@@
Return Last
"""
RT_Met1 = """
FC = Framecount
SOS = RT_DBaseGetField(ScanDB,n,0)
EOS = RT_DBaseGetField(ScanDB,n,1)
SceneNo = RT_DBaseGetField(ScanDB,n,3)+1
FLGS = RT_DBaseGetField(ScanDB,n,4)
A = RT_DBaseGetField(InfoDB,Max(n-1,0),6)
B = RT_DBaseGetField(InfoDB,n,6)
C = RT_DBaseGetField(InfoDB,Min(n+1,FC-1),6)
Ax = (A<DupTh)?Max(RT_FrameDifference(Last,Last,n,n-2,ChromaWeight),A): A
Cx = (C<DupTh)?Max(RT_FrameDifference(Last,Last,n+1,n+3,ChromaWeight),C): C # If Diff(n+1,n+2) too low, use Diff(n+1,n+3)
PxDif = RT_LumaPixelsDifferent(Last,Last,delta2=1,Thresh=1) * 100.0 / 255.0
FLGS = RT_DBaseGetField(ScanDB,n,4)
CorrY = RT_Lumacorrelation(Last,Last,n=n,n2=n+1)

FS=(RT_BitAnd(FLGS,$C0000000)!=0)?"USER OVERRIDE":(RT_BitTst(FLGS,16))?"PIXELCOUNT OVERRIDE":
\ RT_BitTst(FLGS,17)?"MULTIFLASH OVERRIDE" : (RT_BitTst(FLGS,15))?"FADE-TO-BLACK":""
"""
RT_Met2="""
Mn = RT_DBaseGetField(InfoDB,n,0)
Mx = RT_DBaseGetField(InfoDB,n,1)
MnMx = RT_DBaseGetField(InfoDB,n,2)
Med = RT_DBaseGetField(InfoDB,n,3)
AveY = RT_DBaseGetField(InfoDB,n,4)
StdY = RT_DBaseGetField(InfoDB,n,5)
"""
RT_Met1_Fn="""RT_Subtitle(FMT, n,SC,SceneNo,SceneCnt,n-SOS,SOS,EOS,EOS-SOS+1,Ax,B,Cx,PxDif,CorrY,OVR,FLGS,FS)"""
RT_Met2_Fn="""RT_Subtitle(FMT2,mn,mx,MnMx,Med,AveY,StdY,align=1)"""
INSERT_Script="""Last = (SC==1) ? STARTCLIP : (SC==2) ? ENDCLIP : (SC==3) ? SINGLECLIP : Last"""
SUBS_Script="""
SC = RT_DBaseGetField(ScanDB,current_frame,2)
Last = (SC==1) ? STARTCLIP : (SC==2) ? ENDCLIP : (SC==3) ? SINGLECLIP : Last
Return Last
"""
GScript("""
If(Metrics || Metrics2 || Subs) {
SIZE = Round(Height/8.0)
STARTCLIP = SUBS?SubTitle("START OF SCENE",Align=5,size=SIZE,y=(Height+SIZE)/2,text_color=$C080FF) : Last
ENDCLIP = SUBS?SubTitle("END OF SCENE" ,Align=5,size=SIZE,y=(Height-SIZE)/2,text_color=$FF80C0) : Last
SINGLECLIP=SUBS?SubTitle("START OF SCENE\nEND OF SCENE",Align=5,size=SIZE,y=(Height-SIZE)/2,text_color=$C0FF80,lsp=0):Last
if(Metrics||Metrics2) {
FMT="%d ]\nSC=%d#%d/%d:%d{S%d:E%d:L%d}\nPrv=%.2f Cur=%.2f Nxt=%.2f\nPxDif=%.2f%% : Corr=%.2f\nOvrCnt=%d : $%08X %s"
FMT2="Mn=%d Mx=%d MMDf=%d Med=%d\nAve=%.2f Std=%.2f"
nAttribs = RT_DBaseGetAttrib(ScanDB,-1)
OVR = RT_DBaseGetAttrib(ScanDB,nAttribs-4)
SceneCnt=RT_DBaseGetField(ScanDB,RT_DBaseRecords(ScanDB)-1,3)+1
DupTh = RT_DBaseGetID(ScanDB,8)
Script = RT_StrReplace(RT_Script,"@@@@@@" ,(Metrics2?RT_Met2_Fn:""))
Script = RT_StrReplace(Script ,"@@@@@" ,(Metrics?RT_Met1_Fn:""))
Script = RT_StrReplace(Script ,"@@@@" ,(SUBS?INSERT_SCRIPT:""))
Script = RT_StrReplace(Script ,"@@@" ,(Metrics2?RT_Met2:""))
Script = RT_StrReplace(Script ,"@@" ,(Metrics||Metrics2?RT_Met1:""))
Last.ScriptClip(Script,Args="InfoDB,ScanDB,FMT,FMT2,SceneCnt,STARTCLIP,ENDCLIP,SINGLECLIP,OVR,DupTh,ChromaWeight")
} Else {
Last.ScriptClip(SUBS_Script,Args="ScanDB,STARTCLIP,ENDCLIP,SINGLECLIP")
}
}
""")
return Last
}

Function DBSC_StackSceneChanges(clip c,String InfoDB,String ScanDB,Bool "Subs",Bool "Metrics") {
/*
DBSC_StackSceneChanges(), Return clip, End frame of a scene together with Stacked Start frame of following scene.

InfoDB, INFO DBase file created via DBSC_CreateInfo().
ScanDB, SCAN DBase as created via DBSC_Scan().
Metrics, Default True. Write RT_Subtitle Metrics to frame.
Subs, Default True. Write Subtitles to frame, eg "START OF SCENE" and "END OF SCENE", Single frame scene has both written.
Returns NO AUDIO.
*/
myName="DBSC_StackSceneChanges: "
Subs=Default(Subs,True)
Metrics=Default(Metrics,True)
DBSC_SubsClip(c,InfoDB,ScanDB,Metrics,Subs,name=myName)
LT=RT_LocalTimeString Start_Fn=LT+"_Start.txt" End_Fn =LT+"_End.txt"
DBSC_WriteFiles(ScanDB,Start_Fn,End_Fn) S=FrameSel(Last,cmd=Start_fn) E=FrameSel(Last,cmd=End_fn)
RT_FileDelete(Start_Fn) RT_FileDelete(End_Fn)
S=S.Trim(1,0) E=E.Trim(0,E.FrameCount-2)
(Width>=Height) ? StackVertical(E,S) : StackHorizontal(E,S)
return Last.KillAudio
}

[StainlessS]

Post #9 of many

SCSelect equivalent

Function DBSC_Select(clip Motion,String ScanDB,clip "StartClip", clip "EndClip",Clip "SingleClip") {
/*
DBSC_Select(), Similar-ish to SCSelect()

Motion, frames used where not Start nor End of scene, nor Single Frame scene,
Start/End/Single clips default to Motion clip if not supplied.
ScanDB, SCAN DBase as created via DBSC_Scan().
StartClip, Frames used for Start Of Scene Frames, Default Motion clip.
EndClip, Frames used for End Of Scene Frames, Default Motion clip.
SingleClip, Frames used for Single frame Scene, Default Motion clip.
Clip FrameCount's must be same as the ScanDB detection clip, (no other similarities to detection clip enforced).
Returns NO AUDIO.
*/
myName="DBSC_Select: "
Motion StartClip=Default(StartClip,Last) EndClip=Default(EndClip,Last) SingleClip=Default(SingleClip,Last)
Assert(RT_VarIsSame(Last,StartClip,Sig=False),RT_String("%sDissimilar StartClip dimensions OR ColorSpace",myName))
Assert(StartClip.FrameCount==FrameCount,RT_String("%sDissimilar StartClip FrameCount",myName))
Assert(RT_VarIsSame(Last,EndClip,Sig=False),RT_String("%sDissimilar EndClip dimensions OR ColorSpace",myName))
Assert(EndClip.FrameCount==FrameCount,RT_String("%sDissimilar EndClip FrameCount",myName))
Assert(RT_VarIsSame(Last,SingleClip,Sig=False),RT_String("%sDissimilar SingleClip dimensions OR ColorSpace",myName))
Assert(SingleClip.FrameCount==FrameCount,RT_String("%sDissimilar SingleClip FrameCount",myName))
Valid=DBSC_ScanValidate(ScanDB,MustExist=True,name=myName)
Assert(Valid,RT_String("%sInvalid ScanDB",myName))
Assert(RT_DBaseRecords(ScanDB)==FrameCount,RT_String("%sScanDB Records != FrameCount",myName))
ClipClopCmd_Fn=RT_LocalTimeString+".Txt"
DBSC_WriteFiles(ScanDB,ClipClopCmd_Fn=ClipClopCmd_Fn,Comment=False)
Last.ClipClop(StartClip,EndClip,SingleClip,cmd=ClipClopCmd_Fn)
RT_FileDelete(ClipClopCmd_Fn)
return Last.KillAudio
}

[StainlessS]

Post #10 of many

Dynamically changing borders detector (PART 1)

#######################################################
### DYNAMICALLY CHANGING BORDERS CROPPING FUNCTIONS ***
#######################################################

Function DBSC_DynaCrop(clip c,String ScanDB,String "SpliceDB",int "W",int "H",String "Resizer",
\ Int "CropLimit",Int "CropLimit_Left",Int "CropLimit_Top",Int "CropLimit_Right",Int "CropLimit_Bot",
\ int "RLBT",int "LeftAdd",int "TopAdd",int "RightAdd",int "BotAdd",
\ int "DissolveSkip",
\ String "LogFn",String "LogTMPT",
\ Float "CropThresh",Float "ATM",
\ bool "Show",bool "Debug") {
/*

DBSC_DynaCrop(). With Input SCanDB, Creates output SpliceDB and log, to crop dynamically changing borders from PROGRESSIVE clip.
Requires RoboCrop v0.21+ for RoboCrop output LogFile, to get crop coords.
As well as output SpliceDB, also outputs a user configurable log file (always outputs the log file).

ScanDB, DBase as created via DBSC_Scan().
SpliceDB filename, Optional default "", default "" produces no output DBase, only a LogFn log file.
Where provided with an output SpliceDB filename, will:-
If SpliceDB DOES NOT already Exist, Create zero record SpliceDB in STANDARD format.
If SpliceDB DOES already Exist, Create SpliceDB in SAME format as existing DBase, but first initialled to zero Records.
STANDARD Format SpliceDB consists of 12 Int fields and 1 string[256] field, ie to create zero record STANDARD format DB,
use eg, RT_DBaseAlloc(SpliceDB,0,"iiiiiiiiiiiis")

FieldNumber Type Contents
0 Int Start of Scene frame
1 Int End of Scene frame
2 Int Input clip Width (same for all DB).
3 Int Input clip Height (same for all DB).
4 Int Output clip resize Width (same for all DB).
5 Int Output clip resize Height (same for all DB).
6 Int Scene Crop left (X) offset
7 Int Scene Crop left (Y) offset
8 Int Scene Abs Crop Width
9 Int Scene Abs Crop Heigth
10 Int Scene signed crop width (width relative)
11 Int Scene signed crop width (height relative)
12 String Resizer String (256 chars], Same for all DB
You could provide an EXISTING SpliceDB with additional (to STANDARD) fields, where DBSC_DynaCrop() would 1st re-allocate with
zero records (previous contents lost, but will have same number, type and size fields), and then the STANDARD fields are filled
with the dynacrop data, any additional fields will be zeroed, ie Float==0.0, String=="".
You could later fill your addional (to STANDARD) fields in some fashion, with whatever you want via some other function of your
own making. These additional user added fields could be accessed later via a DBSC_DynaSplice() 'Func' user function.

W,H, Output resize dimensions, Defaults to same size as input clip.
Resizer, String, <Default = "Spline64Resize(_W_,_H_)">. Where '_W_' and '_H_' are replaced with output width and height in the function.
CropLimit, Default 16, Default for all below Crop edges limits if not otherwise set below.
CropLimit_Left, Default CropLimit. Maximum allowed border size on Left edge of clip.
CropLimit_Top, Default CropLimit. Maximum allowed border size on Top edge of clip.
CropLimit_Right, Default CropLimit. Maximum allowed border size on Right edge of clip.
CropLimit_Bot, Default CropLimit. Maximum allowed border size on Bottom edge of clip.
RLBT, Default 15($F). bit flags of edges to crop. 1=Top, 2=Bot, 4=Left, 8=Right. (1+2+4+8=15 = all sides)
LeftAdd, Default 0. Amount to add to detected Left edge, and before CropLimit enforced.
TopAdd, Default 0. Amount to add to detected Top edge, and before CropLimit enforced.
RightAdd, Default 0. Amount to add to detected Right edge, and before CropLimit enforced.
BotAdd, Default 0. Amount to add to detected Bottom edge, and before CropLimit enforced.
DissolveSkip, default 5. Number of frames to skip at either end of scene when detecting borders (try to avoid dissolves and wipes
playing havoc with detections). Current scene must be at least 4 * DissolveSkip otherwise DissolveSkip ignored.
LogFn default "DynaCrop.log". User configurable output log file name.
LogTMPT Log Template, Default="V_NNNN_ = Trim(_S_,_E_).Crop(_CX_,_CY_,_-CW_,_-CH_)._RSZ_"+Chr(9)+Chr(9)+"# Crop_W= _CW_ Crop_H= _CH_"
Default produces eg "V0000 = Trim(0,215).Crop(2,4,-14,-6).Spline64Resize(640,400) # Crop_W= 624 Crop_H= 390"
###
Log TEMPLATE Replacements
"_NNNN_" = trim/scene number, 0 relative
"_S_" = Start frame Number, "_E_" = End frame number
"_CX_", "_CY_", "_CW_","_CH_", Crop x,y,w,h, "_-CW_" and "_-CH_" are -ve crop width and height relative
"_IW_", "_IH_" = Input clip width and height.
"_W_", "_H_" = Resize width and height.
"_RSZ_" = The resizer, defaulted or otherwise.

Example:- Usage of DEFAULT output Log to Trim, Crop, Resize, and Splice [instead of DBSC_DynaSplice()].
Function SpliceTrimsString(String VideoString,int maxim,int "Wid",String "Func") {
Func=Default(Func,"")
HasFunc=(Func!="")
Wid=Default(Wid,4)
GSCript("""
S=""
for(i =0,maxim) {
S=(S!="" && HasFunc) ? RT_StrAddStr(S,".",Func) : S
if(S!="") {S=RT_StrAddStr(S,"+")}
S=RT_StrAddStr(S,VideoString+RT_NumberString(i,width=wid)) # RT_StrAddStr, Avoid v2.58 bug
}
""")
return S
}
AviSource("Test.avi")
FUNC="DBSC_Levels(Show=True)" # DEMO user Func for DBSC_DynaSplice

TRIMS=RT_ReadTxtFromFile("DynaCrop.log")
Lines=RT_TxtQueryLines(TRIMS) # Total lines in text file
Eval(TRIMS) # Trim, Crop, Resize
Eval(SpliceTrimsString("V",Lines-1,Func=Func)) # Splice trims
Return last
###
CropThresh is RoboCrop thresh, Default -32 (RoboCrop DEFAULTED AUTOTHRESH). Best leave alone, will auto adapt.
ATM, RoboCrop ATM, Default 16.0. Best leave alone, will auto adapt.

RETURN:- Returns clip that has been trimmed, cropped, resized and spliced, with optional info if SHOW=true.

*/


Oops, Edited in Blue. Managed to insert an old pre-DEMO version of script.

[StainlessS]

Dynamically changing borders detector (PART 2)


ROBOCROP_DEBUG=False
DBUG=FALSE
c
myName="DBSC_DynaCrop: "
Valid=DBSC_ScanValidate(ScanDB,MustExist=True,name=myName)
Assert(Valid,RT_String("%sInvalid ScanDB\n%s",myName,ScanDB))
Records=RT_DBaseRecords(ScanDB)
Assert(Records==FrameCount,RT_String("%sScanDB Records != FrameCount",myName))
SpliceDB=Default(SpliceDB,"")
c2 = 0
_W_=Default(W,width) _H_=Default(H,height)
Resizer=Default(Resizer,"Spline64Resize(_W_,_H_)") # Resizer to use
TResizer="t."+Resizer
CropThresh=Float(Default(CropThresh,-32.0)) # Absolute Thresh
ATM=Float(Default(ATM,16.0)) # ATM Switched almost OFF
RLBT=Default(RLBT,15)
Assert(RLBT>=1 && RLBT<=15,RT_String("%sRLBT 1->15 Only",myName))
LeftAdd=Default(LeftAdd,0) TopAdd=Default(TopAdd,0) RightAdd=Default(RightAdd,0) BotAdd=Default(BotAdd,0)
CSX=RT_ColorSpaceXMod(c) CSY=RT_ColorSpaceYMod(c,laced=False)
CropLimit=Default(CropLimit,16)
CropLimit_Left = (Default(CropLimit_Left,CropLimit) + CSX/2) / CSX * CSX
CropLimit_Top = (Default(CropLimit_Top,CropLimit) + CSY/2) / CSY * CSY
CropLimit_Right= (Default(CropLimit_Right,CropLimit)+ CSX/2) / CSX * CSX
CropLimit_Bot = (Default(CropLimit_bot,CropLimit) + CSY/2) / CSY * CSY
DissolveSkip=Default(DissolveSkip,5)
Assert(DissolveSkip>=0 && DissolveSkip<=10,RT_String("%sDissolveSkip 0->10 only",myName))
LogFn=Default(LogFn,"DBSC_DynaCrop.log")
LogTMPT=Default(LogTMPT,RT_String("V_NNNN_ = Trim(_S_,_E_).Crop(_CX_,_CY_,_-CW_,_-CH_)._RSZ_\t\t# Crop_W= _CW_ Crop_H= _CH_"))
Show=Default(Show,False) Debug=Default(Debug,false)
(Exist(LogFn)) ? RT_FileDelete(LogFn) : NOP # Delete existing Log
EOL=Chr(10)
FND=RT_String("_RSZ_\n_NNNN_\n_S_\n_E_\n_CX_\n_CY_\n_CW_\n_CH_\n_-CW_\n_-CH_\n_W_\n_H_\n_IW_\n_IH_\n")
TmpLog="~DynaCropTmp_"+RT_LocalTimeString+".log"
(Exist(TmpLog)) ? RT_FileDelete(TmpLog) : NOP # Delete existing TmpLog
GScript("""
if(SpliceDB!="") {
if(Exist(SpliceDB)) {
Fields=RT_DBaseFields(SpliceDB)
Assert(Fields>=13,RT_String("%sInvalid SpliceDB, at least 12 Int fields and 1 String[256] field\n%s",myName,SpliceDB))
for(i=0,11) {
Assert(RT_DBaseFieldType(SpliceDB,i)==1,RT_String("%sSpliceDB field %d is NOT type Int\n%s",myName,i,SpliceDB))
}
Assert(RT_DBaseFieldType(SpliceDB,12)==3,RT_String("%sSpliceDB field 12 is NOT type String\n%s",myName,SpliceDB))
if(RT_DBaseRecords(SpliceDB) != 0) {
TypeString="iiiiiiiiiiiis"
if(Fields>13) {
for(i=13,Fields-1) {
Type=RT_DBaseFieldType(SpliceDB,i)
sz=RT_DBaseFieldSize(SpliceDB,i)
TypeString=TypeString+Select(Type,"b","i","f",RT_String("s%d",sz),"b","f")
}
}
(DEBUG)?RT_DebugF("Re-Alloc zero record SpliceDB, TypeString=%s",TypeString,name=myName):NOP
RT_DBaseAlloc(SpliceDB,0,TypeString)
} else {
(DEBUG)?RT_DebugF("Using existing zero record SpliceDB:- %s",SpliceDB,name=myName):NOP
}
} else {
(DEBUG)?RT_DebugF("Allocating Default format SpliceDB:- %s",SpliceDB,name=myName):NOP
RT_DBaseAlloc(SpliceDB,0,"iiiiiiiiiiiis")
}
}
RoboCrop(SAMPLES=Max(Round(Records/100.0),32),ignore=0.2,laced=False,rlbt=RLBT,LogFn=TmpLog,LogAppend=false,Debug=ROBOCROP_DEBUG)
Wid=Width Hit=Height
ts=RT_ReadTxtFromFile(TmpLog) Outer_x=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) Outer_y=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) Outer_w=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) Outer_h=RT_NumberValue(ts)
Outer_XR=c.Width-Outer_x-Outer_w Outer_YB=c.Height-Outer_y-Outer_h
(DEBUG)?RT_DebugF("Outermost crop Coords = %d,%d,%d,%d (%dx%d)",Outer_x,Outer_y,Outer_XR,Outer_YB,Outer_w,Outer_h,name=myName):NOP
Start=0 End=-1 SceneNo=0 SceneCount=RT_DBaseGetField(ScanDB,Records-1,3)+1
for(n=0,Records-1) {
Start=n End=RT_DBaseGetField(ScanDB,n,1)
if(End-Start+1 >= DissolveSkip*4) {
DStart=Start+DissolveSkip DEnd=End-DissolveSkip
} Else {
DStart=Start DEnd=End
}
IGNORE=2.0 # Allow for significant CGI text
CropC = RoboCrop(thresh=CROPTHRESH,atm=ATM,ignore=IGNORE,laced=False,rlbt=RLBT,start=DStart,end=DEnd,
\ LogFn=TmpLog,LogAppend=false,Debug=ROBOCROP_DEBUG,LeftAdd=LeftAdd,TopAdd=TopAdd,RightAdd=RightAdd,BotAdd=BotAdd)
ts=RT_ReadTxtFromFile(TmpLog) XL=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) YT=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) CW=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) CH=RT_NumberValue(ts)
XR=(Wid-XL-CW) YB=(Hit-YT-CH)
(DBUG)?RT_DebugF("XL=%d YT=%d XR=%d YB=%d",XL,YT,XR,YB,name=myName):NOP
if(XL>CropLimit_Left || YT>CropLimit_Top || XR>CropLimit_Right || YB>CropLimit_Bot) {
RT_DebugF("*** Trim(%d,%d) Scene #%d, %d,%d,%d,%d",Start,End,SceneNo+1,XL,YT,XR,YB,name=myName)
(DEBUG)?RT_DebugF("*** RoboCrop, CROP LIMIT EXCEEDED, Repeating Call with RoboCrop Default AutoThresh & ATM & Ignore=1.0",name=myName):NOP
IGNORE=1.0 # Allow for lil bit CGI text
CropC = RoboCrop(ignore=IGNORE,laced=False,rlbt=RLBT,start=DStart,end=DEnd,
\ LogFn=TmpLog,LogAppend=false,Debug=ROBOCROP_DEBUG,LeftAdd=LeftAdd,TopAdd=TopAdd,RightAdd=RightAdd,BotAdd=BotAdd)
ts=RT_ReadTxtFromFile(TmpLog) TXL=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) TYT=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) TCW=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) TCH=RT_NumberValue(ts)
TXR=(Wid-TXL-TCW) TYB=(Hit-TYT-CH)
(DBUG)?RT_DebugF("TMP: XL=%d YT=%d XR=%d YB=%d",TXL,TYT,TXR,TYB,name=myName):NOP
XL=(XL<=CropLimit_Left) ? XL : (TXL<=CropLimit_Left) ? TXL : Min(XL,TXL)
YT=(YT<=CropLimit_Top) ? YT : (TYT<=CropLimit_Top) ? TYT : Min(YT,TYT)
XR=(XR<=CropLimit_Right)? XR : (TXR<=CropLimit_Right)? TXR : Min(XR,TXR)
YB=(YB<=CropLimit_Bot) ? YB : (TYB<=CropLimit_Bot) ? TYB : Min(YB,TYB)
(DBUG)?RT_DebugF("XL=%d YT=%d XR=%d YB=%d",XL,YT,XR,YB,name=myName):NOP
if(XL>CropLimit_Left || YT>CropLimit_Top || XR>CropLimit_Right || YB>CropLimit_Bot) {
(DEBUG)?RT_DebugF("**** RoboCrop, LIMIT EXCEEDED, Repeating Call with RoboCrop Default AutoThresh & ATM=1.0 & Ingore=0.2",name=myName):NOP
IGNORE=0.2 # No CGI text
CropC = RoboCrop(ignore=IGNORE,laced=False,rlbt=RLBT,start=DStart,end=DEnd,ATM=1.0,
\ LogFn=TmpLog,LogAppend=false,Debug=ROBOCROP_DEBUG,LeftAdd=LeftAdd,TopAdd=TopAdd,RightAdd=RightAdd,BotAdd=BotAdd)
ts=RT_ReadTxtFromFile(TmpLog) TXL=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) TYT=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) TCW=RT_NumberValue(ts)
ts=MidStr(ts,FindStr(ts," ")+1) TCH=RT_NumberValue(ts)
TXR=(Wid-TXL-TCW) TYB=(Hit-TYT-CH)
(DBUG)?RT_DebugF("TMP: XL=%d YT=%d XR=%d YB=%d",TXL,TYT,TXR,TYB,name=myName):NOP
XL=(XL<=CropLimit_Left) ? XL : (TXL<=CropLimit_Left) ? TXL : CropLimit_Left
YT=(YT<=CropLimit_Top) ? YT : (TYT<=CropLimit_Top) ? TYT : CropLimit_Top
XR=(XR<=CropLimit_Right)? XR : (TXR<=CropLimit_Right)? TXR : CropLimit_Right
YB=(YB<=CropLimit_Bot) ? YB : (TYB<=CropLimit_Bot) ? TYB : CropLimit_Bot
(DBUG)?RT_DebugF("XL=%d YT=%d XR=%d YB=%d",XL,YT,XR,YB,name=myName):NOP
}
}
Curr_w=Wid-XL-XR Curr_h=Hit-YT-YB
(DEBUG)?RT_DebugF("Trim(%d,%d) Scene #%d, %d,%d,%d,%d (W=%d,H=%d)",Start,End,SceneNo+1,XL,YT,-XR,-YB,Curr_w,Curr_h,name=myName):NOP
XX=Outer_x+XL YY=Outer_y+YT WW=Curr_w -(c.Width -Outer_x-XL) HH=Curr_h -(c.Height-Outer_y-YT)
REP =
\ Resizer + EOL +
\ RT_NumberString(SceneNo,width=4) + EOL +
\ RT_NumberString(Start) + EOL +
\ RT_NumberString(End) + EOL +
\ RT_NumberString(XX) + EOL +
\ RT_NumberString(YY) + EOL +
\ RT_NumberString(Curr_w) + EOL +
\ RT_NumberString(Curr_h) + EOL +
\ RT_NumberString(WW) + EOL +
\ RT_NumberString(HH) + EOL +
\ RT_NumberString(_W_) + EOL +
\ RT_NumberString(_H_) + EOL +
\ RT_NumberString(c.Width) + EOL +
\ RT_NumberString(c.Height) + EOL
LOG=RT_StrReplaceMulti(LogTMPT,FND,REP)
RT_WriteFile(LogFn,"%s",LOG,append=True)
if(SpliceDB!="") {
RT_DBaseExtend(SpliceDB) # Add a single record to DBase
RT_DBaseSetField(SpliceDB,SceneNo,0,Start) # Scene start frame
RT_DBaseSetField(SpliceDB,SceneNo,1,End) # Scene End frame
RT_DBaseSetField(SpliceDB,SceneNo,2,c.Width) # Input clip width (same for all DB).
RT_DBaseSetField(SpliceDB,SceneNo,3,c.Height) # Input clip Height (same for all DB).
RT_DBaseSetField(SpliceDB,SceneNo,4,_W_) # Output clip resize width (same for all DB).
RT_DBaseSetField(SpliceDB,SceneNo,5,_H_) # Output clip resize height (same for all DB).
RT_DBaseSetField(SpliceDB,SceneNo,6,XX) # Scene Crop left offset
RT_DBaseSetField(SpliceDB,SceneNo,7,YY) # Scene Crop top offset
RT_DBaseSetField(SpliceDB,SceneNo,8,Curr_w) # Scene Abs Crop width
RT_DBaseSetField(SpliceDB,SceneNo,9,Curr_h) # Scene Abs Crop height
RT_DBaseSetField(SpliceDB,SceneNo,10,WW) # Scene signed crop width (width relative)
RT_DBaseSetField(SpliceDB,SceneNo,11,HH) # Scene signed crop Height (height relative)
RT_DBaseSetField(SpliceDB,SceneNo,12,Resizer) # Resizer String, Same for all DB
}
t=c.Trim(Start,End).Crop(XX,YY,WW,HH)
t=Eval(TResizer) # Resize using optional resizer
t = (Show) ? t.RT_Subtitle("#%d/%d S=%d E=%d L=%d\nCrop(\aQ%d+\a-%d,\aQ%d+\a-%d,\aQ-%d\a--%d,\aQ-%d\a--%d) : %dx%d",
\ SceneNo+1,SceneCount,Start,End,End-Start+1,Outer_x,XL,Outer_y,YT,Outer_XR,XR,Outer_YB,YB,Curr_w,Curr_h,align=5) : t
c2 = (c2.IsClip()) ? c2++t : t # Splice
SceneNo=SceneNo+1
n = End # Next iteration n = End + 1, ie Start of Next Scene
}
RT_DBaseSetID(SpliceDB,0,DBSC_Str2ID("DYNS"))
(Exist(TmpLog)) ? RT_FileDelete(TmpLog) : NOP # Delete temp Log
""")
return c2
}

[StainlessS]

Dynamic Changing Borders Splicer

Function DBSC_DynaSplice(clip c,String SpliceDB,String "Func",Bool "DEBUG") {
/*
DBSC_DynaSplice(), Function to trim, crop, resize and splice trims in SpliceDB DBase [as created via DBSC_DynaCrop()].
Optionally can call user function 'Func' on each cropped and resized trim, prior to splicing.
The Func arg must (if used) supply all required args (excluding initial clip) and parenthesis to call the required user function.
Eg, Func="MyFunc(Show=True)", where will be called with explicit OOP style clip as 1st arg (ie DONT INCLUDE clip as first arg in
Func string), called as in ClipName.MyFunc(Show=True) from within DBSC_DynaSplice().
See below DBSC_Levels() for example usage.
After viewing output of this function, might like to create OVERRIDE.TXT file and use DBSC_Override() to insert additional
scene changes into ScanDB (it is easlier to find problem areas with remaining borders once this has been processed, also easier to
spot blended scene transitions and force a scene change there).
*/
c
myName="DBSC_DynaSplice: "
Func=Default(Func,"")
DEBUG=Default(DEBUG,TRUE)
Assert(SpliceDB!="",RT_String("%sNeed SpliceDB name",myName))
Assert(Exist(SpliceDB),RT_String("%sSpliceDB Does not exist\n%s",myName,SpliceDB))
Assert(RT_DBaseGetID(SpliceDB,0)==DBSC_Str2ID("DYNS"),RT_String("%sNot a SpliceDB\n%s",myName,SpliceDB))
Records=RT_DBaseRecords(SpliceDB) Fields=RT_DBaseFields(SpliceDB)
Assert(Records>0,RT_String("%sInvalid SpliceDB, Zero records\n%s",myName,SpliceDB))
Assert(Fields>=13,RT_String("%sInvalid SpliceDB, at least 12 Int fields and 1 String[256] field\n%s",myName,SpliceDB))
oc=0
GScript("""
for(i=0,11) {Assert(RT_DBaseFieldType(SpliceDB,i)==1,RT_String("%sSpliceDB field %d is NOT type Int\n%s",myName,i,SpliceDB))}
Assert(RT_DBaseFieldType(SpliceDB,12)==3,RT_String("%sSpliceDB field 12 is NOT type String\n%s",myName,SpliceDB))
Resizer=RT_DBaseGetField(SpliceDB,0,12) # Resizer String, "_W_" and "_H_" are width and Height
OutW=RT_DBaseGetField(SpliceDB,0,4)
OutH=RT_DBaseGetField(SpliceDB,0,5)
FND_S=RT_String("_W_\n_H_\n")
REP_S=RT_String("%d\n%d\n",OutW,OutH)
Resizer="c.trim(Start,End).Crop(XX,YY,WW,HH)."+RT_StrReplaceMulti(Resizer,FND_S,REP_S)
HasFunc=(Func!="")
Func=(HasFunc)?"tmpc."+Func:Func
for(SceneNo=0,Records-1) {
Start = RT_DBaseGetField(SpliceDB,SceneNo,0)
End = RT_DBaseGetField(SpliceDB,SceneNo,1)
Length = End-Start+1
XX = RT_DBaseGetField(SpliceDB,SceneNo,6)
YY = RT_DBaseGetField(SpliceDB,SceneNo,7)
WW = RT_DBaseGetField(SpliceDB,SceneNo,8)
HH = RT_DBaseGetField(SpliceDB,SceneNo,9)
(DEBUG)?RT_DebugF("#%d Start=%d End=%d Length=%d Crop(%d,%d,%d,%d)",SceneNo,Start,End,Length,XX,YY,WW,HH,name=myName):NOP
tmpc=Eval(Resizer)
tmpc=(HasFunc)?Eval(Func):tmpc
oc = (oc.IsClip()) ? oc++tmpc : tmpc # Splice
}
""")
Return oc
}

[StainlessS]

DEMO User function

Function DBSC_Levels(clip c,Float "Strength",Float "Gamma",Float "MinRngPerc",int "MinLen",Bool "Show",String "SpliceDB",Int "SceneNo") {
/*
DBSC_Levels(), Function to DEMO calling of user function on each trim of Dynamic border cropped clip.
This function can be called via DBSC_DynaSplice(func=FUNC) where FUNC argument string is eg
Func="DBSC_Levels(Strength=0.5,Show=True,SpliceDB=SpliceDB,SceneNo=SceneNo)".
Where, SpliceDB is a DBase as created via DBSC_DynaCrop and SceneNo is record number in that DBase.
Where using SpliceDB and SceneNo args then MUST be supplied exactly as "SpliceDB=SpliceDB,SceneNo=SceneNo" in the Func string.
NOTE, the user function MUST have an initial clip argument, in this case clip c as above. Any other args would best be optional.
NOTE, DBSC_DynaSplice() User functions MUST return clip of same attributes as supplied (size colorspace etc).
See DBSC_DynaCrop() and DBSC_DynaSplice().
*/
c
myName="DBSC_Levels: "
Strength=Max(Min(Float(Default(Strength,0.5)),1.0),0.0)
Gamma=Float(Default(Gamma,1.0))
MinRngPerc=Max(Min(Float(Default(MinRngPerc,33.3)),100.0),0.0)
MinLen=Default(MinLen,25) # If less than MinLen frames, then ignore
Show=Default(Show,False)
SpliceDB=Default(SpliceDB,"")
SceneNo=Default(SceneNo,-1)
DBUG=FALSE # Set TRUE to Demo DBase and SceneNo Usage
GScript("""
Valid = (SpliceDB!="" && SceneNo!= -1) # Basic Validity test
if(DBUG && Show && Valid) {
Assert(RT_DBaseGetID(SpliceDB,0)==DBSC_Str2ID("DYNS"),RT_String("%sNot a SpliceDB\n%s",myName,SpliceDB))
Start = RT_DBaseGetField(SpliceDB,SceneNo,0) # SpliceDB Start Of Scene frame (0 relative)
End = RT_DBaseGetField(SpliceDB,SceneNo,1) # SpliceDB End Of Scene frame (0 relative)
Length = End-Start+1
RT_DebugF("#%d Start=%d End=%d Len=%d",SceneNo+1,Start,End,Length,name=myName) # Show SceneNo as 1 relative
}
if(MinLen<=Framecount && STRENGTH>0.0) { # Auto Luma Levelling
Eval(RT_QueryLumaMinMax(X=8,Y=8,W=-8,H=-8)) # Ignore possible remaining borders
if(IsRGB()) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
QLRng=QLMMMax-QLMMMin+1
CSRng=CSMax-CSMin+1
CSRngMin=CSRng*MinRngPerc/100.0
if(QLRng >= CSRngMin) { # Ignore if < MinRngPerc of CS range
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
(Show)?RT_DebugF("Levels(%3d,%.2f,%3d,%d,%d,Coring=False)",ALMin,Gamma,ALMax,CSMin,CSMax,name=myName):NOP
Levels(ALMin,Gamma,ALMax,CSMin,CSMax,Coring=False)
(Show) ?RT_Subtitle("%sLevels(%3d,%.2f,%3d,%d,%d,Coring=False)",myName,ALMin,Gamma,ALMax,CSMin,CSMax) :NOP
}
} Else if(Show){
RT_DebugF("Skip %3d,%3d Range=%d %.2f%%",QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng,name=myName)
RT_Subtitle("%sSkip %3d,%3d Range=%d %.2f%%",myName,QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng)
}
}
""")

Return Last
}

[StainlessS]

Scan Client Script


# Start DebugView before script to view output

Import("DBSC.avs")

VideoFileName = "AP10_Out.avi"

Start_Fn = "Start_Frames.Txt"
End_Fn = "End_Frames.Txt"
Range_Fn = "Range_Frames.Txt"
Mid_Fn = "Mid_Frames.Txt"
ClipClop_Fn = "ClipClop_Frames.Txt"

InfoDB = VideoFileName+"_Info.DB" # INFO DBase
ScanDB = VideoFileName+"_Scan.DB" # SCAN DBase

Avisource(VideoFileName)

RECYCLE_INFO_DB= True
RECYCLE_SCAN_DB= True

OVERRIDE = "Override.txt" # Override as Filename

DBSC_CreateInfo(InfoDB,RecycleDB=RECYCLE_INFO_DB) # Create INFO DB (REQUIRED), for fast scanning
DBSC_Scan(InfoDB,ScanDB,RecycleDB=RECYCLE_SCAN_DB) # Detect Scene Changes and store in ScanDB

#DBSC_Override(ScanDB,OVERRIDE)
#DBSC_WriteFiles(ScanDB,Start_Fn,End_Fn,Range_fn,Mid_Fn,ClipClop_Fn,VideoFilename=VideoFileName,comment=True)

#Return Last

Return DBSC_SubsClip(InfoDB,ScanDB,Subs=True,Metrics=true,Metrics2=true) # View detections
#Return DBSC_StackSceneChanges(Last,InfoDB,ScanDB) # Check detections START/END of scene STACKED.

# Or Drop through to below to demo DBSC_Select()
#BilinearResize(Width*2,Height*2) # DBSC_Select clips need only be same number of frames as ScanDB detection clip.
SIZE = Round(Height/8.0)
MOTIONCLIP = SubTitle("MOTION",Align=5,size=SIZE,text_color=$80FFC0)
STARTCLIP = SubTitle("START OF SCENE",Align=5,size=SIZE,y=(Height+SIZE)/2,text_color=$C080FF)
ENDCLIP = SubTitle("END OF SCENE" ,Align=5,size=SIZE,y=(Height-SIZE)/2,text_color=$FF80C0)
SINGLECLIP = SubTitle("START OF SCENE\nEND OF SCENE",Align=5,size=SIZE,y=(Height-SIZE)/2,text_color=$C0FF80,lsp=0)
Return DBSC_Select(MOTIONCLIP,ScanDB,STARTCLIP,ENDCLIP,SINGLECLIP)

[StainlessS]

Dynamic Changing Borders Client script

Import("DBSC.avs")

VideoFileName = "AP10_Out.avi"

InfoDB = VideoFileName+"_Info.DB" # INFO DBase
ScanDB = VideoFileName+"_Scan.DB" # SCAN DBase
LogFn = VideoFileName+"_User.Log"
SpliceDB = VideoFileName+"_Splice.DB"

RECYCLE_INFO_DB= True
RECYCLE_SCAN_DB= True
OVERRIDE = "Override.txt" # Override as Filename

Avisource(VideoFileName)

DBSC_CreateInfo(InfoDB,RecycleDB=RECYCLE_INFO_DB) # Create INFO DB (REQUIRED), for fast scanning
DBSC_Scan(InfoDB,ScanDB,RecycleDB=RECYCLE_SCAN_DB) # Detect Scene Changes and store in ScanDB

#DBSC_Override(ScanDB,OVERRIDE)

RLBT=$C # Left and Right only


# Return one of two below funcs, DBSC_DynaCrop MUST have been called (at some time) before DBSC_DynaSplice
Return DBSC_DynaCrop(ScanDB,SpliceDB=SpliceDB,rlbt=RLBT,LeftAdd=0,RightAdd=0,TopAdd=1,BotAdd=1,LogFn=LogFn,Show=True,Debug=True)

######################
######################
######################

# This one is best done is separate script when DBSC_DynaCrop() has already created Splice DBase, OR call DBSC_DynaCrop first
# and then comment out DBSC_DynaCrop line.

#FUNC="DBSC_Levels(Show=True,SpliceDB=SpliceDB,SceneNo=SceneNo)" # DEMO user Func for DBSC_DynaSplice
#FUNC="DBSC_Levels(Show=False)"
FUNC=RT_Undefined # Dont call user Func

#Return DBSC_DynaSplice(SpliceDB,Func)

######################
######################
######################

# Or Even below if comment out both DBSC_DynaCrop and DBSC_DynaSplice, but with no user Func.

TRIMS=RT_ReadTxtFromFile(LogFn)
Lines=RT_TxtQueryLines(TRIMS) # Total lines in text file (Or RT_FileQueryLines)
Eval(TRIMS) # Eval trims,crop,resize
Eval(SpliceTrimsString("V",Lines-1,Func=Func)) # Splice trims
Return last

Function SpliceTrimsString(String VideoString,int maxim,int "Wid",String "Func") {
Func=Default(Func,"")
HasFunc=(Func!="")
Wid=Default(Wid,4)
GSCript("""
S=""
for(i =0,maxim) {
S=(S!="" && HasFunc) ? RT_StrAddStr(S,".",Func) : S
if(S!="") {S=RT_StrAddStr(S,"+")}
S=RT_StrAddStr(S,VideoString+RT_NumberString(i,width=wid)) # RT_StrAddStr, Avoid v2.58 bug
}
""")
return S
}

[StainlessS]

DBSC Update to v0.04, zip in 1st post.

[StainlessS]

Post #10 updated (FIXED): Oops, Managed to insert an old pre-DEMO version of script [EDIT: Added Func related stuff].

Example:- Usage of DEFAULT output Log to Trim, Crop, Resize, and Splice [instead of DBSC_DynaSplice()].

Function SpliceTrimsString(String VideoString,int maxim,int "Wid",String "Func") {
Func=Default(Func,"")
HasFunc=(Func!="")
Wid=Default(Wid,4)
GSCript("""
S=""
for(i =0,maxim) {
S=(S!="" && HasFunc) ? RT_StrAddStr(S,".",Func) : S
if(S!="") {S=RT_StrAddStr(S,"+")}
S=RT_StrAddStr(S,VideoString+RT_NumberString(i,width=wid)) # RT_StrAddStr, Avoid v2.58 bug
}
""")
return S
}
AviSource("Test.avi")
FUNC="DBSC_Levels(Show=True)" # DEMO user Func for DBSC_DynaSplice

TRIMS=RT_ReadTxtFromFile("DynaCrop.log")
Lines=RT_TxtQueryLines(TRIMS) # Total lines in text file
Eval(TRIMS) # Trim, Crop, Resize
Eval(SpliceTrimsString("V",Lines-1,Func=Func)) # Splice trims
Return last


Also changed in post#15


TRIMS=RT_ReadTxtFromFile(LogFn)
Lines=RT_TxtQueryLines(TRIMS) # Total lines in text file (Or RT_FileQueryLines)
Eval(TRIMS) # Eval trims,crop,resize
Eval(SpliceTrimsString("V",Lines-1,Func=Func)) # Splice trims
Return last

Function SpliceTrimsString(String VideoString,int maxim,int "Wid",String "Func") {
Func=Default(Func,"")
HasFunc=(Func!="")
Wid=Default(Wid,4)
GSCript("""
S=""
for(i =0,maxim) {
S=(S!="" && HasFunc) ? RT_StrAddStr(S,".",Func) : S
if(S!="") {S=RT_StrAddStr(S,"+")}
S=RT_StrAddStr(S,VideoString+RT_NumberString(i,width=wid)) # RT_StrAddStr, Avoid v2.58 bug
}
""")
return S
}


EDIT: Also might want to change to This, Post 13 Updated.

Function DBSC_Levels(clip c,Float "Strength",Float "Gamma",Float "MinRngPerc",int "MinLen",Bool "Show",String "SpliceDB",Int "SceneNo") {
/*
DBSC_Levels(), Function to DEMO calling of user function on each trim of Dynamic border cropped clip.
This function can be called via DBSC_DynaSplice(func=FUNC) where FUNC argument string is eg
Func="DBSC_Levels(Strength=0.5,Show=True,SpliceDB=SpliceDB,SceneNo=SceneNo)".
Where, SpliceDB is a DBase as created via DBSC_DynaCrop and SceneNo is record number in that DBase.
Where using SpliceDB and SceneNo args then MUST be supplied exactly as "SpliceDB=SpliceDB,SceneNo=SceneNo" in the Func string.
NOTE, the user function MUST have an initial clip argument, in this case clip c as above. Any other args would best be optional.
NOTE, DBSC_DynaSplice() User functions MUST return clip of same attributes as supplied (size colorspace etc).
See DBSC_DynaCrop() and DBSC_DynaSplice().
*/
c
myName="DBSC_Levels: "
Strength=Max(Min(Float(Default(Strength,0.5)),1.0),0.0)
Gamma=Float(Default(Gamma,1.0))
MinRngPerc=Max(Min(Float(Default(MinRngPerc,33.3)),100.0),0.0)
MinLen=Default(MinLen,25) # If less than MinLen frames, then ignore
Show=Default(Show,False)
SpliceDB=Default(SpliceDB,"")
SceneNo=Default(SceneNo,-1)
DBUG=FALSE # Set TRUE to Demo DBase and SceneNo Usage
GScript("""
Valid = (SpliceDB!="" && SceneNo!= -1) # Basic Validity test
if(DBUG && Show && Valid) {
Assert(RT_DBaseGetID(SpliceDB,0)==DBSC_Str2ID("DYNS"),RT_String("%sNot a SpliceDB\n%s",myName,SpliceDB))
Start = RT_DBaseGetField(SpliceDB,SceneNo,0) # SpliceDB Start Of Scene frame (0 relative)
End = RT_DBaseGetField(SpliceDB,SceneNo,1) # SpliceDB End Of Scene frame (0 relative)
Length = End-Start+1
RT_DebugF("#%d Start=%d End=%d Len=%d",SceneNo+1,Start,End,Length,name=myName) # Show SceneNo as 1 relative
}
if(MinLen<=Framecount && STRENGTH>0.0) { # Auto Luma Levelling
Eval(RT_QueryLumaMinMax(X=8,Y=8,W=-8,H=-8)) # Ignore possible remaining borders
if(IsRGB()) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
QLRng=QLMMMax-QLMMMin+1
CSRng=CSMax-CSMin+1
CSRngMin=CSRng*MinRngPerc/100.0
if(QLRng >= CSRngMin) { # Ignore if < MinRngPerc of CS range
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
(Show)?RT_DebugF("Levels(%3d,%.2f,%3d,%d,%d,Coring=False)",ALMin,Gamma,ALMax,CSMin,CSMax,name=myName):NOP
Levels(ALMin,Gamma,ALMax,CSMin,CSMax,Coring=False)
(Show) ?RT_Subtitle("%sLevels(%3d,%.2f,%3d,%d,%d,Coring=False)",myName,ALMin,Gamma,ALMax,CSMin,CSMax) :NOP
}
} Else if(Show){
RT_DebugF("Skip %3d,%3d Range=%d %.2f%%",QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng,name=myName)
RT_Subtitle("%sSkip %3d,%3d Range=%d %.2f%%",myName,QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng)
}
}
""")

Return Last
}

[StainlessS]

v0.051, see first post for zip.

Added a DBSC_DynaSplice User function to the DBSC.avs script, DBSC_ProcessByDAR.


Function DBSC_ProcessByDAR(clip c,String "SpliceDB",Int "SceneNo",Float "FDAR",
\ Float "AR1",Float "AR2",Float "AR3",Float "AR4",Float "AR5",Float "AR6",Float "AR7",Float "AR8",
\ String "FN",Bool "Show") {
/*
DBSC_DynaSplice() User function (called by DBSC_DynaSplice on each scene).
Creates ClipClop command text file based on letterbox cropped Display Aspect Ratio of each scene, enables selective
processing based on a scenes Display Aspect Ratio where DAR changes within a clip.

FDAR = Full Frame Display Aspect Ratio of Source clip (incl letterboxing).
ARx Display Aspect Ratio of image area only (Excluding letterboxing)
Always chooses one of the given ARx clip index numbers (ie never uses source clip ie clipclop clip 0
unless all ARx Aspect ratios supplied as 0.0.)
You can switch off ARx by setting to zero, however, you need to supply ClipClop Replacement clips up
to the maximum ARx number that may be non zero. Suggest always supply Source + 8 replacment clips to clipclop,
and just use dummy clip where unused (ARx==0.0).

Set Eg

FUNC="DBSC_ProcessByDar(SpliceDB=SpliceDB,SceneNo=SceneNo,FDAR=4.0/3.0,Show=True)"

And use in Dynacrop client script eg
Return DBSC_DynaSplice(SpliceDB,func=FUNC)

Use avs script eg (where ARx are processed clips for chosen AR)
ClipClop(SourceClip,AR1,AR2,AR3,AR4,AR5,AR6,AR7,AR8,cmd="ClipClopCmd.txt",Show=True)
*/
myName="DBSC_ProcessByDar: "
c
Assert(Defined(SpliceDB),RT_String("%sSpliceDB Not Supplied",myName))
Assert(Defined(SceneNo),RT_String("%sSceneNo Not Supplied",myName))
FDAR = Float(Default(FDAR,16.0/9.0))
###
AR1 = Float(Default(AR1,2.40)) # If nearest to this, then use ClipClop clip 1
AR2 = Float(Default(AR2,2.35))
AR3 = Float(Default(AR3,1.85))
AR4 = Float(Default(AR4,16.0/9.0))
AR5 = Float(Default(AR5,1.66))
AR6 = Float(Default(AR6,16.0/10.0))
AR7 = Float(Default(AR7,1.5))
AR8 = Float(Default(AR8,4.0/3.0)) # If nearest to this, then use ClipClop clip 8
###
FN=Default(FN,"ClipClopCmd.txt")
Show=Default(Show,False)
GScript("""
Valid = (SpliceDB!="" && SceneNo>= 0) # Basic Validity test, DO NOTHING if bad
if(Valid) {
(SceneNo==0) ? RT_FileDelete(FN) : NOP
Start = RT_DBaseGetField(SpliceDB,SceneNo,0) # SpliceDB Start Of Scene frame (0 relative)
End = RT_DBaseGetField(SpliceDB,SceneNo,1) # SpliceDB End Of Scene frame (0 relative)
Length = End-Start+1
IWID = RT_DBaseGetField(SpliceDB,SceneNo,2) # Source Clip Width
IHIT = RT_DBaseGetField(SpliceDB,SceneNo,3) # Source Clip Height
XX = RT_DBaseGetField(SpliceDB,SceneNo,6) # Left Crop
YY = RT_DBaseGetField(SpliceDB,SceneNo,7) # Top Crop
WW = RT_DBaseGetField(SpliceDB,SceneNo,8) # Width
HH = RT_DBaseGetField(SpliceDB,SceneNo,9) # Height
SAR = (FDAR * IHIT)/ IWID # Sample (pixel) Aspect ratio from source clip
AR = (SAR * WW) / HH # Display Aspect Ratio of cropped scene
AR0 = 99.9 BestAR = AR0 BestARix = 0 BestARdif=AR0 # If ALL ARx 0.0, Cause ClipCLop to choose source clip 0.

TmpDif = Abs(AR-AR1) If(AR1>0.0 && TmpDif<BestARdif) {BestAR = AR1 BestARix = 1 BestARdif=TmpDif}
TmpDif = Abs(AR-AR2) If(AR2>0.0 && TmpDif<BestARdif) {BestAR = AR2 BestARix = 2 BestARdif=TmpDif}
TmpDif = Abs(AR-AR3) If(AR3>0.0 && TmpDif<BestARdif) {BestAR = AR3 BestARix = 3 BestARdif=TmpDif}
TmpDif = Abs(AR-AR4) If(AR4>0.0 && TmpDif<BestARdif) {BestAR = AR4 BestARix = 4 BestARdif=TmpDif}
TmpDif = Abs(AR-AR5) If(AR5>0.0 && TmpDif<BestARdif) {BestAR = AR5 BestARix = 5 BestARdif=TmpDif}
TmpDif = Abs(AR-AR6) If(AR6>0.0 && TmpDif<BestARdif) {BestAR = AR6 BestARix = 6 BestARdif=TmpDif}
TmpDif = Abs(AR-AR7) If(AR7>0.0 && TmpDif<BestARdif) {BestAR = AR7 BestARix = 7 BestARdif=TmpDif}
TmpDif = Abs(AR-AR8) If(AR8>0.0 && TmpDif<BestARdif) {BestAR = AR8 BestARix = 8 BestARdif=TmpDif}

RT_WriteFile(FN,"%d %d,%d # SceneNo=%d Crop(%d,%d,%d,%d) CropAR=%.2f BestAR=%.2f",
\ BestARix,Start,End,SceneNo+1,XX,YY,WW,HH,AR,BestAR,Append=True)
RT_DebugF("#%d Start=%d End=%d Len=%d CropAR=%.2f BestAR=%.2f Clip=AR%d",
\ SceneNo+1,Start,End,Length,AR,BestAR,BestARix,
\ name=myName)
if(Show) {
RT_Subtitle("#%d Start=%d End=%d Len=%d CropAR=%.2f BestAR=%.2f Clip=AR%d",
\ SceneNo+1,Start,End,Length,AR,BestAR,BestARix)
}
}
""")
Return Last # Return Clip Unchanged # EDIT: Unless Show=True
}


I cant update to current script as it uses quite a few updates to RT_Stats that will not be available for a time.
Previous posts not updated.

EDIT: Function above added due to this post: http://forum.doom9.org/showthread.php?t=171777

EDIT: Produces ClipClop command file, eg

5 0,254 # SceneNo=1 Crop(18,2,692,572) CropAR=1.29 BestAR=1.33
5 255,332 # SceneNo=2 Crop(16,2,694,572) CropAR=1.29 BestAR=1.33
5 333,347 # SceneNo=3 Crop(20,2,688,572) CropAR=1.28 BestAR=1.33
5 348,355 # SceneNo=4 Crop(10,2,698,572) CropAR=1.30 BestAR=1.33
5 356,390 # SceneNo=5 Crop(10,2,698,572) CropAR=1.30 BestAR=1.33
5 391,455 # SceneNo=6 Crop(12,2,690,572) CropAR=1.29 BestAR=1.33
5 456,468 # SceneNo=7 Crop(10,2,700,572) CropAR=1.31 BestAR=1.33
5 469,476 # SceneNo=8 Crop(8,2,702,572) CropAR=1.31 BestAR=1.33
5 477,488 # SceneNo=9 Crop(12,2,698,572) CropAR=1.30 BestAR=1.33
5 489,548 # SceneNo=10 Crop(10,2,700,572) CropAR=1.31 BestAR=1.33
5 549,569 # SceneNo=11 Crop(10,2,700,572) CropAR=1.31 BestAR=1.33

I did not have a suitable clip handy so is above always selecting the same clip (clip 5).
EDIT: In above changed script, would have always returned clip 8.

EDIT: Purpose, to create a ClipClop command file that selects clip based on scene Display Aspect Ratio of the image
only area (ie without letter boxing).

[StainlessS]

Updated previous post, and zip updated to v0.051 in 1st post.

Changed slightly, and had a name change to DBSC_ProcessByDAR() [So much nicer than DBSC_Lyris() :) ]

[johnmeyer]

I have a huge analog video transfer project (dozens of hours of home video) and I want to cut it into scenes. So, I finally have a reason to put this wonderful technology through the ringer.

My video is interlaced NTSC DV video, captured via Firewire. Scene detection works well most of the time, but I am having some failures that don't make sense. At the moment, I am using the default, using your "demo" script to drive the process. Here is a two-second clip where the scene detection fails:

Scene Detect Fail.avi (https://www.mediafire.com/?644av8hfb1orifv)

As you can see, the scene change actually happens in the middle of a frame, where one field in the frame is from the previous frame, and the other field is from the next frame.

If I bob the video, the scene is detected correctly.

Before I start playing around with all the detection parameters, I thought I'd ask whether I need to change something when dealing with interlaced video. I searched this thread and also the documentation for "interlaced," but didn't find anything.

Of course I can just simply bob the video and then divide the frame detect numbers by 2, but I thought I'd ask first.

[johnmeyer]

OK, I used separatefields() because it seems like, compared to bob() it would halve the number of pixels the function must analyze. This seems to work perfectly for my purposes. After I run the function, I put the vectors in Excel and divide by two (I'm too lazy to mod DBSC to figure out where to divide by 2).

So, I'm up and running on the first part of this project.

Thanks!

[edit]P.S. It sure would be nice to have a progress dialog during the long analysis time. It runs much faster than real time, but for a 5-6 hour project, it would be nice to see how close it's getting to the end.

[StainlessS]

Mobile in pub at minute, try debug =true for progress.
Edit: debugview

EDIT: Also check out the Override option.


EDIT: I've just seen the most delicious device ever, Samsung Galaxy Beam, not available in the UK due to import restrictions
(copyright / patent or something). Not the latest but blew my socks off with the projector. Me wants one.

Samsung Galaxy Beam GT-i8530 (dont forget the GT bit). Totally amazing.

EDIT: Here- http://www.samsung.com/uk/consumer/mobile-devices/smartphones/android/GT-I8530BAABTU



and http://tech.firstpost.com/reviews/samsung-galaxy-beam-gt-i8530-review-38165.html

I wants a projector device now I've seen one. (above one is old, wants a better one).

[johnmeyer]

A handheld projector ?? Wow! Next they'll figure out how to make it do 3D and you can project Princess Lea in the pub. Quite a conversation starter.

I'm editing the project that I cut into scenes. About every ten scenes it either misses a scene, or it cuts a scene where there isn't one. I've created about half a dozen scene detection scripts over the years, and when they failed, there was usually a common reason (e.g., person entering the scene close the camera; big camera shake; or, for missing a scene, two scenes that have virtually the same content and contrast, e.g., a jump cut). With this approach, and the default values, it misses some absolutely dead-obvious scene changes, but then sometimes creates a new scene where nothing much has changed from one frame to the next.

I have another set to do tomorrow, and perhaps I'll look into doing a little tweaking.

[StainlessS]

I have not tried this DBSC thing with interlaced material, in fact the main test clips I've used were in fact deinterlaced 80's
UK TV series (Auf Wiedersehen Pet series 1 & 2) which looks to be taken from VHS and 'restored' (badly, everything way too sharp) to make the DVD.
It was good material to do the Dynamic cropping stuff, as it changes at pretty much every scene change.
Myself, I always deinterlace if required, I am not interested in my mp4's playing on TV (I dont even have a TV for some some years),
I usually watch mp4 on a laptop about 2 feet away, situated just left of my computer
monitor and I usually mostly listen rather than watch, but sit up for any good bits, also sometimes watch on tablet. [That reminds me, I was watching
24 Series 2 last night, just cranking up my old laptop to watch Episode 9 {15 more episodes and 3 more following series to watch till I can get some sleep, got S1 to S5, gonna be the longest day of my life}].
Spotted on UK's CEX (WeBuy.com) Lexx Sci-Fi saga for £55 (full set), gonna have to get that, hope it aint the USA version with all of the cuts, hear the German release is the better one.

I'm still working on RT_Stats updates, quite a lot of things to do there and these scripts will not be updated until that is done.
Maybe at some future point I could look into interlaced detection but I would want to get it working very well with progressive beforehand.

I downloaded your clip and will take a peek.

Here Galaxy Beam 2, (apr 2014) on ebay http://www.ebay.co.uk/bhp/samsung-galaxy-beam
seen some ridiculously priced at $3000.00.

and on Wikipedia, list of projector phones: http://en.wikipedia.org/wiki/Projector_phone (Rare in the west)

Guy showing me his Beam last night, said that there were import restriction into UK, he (Egyptian) bought his in Pakistan
where they are quite popular. Next thing I knew, his was projecting a porno clip onto the ceiling (in the pub), I knew then
that I gotta have one :)

[johnmeyer]

I wanted to try this again, this time on a progressive 16 fps video made from a movie film that I transferred. I've re-read all your code, and your DEMO function, but I must be really thick: how do I adjust the sensitivity?

[StainlessS]

I decided that would be better to have too sensitive defaults as it is easier to spot SC detected mistakes and override them,
rather than not detect and override to detected. The DBSC_StackSceneChanges() func allows to view detected SC's and so can easily
spot detection mistakes and set User Overrides (whereas spotting not-detected mistakes is a lot more user time consuming).
The lousy docs on args had wrong default values, have amended to same as real defaults below, and added a little bit more.
Low framerate is always gonna be problematic for scene change detection due to inherent jerkiness, may need to raise both dFact and dMinim
for low framerate.


DBSC_Scan().
InfoDB, EXISTING INFO DBSC_CreateInfo created DBase
ScanDB, Name of Scene Change SCAN DBase, created by this function.
dFact, Default 1.25 (greater than 1.0)
dMinim, Default 6.0, (greater than 0.1) dFact and dMinim , Control how much bigger difference between Curr->Next has to be compared
to Prev->Curr and Next->Next+1, to detect scene change.
Curr->Next has to be greater than (dFact * Prev->Curr + dMinim), and also greater than (dFact * Next->Next+1 + dMinim)
Dfact=1.25, works pretty good but is on a hair trigger ie very sensitive, dMinim=6.0 reduces/controls that sensitivity.
If clip is anime, then is quite likely that some frame differences are 0.0, so even a small difference next to 0.0 diff frames
will be infinitely greater than 0.0, in this situation dMinim will be the sole govenor of scene detection and so should be of
reasonable value, maybe about 6.0 -> 12.0 would be good choices.
dMinLen, Default 6. Minimum frame length of current scene that qualifies for scene change detection.
dCorr, Default 0.75. Used in flash detection, Luma correlation between pre-flash frame and potential no-longer-flashing frame.
Probably best left at default.
DupTh, Default 0.5. For Anime, if Average Pixel difference Diff(n-1,n) < DupTh, we assume duplicate frame and instead use n-2 in
comparisons [same for Diff(n+1,n+2)]. For non-anime can switch off ie 0.0.
POV, Default 60.0%. If fewer than POV% luma pixels changed by at least 1 luma level, then NOT a scene change, overrides any
dFact,dMinim detection.


I must be really thick
nah, I'm just really bad at docs.

EDIT: DBSC_SubsClip(metrics=true), shows metrics.
DBSC_StackSceneChanges() shows metrics at detected scene changes

[johnmeyer]

I am doing another set of film today, so tomorrow I'll have a chance to play with this again. Thanks!

[johnmeyer]

I spent quite a few hours today trying to create a better scene detection function. I have something that works pretty well, but this RT_Stats function should beat my approach.

I then finally rolled up my sleeves and started reading the code in these various functions, and I think one of the reasons why DBSC_Scan misses so many scene changes in my low frame rate film captures has to do with the decision to look at differences between Curr, Next, & Prev, rather than using ratios. In looking at the parameters as I walk through my video frame by frame, some sections have Prv, Cur, Nxt numbers in the range of 2-5, and other sections have numbers in the range of 10-15. Obviously, a difference of 2-3 is s BIG deal when the numbers are down in the low single digits, but that same difference doesn't indicate much of a change when the numbers are 3x to 5x larger. Thus, I think you'd get much more uniform detection if you change the code to base decisions on the ratio of cur/prv and cur/nxt rather than (cur - prv) and (cur - nxt).

[StainlessS]

John,
If I understand you correctly, what you are suggesting is pretty much what is already in use.
from two (EDIT: three) posts ahead:
Curr->Next has to be greater than (dFact * Prev->Curr + dMinim), and also greater than (dFact * Next->[Next+1] + dMinim)
(EDIT: where Curr->Next means average pixel difference between current and next frames)

where Curr->Next difference is what you have referred to as 'cur' etc.
dFact is the minimum required ratio but with a dMinim offset to avoid problems with infinite ratio where Prev->Curr [or Next->(Next+1)] has a difference of zero (eg anime).
[EDIT: When Prev->Curr is zero, any non zero value of Curr->Next is infinitely greater than 0*dFact, hence dMinim.
cur/prv > dFact same as cur > (dFact * prv) ignoring dMinim.
We actually use multiply and compare instead of divide, to avoid divide by zero where eg Prev->Curr is static,
so you could actually allow dMinim to be zero (remove Assert check) without a divide by zero error, but erroneous detections will be rife where eg
static sequences with just a single pixel changing by 1 level.
You can reduce dMinim to as low as 0.1 if required [without Assert check removal]
(but I would not advise that, with dMinim very low, you get scene change detections in eg black only frame sequences with just a few
non pure black pixels, although for that to happen I think dMinim would have to be below what is minimum allowed.
Suggest lowest in-use dMinim about 0.5, and about 6 -> 12 for jerky anime with dupes)]

Perhaps I'm missing something.

[johnmeyer]

Perhaps I'm missing something.First of all, I don't know what code I was reading when I saw the "pvr" and "nxt" variables. I must have been dreaming.

The part that I think you are missing is the idea of scaling your metrics. What I have found with various frame difference metrics, including the ones you have developed, is that the metrics for some sections of video show very little difference between frames, especially if you have really low contrast (typical of an overexposed movie film clip). You might have metric numbers like 2.2, 3.2, 1.5, 2.1, etc., and then the scene change might only show up as 4.0. As a difference, this isn't much, but as a ratio -- especially if you keep a moving average of the last 2-5 frames, it sticks out like a sore thumb.

Your detection scheme basically uses a linear equation: y = mx +b, with the intercept moved in order to get around divide by zero problems, but which also changes the sensitivity. Using an intercept shift to deal with divide by zero seems like the wrong solution to that problem, especially since it also affects the sensitivity.

To give you an idea of what I'm talking about, here is some code I posted a few years ago. The problem was figuring out how to handle a fairly common problem with both capture cards and also streaming video. The problem is that if they can't keep up, they will drop a frame. Then, to avoid getting the audio out of sync, they will later duplicate a frame. I wrote a script that did two things:

1. Whenever a duplicate is detected, delete that dup.

2. In the vicinity of the duplicate frame, look for the biggest "jump" between frames, which indicates that a frame was dropped, and put a motion estimated frame into that gap area.

Didée came up with a very clever way to make this work, which he posted in a thread entitled: Inverse of Decimate() ...? (http://forum.doom9.org/showthread.php?p=1409324#post1409324).

I needed a more reliable way to detect the big jumps in motion. Here is one function from that code which demonstrates two things:

1. Using a ratio rather than a difference for detecting large gaps in motion. By using a ratio, the detection logic self-regulates to be more sensitive when scene changes are subtle.

2. Using a moving average to smooth out the variation from one frame to the next, so that the scene change "blip" becomes more obvious, and less prone to the somewhat random variations in the difference metrics.

I originally posted this code in this thread (edit: fixed link):

Automatically fix dups followed (eventually) by drops (http://forum.doom9.org/showthread.php?t=161758)

#----------------
#This function returns a white clip whenever a big jump is detected; otherwise a black clip is returned
#Each YDiff must eliminate Ydiff=0 (duplicate) from moving average

function GenerateMask (clip c)
{
MyMask=c.ScriptClip("""
\ (( (YDifferenceFromPrevious(selectevery(source, 1, 2)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, 1)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, 2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, 1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -1)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -1)) )
\ +
\ (YDifferenceFromPrevious(selectevery(source, 1, -2)) < 0.3 ?
\ YDifferenceFromPrevious(selectevery(source, 1, -3)) :
\ YDifferenceFromPrevious(selectevery(source, 1, -2)) )
\ )/4) /
\ (YDifferenceFromPrevious(source) + 0.01) <= JumpThresh
\ ? WhiteFrame : BlackFrame """)
return MyMask
}

[StainlessS]

The dMinim arg was introduced to deliberately reduce sensitivity as in anime dupe followed big jerky difference.
I'll look at your script soon (bit busy).

The link shows as problem (suspect copied from search list of links which expire after a time).

EDIT: Here:- Automatically fix dups followed (eventually) by drops http://forum.doom9.org/showthread.php?p=1510780#post1510780

[johnmeyer]

Thanks. I fixed the link.

[StainlessS]

John, I've squished equivalent functionality into an existing multi-instance function, not really of much use as it stands but perhaps of use in testing.

The function will produce nonsense results as the base logic expects some kind of interlock where you cannot have consecutive end of scene frames.
We switched your script around a bit as we nearly always detect on EOS (with following mandatory SOS).

Here tis


Function JMSC(clip dclip,clip start,clip end,clip motion,float "dLim",Float "dMin",Float "JumpThresh",Float "dChromaW",bool "Show") {
myName="JMSC: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
Assert(RT_FunctionExist("RoboCrop"),myName+"Essential RoboCrop plugin installed, http://forum.doom9.org/showthread.php?t=168053")
dLim=Float(Default(dLim,0.2))
dMin=Float(Default(dLim,0.01))
JumpThresh=Float(Default(JumpThresh,0.75))
dChromaW=Float(Default(dChromaW,1.0/3.0))
Show=Default(Show,False)
Assert(dChromaW>=0.0 && dChromaW<=1.0,myName+"dChromaW 0.0->1.0")
Assert(dMin>0.0,myName+"0.0 < dMin")

dclip=dclip.RoboCrop(WMod=4,HMod=4,Laced=False,align=True)
SIZE = Motion.Height / 8
CENT = Motion.Height / 2
Start =(Show) ? Start.SubTitle("START OF SCENE",align=5,size=SIZE,y=CENT-SIZE,Text_Color=$00FF00) : Start
End =(Show) ? End.SubTitle("END OF SCENE",align=5,size=SIZE,y=CENT+SIZE,Text_Color=$FF00FF) : End
Motion=(Show) ? Motion.SubTitle("MOTION",align=5,size=SIZE) : Motion
# Make format string only once, not at every frame
Fmt=(Show) ? "%d ] %d#%-4d MovTot=%.3f : MovAvg=%.3f : Result=%.3f\n" +
\ "%6.2f %6.2f %6.2f \a%c%6.2f\a- %6.2f %6.2f %6.2f\n"
\ : ""
FuncS="""
Function Fn@@@(clip clp,clip start,clip end,clip dclip,Float dLim,Float dMin,Float JumpThresh,Float dChromaW,Bool Show,String Fmt) {
clp
n = current_frame
If(Prev@@@ == n) { # Cache failure, requested same frame again. Rightshift everything, will be Leftshifted later.
Global G@@@=F@@@ Global F@@@=E@@@ Global E@@@=D@@@ Global D@@@=C@@@ Global C@@@=B@@@ Global B@@@=A@@@
} Else If(Prev@@@ + 1 != n) { # User jumped about, dont you just hate users!
# We want status at current_frame-1, however to re-detect there we need fill Globals as if at current_frame-2
# You could Preroll more than 1 for some other reason but need to read Globals @ 1 prior to Preroll count.
n = n - 2 # Pretend we are 2 prior to current_frame, we need to fill globals and re-detect
Global Start@@@ = (Start@@@ > n) ? n : Start@@@
Global EOS@@@=False
# Get ALL Globals as if n is at correct current_frame, ie D = Curr->Next at n
Global A@@@ = RT_FrameDifference(dclip,dclip,n=n-3,n2=n-2,chromaweight=dChromaW) # Not really needed
Global B@@@ = RT_FrameDifference(dclip,dclip,n=n-2,n2=n-1,chromaweight=dChromaW)
Global C@@@ = RT_FrameDifference(dclip,dclip,n=n-1,n2=n+0,chromaweight=dChromaW)
Global D@@@ = RT_FrameDifference(dclip,dclip,n=n+0,n2=n+1,chromaweight=dChromaW) # Curr->Next at pretend n
Global E@@@ = RT_FrameDifference(dclip,dclip,n=n+1,n2=n+2,chromaweight=dChromaW)
Global F@@@ = RT_FrameDifference(dclip,dclip,n=n+2,n2=n+3,chromaweight=dChromaW)
Global G@@@ = RT_FrameDifference(dclip,dclip,n=n+3,n2=n+4,chromaweight=dChromaW)
n = n + 1 # Now that Globals are filled, continue as if at the following iteration.
}
# We Detect at End Of Scene where D is Curr->Next (D is middle of A -> G)
A=A@@@ B=B@@@ C=C@@@ D=D@@@ E=E@@@ F=F@@@ G=G@@@ # Get locals, As set by previous iteration
# Below loop can do a 'Rewind/Preroll' re-detect where SC status is dependent upon previous frames eg user jumped about.
# In normal (non-jump-about) use, below loop is one iteration only.
for(n=n,current_frame) {
SOS = EOS@@@ # Start Of Scene if Prev frame was End Of Scene
A=B B=C C=D D=E E=F F=G # Pass the parcel, from previous interation (re-use metrics)
G = RT_FrameDifference(dclip,dclip,n=n+3,n2=n+4,chromaweight=dChromaW) # New single metric
MovTot = ((F < dLim) ? G : F) # Remaining lines of JMSC detector have no interlock.
MovTot = MovTot + ((E < dLim) ? F : E) # Suffers from same unmatched EOS/SOS detects as SCSelect().
MovTot = MovTot + ((C < dLim) ? B : C) # Detects unreliable frame differences rather than scene change.
MovTot = MovTot + ((B < dLim) ? A : B) # We would not find much use for such a detector, other than to
MovAvg = (MovTot / 4.0) # stop reliance upon a frame in some eg MC script.
Result = MovAvg / (D + dMin) # To actually split into scenes, would be of little if any use.
Global EOS@@@ = (Result <= JumpThresh) # End Of Scene detects HERE & passed on to following iteration via Global.
}
n = current_frame # Restore to current frame, where n = current_frame + 1 after for loop exit.
SOS_T=(SOS||n==0) # Take into account if start or end of clip frame.
EOS_T=(EOS@@@||n==Framecount-1)
# Below, will ONLY set Start Of Scene if SOS_T and NOT EOS_T, and End of Scene if EOS_T and NOT SOS_T
# otherwise will show Motion clip frames
# Without some kind of interlock preventing consecutive EOS will not be very functional
# (detector inside for loop can produce many consecutive scene change EOS detections.)
Last = (SOS_T&&!EOS_T)?start:(EOS_T&&!SOS_T)?end:Last # Choose frame to show to user, Start, End, Motion
Global Start@@@ = (SOS_T&&!EOS_T)?n:Start@@@ # Update Start frame Of Current Scene Change
if(Show) {
CC=(EOS_T)?33:45 # Hilite or Default color ? (Subtitle Last frame before Scene Change Status)
RT_Subtitle(Fmt,n,TrimN@@@,n-Start@@@,MovTot,MovAvg,Result,A,B,C,CC,D,E,F,G)
RT_Subtitle("EOS=%s SOS=%s",EOS@@@,SOS,align=3)
}
Global TrimN@@@ = (EOS_T) ? TrimN@@@+1 : TrimN@@@ # Incr for next interation (Scene number, zero rel)
Global Prev@@@=current_Frame # Previous frame (For next interation jump about detect)
Global A@@@=A Global B@@@=B Global C@@@=C Global D@@@=D Global E@@@=E Global F@@@=F Global G@@@=G
Return Last
}
#######################################
# Unique Global Variables Initialization
#######################################
Global A@@@=0.0 Global B@@@=0.0 Global C@@@=0.0 Global D@@@=0.0 Global E@@@=0.0 Global F@@@=0.0 Global G@@@=0.0
Global Start@@@=0
Global EOS@@@=False Global TrimN@@@=0 Global Prev@@@= -666 # Init vars, Prev = -666 forces initalize
#######################################
# Unique Runtime Call, GScriptClip must be a one-liner:
#######################################
ARGS = "start,end,dclip,dLim,dMin,JumpThresh,dChromaW,Show,Fmt"
Motion.GScriptClip("Fn@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
#######################################
# Unique Identifier Definition
#######################################
GIFunc ="JMSC" # Function Name, Supply unique name for your multi-instance function.
GIName =GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
Return GScript(InstS)
}

Avisource("D:\StarWars.avi")

DLIM = 0.02
DMIN = 0.01
JUMPTHRESH = 0.74

DLIM2 = 0.02
DMIN2 = 0.5
JUMPTHRESH2 = 0.5

#return JMSC(Last,Last,Last,Last,dLim=DLIM,dmin=DMIN,JumpThresh=JUMPTHRESH,Show=true) # Single Instance

# Multi-instance with diffent args, compare results
a = JMSC(Last,Last,Last,Last,dLim=DLIM, dmin=DMIN, JumpThresh=JUMPTHRESH, Show=true)
b = JMSC(Last,Last,Last,Last,dLim=DLIM2,dmin=DMIN2,JumpThresh=JUMPTHRESH2,Show=true)
StackHorizontal(a,b)



You can try out different args, changed your 0.3 (later in other thread 0.2) to dLim with default = 0.2 and JumpThresh to 0.74 (which I think I saw somewhere
in other thread)

EDIT: I have imposed no restrictions on dLim or JumpThresh.
Start, End and Motion clips have to be same size colorspace etc, dClip (detection) can be other.
All metrics are shown, not just the result.

EDIT: To change to an 'Unreliable Frame' detector rather than SC, need to (at least) remove EOS->SOS conflicts in these lines

# Below, will ONLY set Start Of Scene if SOS_T and NOT EOS_T, and End of Scene if EOS_T and NOT SOS_T
# otherwise will show Motion clip frames
# Without some kind of interlock preventing consecutive EOS will not be very functional
# (detector inside for loop can produce many consecutive scene change EOS detections.)

Last = (SOS_T&&!EOS_T)?start:(EOS_T&&!SOS_T)?end:Last # Choose frame to show to user, Start, End, Motion
Global Start@@@ = (SOS_T&&!EOS_T)?n:Start@@@ # Update Start frame Of Current Scene Change


EDIT: Added dMin (default 0.01) to JMSC args, to give a little more control, replaces fixed 0.01 addition.

Result = MovAvg / (D + dMin)

[StainlessS]

John,
Added interlock to prevent two consecutive EOS detections.
Changed arg defaults.
Added H difference at n+4->n+5, used only to get ResultN as if at frame n + 1, EOS only detected @ current frame if Result < ResultN.
Added 4 'T' condition flags, T1 to T4.

Here code to detect as if at Next Frame (where uses B -> H (E=pretend curr) rather than A -> G (D=curr) when Result calc'ed for current frame).

MovTotN = ((G < dLim) ? H : G)
MovTotN = MovTotN + ((F < dLim) ? G : F)
MovTotN = MovTotN + ((D < dLim) ? C : D)
MovTotN = MovTotN + ((C < dLim) ? B : C)
MovAvgN = (MovTotN / 4.0)
ResultN = MovAvgN / (E + dMin)


And here T condition flags

T1 = (!SOS) # Interlock, Cannot have two consecutive EOS
T2 = (Result < ResultN) # Here better EOS than if @ next frame ?
T3 = (D > 1.0 && Result > 0.01) # Avoid some ridiculous static scene detections
T4 = (Result <= JumpThresh) # Comply with JumpThresh ?
Global EOS@@@ = (T1 && T2 && T3 && T4) # End Of Scene detects HERE & passed on to following iteration via Global.


Full code

Function JMSC(clip dclip,clip start,clip end,clip motion,float "dLim",Float "dMin",Float "JumpThresh",Float "dChromaW",bool "Show") {
myName="JMSC: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
Assert(RT_FunctionExist("RoboCrop"),myName+"Essential RoboCrop plugin installed, http://forum.doom9.org/showthread.php?t=168053")
dLim=Float(Default(dLim,0.25))
dMin=Float(Default(dMin,0.25))
JumpThresh=Float(Default(JumpThresh,0.5))
dChromaW=Float(Default(dChromaW,1.0/3.0))
Show=Default(Show,False)
Assert(dChromaW>=0.0 && dChromaW<=1.0,myName+"dChromaW 0.0->1.0")
Assert(dMin>0.0,myName+"0.0 < dMin")

dclip=dclip.RoboCrop(WMod=4,HMod=4,Laced=False,align=True)
SIZE = Motion.Height / 8
CENT = Motion.Height / 2
Start =(Show) ? Start.SubTitle("START OF SCENE",align=5,size=SIZE,y=CENT-SIZE,Text_Color=$00FF00) : Start
End =(Show) ? End.SubTitle("END OF SCENE",align=5,size=SIZE,y=CENT+SIZE,Text_Color=$FF00FF) : End
Motion=(Show) ? Motion.SubTitle("MOTION",align=5,size=SIZE) : Motion
# Make format string only once, not at every frame
Fmt=(Show) ? "%d ] #%d:%-4d {\a%c%s\a-:\a%c%s\a-:\a%c%s\a-:\a%c%s\a-}\n" +
\ "\aR%6.2f %6.2f %6.2f \a%c%6.2f\aR %6.2f %6.2f %6.2f %6.2f\n\n" +
\ "MovTot=%7.3f : MovAvg=%7.3f : \a%cResult=%7.3f"
\ : ""
FuncS="""
Function Fn@@@(clip clp,clip start,clip end,clip dclip,Float dLim,Float dMin,Float JumpThresh,Float dChromaW,Bool Show,String Fmt) {
clp
n = current_frame
If(Prev@@@ == n) { # Cache failure, requested same frame again. Rightshift everything, will be Leftshifted later.
Global H@@@=G@@@ Global G@@@=F@@@ Global F@@@=E@@@ Global E@@@=D@@@
Global D@@@=C@@@ Global C@@@=B@@@ Global B@@@=A@@@
} Else If(Prev@@@ + 1 != n) { # User jumped about, dont you just hate users!
# We want status at current_frame-1, however to re-detect there we need fill Globals as if at current_frame-2
# You could Preroll more than 1 for some other reason but need to read Globals @ 1 prior to Preroll count.
PreRoll = 1
n = n - PreRoll - 1 # Pretend we are 2 prior to current_frame, we need to fill globals and re-detect
Global Start@@@ = (Start@@@ > n) ? n : Start@@@
Global EOS@@@=False
# Get ALL Globals as if n is at correct current_frame, ie D = Curr->Next at n
Global A@@@ = RT_FrameDifference(dclip,dclip,n=n-3,n2=n-2,chromaweight=dChromaW) # Not really needed
Global B@@@ = RT_FrameDifference(dclip,dclip,n=n-2,n2=n-1,chromaweight=dChromaW)
Global C@@@ = RT_FrameDifference(dclip,dclip,n=n-1,n2=n+0,chromaweight=dChromaW)
Global D@@@ = RT_FrameDifference(dclip,dclip,n=n+0,n2=n+1,chromaweight=dChromaW) # Curr->Next at pretend n
Global E@@@ = RT_FrameDifference(dclip,dclip,n=n+1,n2=n+2,chromaweight=dChromaW)
Global F@@@ = RT_FrameDifference(dclip,dclip,n=n+2,n2=n+3,chromaweight=dChromaW)
Global G@@@ = RT_FrameDifference(dclip,dclip,n=n+3,n2=n+4,chromaweight=dChromaW)
Global H@@@ = RT_FrameDifference(dclip,dclip,n=n+4,n2=n+5,chromaweight=dChromaW)
n = n + 1 # Now that Globals are filled, continue as if at the following iteration.
}
# We Detect at End Of Scene where D is Curr->Next (D is middle of A -> G)
A=A@@@ B=B@@@ C=C@@@ D=D@@@ E=E@@@ F=F@@@ G=G@@@ H=H@@@ # Get locals, As set by previous iteration
# Below loop can do a 'Rewind/Preroll' re-detect where SC status is dependent upon previous frames eg user jumped about.
# In normal (non-jump-about) use, below loop is one iteration only.
for(n=n,current_frame) {
SOS = EOS@@@ # Start Of Scene if Prev frame was End Of Scene
A=B B=C C=D D=E E=F F=G G=H # Pass the parcel, from previous interation (re-use metrics)
H = RT_FrameDifference(dclip,dclip,n=n+4,n2=n+5,chromaweight=dChromaW) # New single metric
MovTot = ((F < dLim) ? G : F) # Remaining lines of JMSC detector have no interlock.
MovTot = MovTot + ((E < dLim) ? F : E) # Suffers from same unmatched EOS/SOS detects as SCSelect().
MovTot = MovTot + ((C < dLim) ? B : C) # Detects unreliable frame differences rather than scene change.
MovTot = MovTot + ((B < dLim) ? A : B) # We would not find much use for such a detector, other than to
MovAvg = (MovTot / 4.0) # stop reliance upon a frame in some eg MC script.
Result = MovAvg / (D + dMin) # To actually split into scenes, would be of little if any use.
# # Repeat calcs as if at NEXT frame, ie E will then be current
MovTotN = ((G < dLim) ? H : G)
MovTotN = MovTotN + ((F < dLim) ? G : F)
MovTotN = MovTotN + ((D < dLim) ? C : D)
MovTotN = MovTotN + ((C < dLim) ? B : C)
MovAvgN = (MovTotN / 4.0)
ResultN = MovAvgN / (E + dMin)
T1 = (!SOS) # Interlock, Cannot have two consecutive EOS
T2 = (Result < ResultN) # Here better EOS than if @ next frame ?
T3 = (D > 1.0 && Result > 0.01) # Avoid some ridiculous static scene detections
T4 = (Result <= JumpThresh) # Comply with JumpThresh ?
Global EOS@@@ = (T1 && T2 && T3 && T4) # End Of Scene detects HERE & passed on to following iteration via Global.
}
n = current_frame # Restore to current frame, where n = current_frame + 1 after for loop exit.
SOS_T=(SOS||n==0) # Take into account if start or end of clip frame.
EOS_T=(EOS@@@||n==Framecount-1)
# Below, will ONLY set Start Of Scene if SOS_T and NOT EOS_T, and End of Scene if EOS_T and NOT SOS_T
# otherwise will show Motion clip frames
# Without some kind of interlock preventing consecutive EOS will not be very functional
# (detector inside for loop can produce many consecutive scene change EOS detections.)
Last = (SOS_T&&!EOS_T)?start:(EOS_T&&!SOS_T)?end:Last # Choose frame to show to user, Start, End, Motion
if(Show) {
# Update Start@@@ and TrimN@@@ only if needed ie Show==True
Global Start@@@ = (SOS_T&&!EOS_T)?n:Start@@@ # Update Start frame Of Current Scene Change
CC=(EOS_T)?33:45 # Hilite or Default color ? (Subtitle Last frame before Scene Change Status)

RT_Subtitle(Fmt,n,TrimN@@@,n-Start@@@,
\ CC,(T1)?"T":"F",CC,(T2)?"T":"F",CC,(T3)?"T":"F",CC,(T4)?"T":"F",
\ A,B,C,CC,D,E,F,G,H,
\ MovTot,MovAvg,CC,Result
\ )

RT_Subtitle("dLim=%.3f dMin=%.3f JumpThresh=%.3f",dLim,dMin,JumpThresh,align=3)
Global TrimN@@@ = (EOS_T) ? TrimN@@@+1 : TrimN@@@ # Incr for next interation (Scene number, zero rel)
}
Global Prev@@@=current_Frame # Previous frame (For next interation jump about detect)
Global A@@@=A Global B@@@=B Global C@@@=C Global D@@@=D
Global E@@@=E Global F@@@=F Global G@@@=G Global H@@@=H
Return Last
}
#######################################
# Unique Global Variables Initialization
#######################################
Global A@@@=0.0 Global B@@@=0.0 Global C@@@=0.0 Global D@@@=0.0
Global E@@@=0.0 Global F@@@=0.0 Global G@@@=0.0 Global H@@@=0.0
Global Start@@@=0
Global EOS@@@=False Global TrimN@@@=0 Global Prev@@@= -666 # Init vars, Prev = -666 forces initalize
#######################################
# Unique Runtime Call, GScriptClip must be a one-liner:
#######################################
ARGS = "start,end,dclip,dLim,dMin,JumpThresh,dChromaW,Show,Fmt"
Motion.GScriptClip("Fn@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
#######################################
# Unique Identifier Definition
#######################################
GIFunc ="JMSC" # Function Name, Supply unique name for your multi-instance function.
GIName =GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
Return GScript(InstS)
}

Avisource("D:\AVS\StarWars.avi")

DLIM = 0.25
DMIN = 0.25
JUMPTHRESH = 0.5

DLIM2 = 0.25
DMIN2 = 0.5
JUMPTHRESH2 = 0.4

#return JMSC(Last,Last,Last,Last,dLim=DLIM,dmin=DMIN,JumpThresh=JUMPTHRESH,Show=true) # Single Instance

# Multi-instance with diffent args, compare results
a = JMSC(Last,Last,Last,Last,dLim=DLIM, dmin=DMIN, JumpThresh=JUMPTHRESH, Show=true)
b = JMSC(Last,Last,Last,Last,dLim=DLIM2,dmin=DMIN2,JumpThresh=JUMPTHRESH2,Show=true)
StackHorizontal(a,b)




It does work but still way too sensitive.

[johnmeyer]

I'll try it on my source footage. Perhaps I should upload a few minutes of material so you can see the scene problems I am having with your code.

The code I posted had nothing to do with my quest for better scene detection, and I posted it here merely as a way to show you the idea of using a ratio rather than a difference, and to have the numerator of that ratio based on a moving average of previous frames, rather than just the immediately preceding frame.

After I struck out with your RT_Stats scene detection, I went back and tried the five other approaches that I've posted in the past. As before, the simple YDifference approach proved the most reliable, but gave lots of false detections when the camera panned quickly. I solved that by feeding it motion estimated frames. For my work, it beat everything else, although I couldn't get it to ignore someone entering the frame close to the camera. That was always treated as a scene change.

I'll take a look at your latest revisions and see if they are going to work.

Thanks!

[StainlessS]

I've tried to comment the func reasonably well to clarify what it is doing, for the most part all you might be interested in
tweaking is contained inside of the for loop. If some eg Flash frame detection were implemented, it should be done inside for loop
and where user-jump detected, should Preroll enough frames to start detect prior to flash so that any prior frame status is propagated
though subsequent frames up to current frame.

I just did the function so that you could play with it and see all metrics together with the Black/White frame capability if required.

EDIT:
I solved that by feeding it motion estimated frames
Which I guess relies on it's own SC detection to do the motion estimation, maybe try switch OFF
any MC SC detection and see how it works like that, I was not too enthused by MC SC detection.

[johnmeyer]

Which I guess relies on it's own SC detection to do the motion estimation, maybe try switch OFF
any MC SC detection and see how it works like that, I was not too enthused by MC SC detection.Yes, I tried MVTools, RemoveDirt, and Depan's built-in SD, and I didn't get satisfactory results. Also, they suffered from not having any easy way (that I could find) to see the metrics so I could tweak and tune the results.

However, by switching off scene detection in MVTools2 (it wouldn't be useful otherwise), I was able to create my own scene detection logic, while simultaneously taking advantage of the motion estimation's ability to "calm" frame-to-frame differences caused by either camera motion or object motion. This lowered the difference metrics between frames within each scene, thus making the discontinuity at a scene change more prominent.

What I did, basically, was to feed the motion compensated frames to the built-in YDifference logic, computing ratios between current and previous. It got a little more complicated than that (doesn't it always?), but that's the basic idea.

Perhaps I should post my own "work in progress" and see if it holds up in the tests you are doing? I don't want to hijack your thread, however.

[StainlessS]

Sounds interesting John, yes would like to see what you have done.
You can post here (if you like) or in your own thread, I have in past noted that you do add your scripts to other threads,
eg your mod of Fred's script. Think might be best with it's own thread instead of hunting through a different but
related thread. I dont mind either way but perhaps easier for viewers to skip through a dedicated thread rather than
hunting out isolated posts in another.

Hope you dont mind if I nick any good bits you have mastered.

EDIT:
"Time for bed" said Florence. "Boing" said Zebedee.
https://www.google.co.uk/search?q=Zebedee&biw=1280&bih=837&tbm=isch&tbo=u&source=univ&sa=X&ei=UmiTVeX8B6TP7QaG7rvwAg&ved=0CC8Q7Ak&dpr=1

[johnmeyer]

If I post it as a new thread, then I have to make it work better, which I intend to do. I haven't yet posted, because I keep getting called away to do this pesky work stuff. I'll try to get to it shortly.

[StainlessS]

John, suggest use of the supplied multi-instance function as a template, think I'll use it as a template myself for future projects as reasonably
clear what it is doing and should be easy to mod (only caveat is to be aware of using 'Global' in assignments to glob vars).
The multi-instance thing is a brilliant result of work by Martin53 and Gavino, much thanks to both.

Looking forward to seeing your script.

[johnmeyer]

I think I'm going to keep my script private, for the time being. It works better than anything else I've tried, but still doesn't work well enough to make me think I've advanced the state of the art.

[StainlessS]

Here a better example Scene change detector template, with dFactHint to assist in choosing dFact arg.

Scss.avs

###############################
# SCSS.avs
# SCSS by StainlessS. Requires RT_Stats, Grunt, GScript.
###############################

Function SCSS(clip dclip,clip start,clip end,clip motion,float "dFact",float "dMinim",int "dMinLen", Float "ChromaW",bool "Show") {
/*
Selects frames depending upon scene change status. Similar to SCSelect().
Start and End of scene frames detected strictly in pairs, detects on End Of Scene frame.

Multi-Instance capable, Can use multiple instances of this script function despite use of Global Variables.
Many Thanks to Martin53 and Gavino for development of multi-instance runtime scripts.

dclip, detection clip.
Start clip, Frames to use at frame after scene change.
End clip, Frames to use at frame before scene change.
Motion clip, Frames for Not a scene change.
Start, End and Motion must be same size and colorspace, dclip can be other. Should all be same length but no check on this.

dFact, Default 2.0 (greater than 1.0)
dMinim, Default 2.0, (greater than 0.1) dFact and dMinim , Control how much bigger difference between Curr->Next has to be compared to Prev->Curr
and Curr->Next to [Next->Next+1], to detect scene change.
Curr->Next has to be greater than (dFact * 'Prev->Curr' + dMinim), and also greater than (dFact * 'Next->Next+1' + dMinim)
dMinLen, Default 6 (greater than 1). Minimum frame length of current scene that qualifies for scene change detection.
ChromaW, Default 0.333333 ChromaWeight range 0.0->1.0. YUV Only, default gives twice as much weight to luma as combined chroma.
Show, Default False. Show metrics.

Metrics:
Row 2 shows difference metrics between frames, the center value shows metric between Curr->Next frames, value left of center
shows metric between Prev->Curr frames, right of center shows [Next->Next+1] etc.
Three 'T' conditions must be satified for an End Of Scene to be detected (and therefore the following Start Of Scene, matched pairs only).
T1) Current Scene must be at least dMinLen frames long.
T2) The Row 2 'Curr->Next' (center) value must be greater than (dFact * 'Prev->Curr' + dMinim).
T3) The Row 2 'Curr->Next' (center) value must be greater than (dFact * 'Next->Next+1' + dMinim).
The True/False (T/F) flags in Row 1 show result of above conditions, in left to right order T1 -> T3.
dFactHint, also shown on Row 1 is "dFactHint', assists choosing dFact arg (given current dMinim setting).
dFactHint useful to perhaps lower dFact when currently on last frame in scene that fails to detect scene change.
dFactHint shows the highest value of dFact that could pass T2 and T3 conditions for the differences either side of centre in Row 2,
0.0 means no setting of dFact could pass T2 and T3 conditions. If dFactHint shows 0.0 on non-detected End of Scene frame, then
this would suggest that dMinim could be set too high.
9999.99 means any dFact setting above 1.0 would pass T2 and T3 conditions. (differences either side of center are so small that
anything would work). dFactHint does not take T1 condition into account, will be set regardless of whether current Scene is
dMinLen frames long or not.
dFactHint is only meaningful on a true End Of Scene frame, whether detected as such or not.
*/
myName="SCSS: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
dFact=Float(Default(dFact,2.0))
dminim=Float(Default(dMinim,2.0))
dMinLen=Default(dMinLen,6)
ChromaWeight=Float(Default(ChromaW,1.0/3.0))
Show=Default(Show,False) # True = Show metrics (Default False)
Assert(dFact>1.0,myName+"dFact MUST be greater than 1.0")
Assert(dMinim>0.1,myName+"dMinim MUST be greater than 0.1")
Assert(dMinLen>1,myName+"dMinLen MUST be greater than 1")
SIZE = Motion.Height / 8
CENT = Motion.Height / 2
Start =(Show) ? Start.SubTitle("START OF SCENE",align=5,size=SIZE,y=CENT-SIZE,Text_Color=$00FF00) : Start
End =(Show) ? End.SubTitle("END OF SCENE",align=5,size=SIZE,y=CENT+SIZE,Text_Color=$FF00FF) : End
Motion=(Show) ? Motion.SubTitle("MOTION",align=5,size=SIZE) : Motion
# Make format string only once, not at every frame
Fmt=(Show)? "%d ] %d#%-4d {\a%c%.1s\a-:\a%c%.1s\a-:\a%c%.1s\a-} dFactHint=%.2f\n" +
\ "%6.2f \a%c%6.2f %6.2f %6.2f\a- %6.2f"
\ : ""
FuncS="""
Function Fn@@@(clip clp,clip start,clip end,clip dclip,float dFact,float dMinim,int dMinLen,Float dChromaW,bool Show,String Fmt) {
clp
n = current_frame
If(Prev@@@!=n && Prev@@@ + 1 != n) { # User jumped about, dont you just hate users!
# We want status at current_frame-1, however to re-detect there we need fill Globals as if at current_frame-2
# You could Preroll more than 1 for some other reason but need to read Globals @ 1 prior to Preroll count.
PreRoll = 1
n = n - PreRoll - 1 # Pretend we are 2 prior to current_frame, we need to fill globals and re-detect
Global Start@@@ = (Start@@@ > n) ? n : Start@@@
Global EOS@@@=False
# Get ALL Globals as if n is at correct current_frame, ie D = Curr->Next at n
Global A@@@ = RT_FrameDifference(dclip,dclip,n=n-2,n2=n-1,chromaweight=dChromaW) # Not really needed
Global B@@@ = RT_FrameDifference(dclip,dclip,n=n-1,n2=n+0,chromaweight=dChromaW)
Global C@@@ = RT_FrameDifference(dclip,dclip,n=n+0,n2=n+1,chromaweight=dChromaW) # Curr->Next at pretend n
Global D@@@ = RT_FrameDifference(dclip,dclip,n=n+1,n2=n+2,chromaweight=dChromaW)
Global E@@@ = RT_FrameDifference(dclip,dclip,n=n+2,n2=n+3,chromaweight=dChromaW)
n = n + 1 # Now that Globals are filled, continue as if at the following iteration.
}
# We Detect at End Of Scene where D is Curr->Next (D is middle of A -> G)
A=A@@@ B=B@@@ C=C@@@ D=D@@@ E=E@@@ # Get locals, As set by previous iteration
# Below loop can do a 'Rewind/Preroll' re-detect where SC status is dependent upon previous frames eg user jumped about.
# In normal (non-jump-about) use, below loop is one iteration only.
for(n=n,current_frame) {
SOS = EOS@@@ # Start Of Scene if Prev frame was End Of Scene
A=B B=C C=D D=E # Pass the parcel, from previous interation (re-use metrics)
E = RT_FrameDifference(dclip,dclip,n=n+2,n2=n+3,chromaweight=dChromaW) # New single metric
T1 =((n - Start@@@+1) >= dMinLen) # Scene length at least dMinLen frames ?
T2 =(dFact*B+dMinim) < C # sufficiently bigger than 'Prev->Curr' (LHS) ?
T3 =(dFact*D+dMinim) < C # sufficiently bigger than 'Next->Next+1' (RHS) ?
Global EOS@@@=(T1&&T2&&T3) # End Of Scene detects HERE & passed on to following iteration via Global.
}
n = current_frame # Restore to current frame, where n = current_frame + 1 after for loop exit.
SOS_T=(SOS||n==0) # Take into account if start or end of clip frame.
EOS_T=(EOS@@@||n==Framecount-1)
# Below, will ONLY set Start Of Scene if SOS_T and NOT EOS_T, and End of Scene if EOS_T and NOT SOS_T
# otherwise will show Motion clip frames
Last = (SOS_T&&!EOS_T)?start:(EOS_T&&!SOS_T)?end:Last # Choose frame to show to user, Start, End, Motion
if(Show) {
# Update Start@@@ and TrimN@@@ only if needed ie Show==True
Global Start@@@ = (SOS_T&&!EOS_T)?n:Start@@@ # Update Start frame Of Current Scene Change
LFACT=((C-dMinim)<=B)?0.0:(B<0.000001)?9999.99:((C-dMinim)/B)-0.005
RFACT=((C-dMinim)<=D)?0.0:(D<0.000001)?9999.99:((C-dMinim)/D)-0.005
dFactHint=Min(LFACT,RFACT)
CC=(EOS_T)?33:45 # Hilite or Default color ? (Subtitle Last frame before Scene Change Status)
RT_Subtitle(Fmt,n,TrimN@@@,n-Start@@@,CC,T1,CC,T2,CC,T3,dFactHint,A,CC,B,C,D,E)
#RT_SubTitle("dFact=%.2f : dMinim=%.2f : dMinLen=%d", dFact,dMinim,dMinlen,align=3)
Global TrimN@@@ = (EOS_T) ? TrimN@@@+1 : TrimN@@@ # Incr for next interation (Scene number, zero rel)
}
Global Prev@@@=n # Previous frame (For next interation jump about detect)
Global A@@@=A Global B@@@=B Global C@@@=C Global D@@@=D Global E@@@=E
Return Last
}
#######################################
# Unique Global Variables Initialization
#######################################
Global A@@@=0.0 Global B@@@=0.0 Global C@@@=0.0 Global D@@@=0.0 Global E@@@=0.0
Global Start@@@=0
Global EOS@@@=False Global TrimN@@@=0 Global Prev@@@= -666 # Init vars, Prev = -666 forces initalize
#######################################
# Unique Runtime Call, GScriptClip must be a one-liner:
#######################################
ARGS = "start,end,dclip,dFact,dMinim,dMinLen,ChromaWeight,Show,Fmt"
Motion.GScriptClip("Fn@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
#######################################
# Unique Identifier Definition
#######################################
GIFunc ="SCSS" # Function Name, Supply unique name for your multi-instance function.
GIName =GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
Return GScript(InstS)
}



Client

Avisource("D:\AVS\FlashTest.avi")

dFact = 2.0
dMinim = 2.0
dMinLen = 6

dFact2 = 1.5
dMinim2 = 6.0
dMinLen2 = 6

dclip=Last
dclip=dclip.RoboCrop(WMod=4,align=True) # Dont allow common border to 'water down' metrics

return dclip.SCSS(Last,Last,Last,dFact=dFact,dMinim=dMinim,dMinlen=dMinLen,Show=true) # Single Instance

################################

# Multi-instance with diffent args, compare results
a = dclip.SCSS(Last,Last,Last,dFact=dFact,dMinim=dMinim,dMinlen=dMinLen,Show=true)

b = dclip.SCSS(Last,Last,Last,dFact=dFact2,dminim=dMinim2,dMinlen=dMinLen2,Show=true)
StackHorizontal(a,b)


EDIT: Detection clip could be prepared in some way eg ColorYUV(AutoGain=True).

EDIT: Metrics, Shows End Of Scene @frame 512, Scene 2, Frame 27 of Scene, Prev->Curr=3.05, Curr->Next=37.59, Next->Next+1=4.63.

512 ] 2#27 {T:T:T} dFactHint=7.69
3.32 3.05 37.59 4.63 4.44


EDIT: Moved RoboCrop out of script function.

[StainlessS]

John, would like to give this a whirl with your problem clips.


###############################
# SCMC.avs
# SCMC by StainlessS. Requires RT_Stats, Grunt, GScript.
###############################

Function SCMC(clip dclip,clip start,clip end,clip motion,float "dFact",float "dMinim",int "dMinLen", Float "ChromaW",bool "Show") {
/*
Selects frames depending upon scene change status. Similar-ish to SCSelect().
Start and End of scene frames detected strictly in pairs, detects on End Of Scene frame.
Requires pre-prepared MC Interleaved dclip (detection clip) where FrameCount is 3x Motion clip Framecount.

Multi-Instance capable, Can use multiple instances of this script function despite use of Global Variables.
Many Thanks to Martin53 and Gavino for development of multi-instance runtime scripts.

dclip, detection clip. mvTools Interleaved MC clip, 3x number of frames in Motion clip else error.
Can be different Colorspace, size etc to Motion, Start and End clips.
Start clip, Frames to use at frame after scene change.
End clip, Frames to use at frame before scene change.
Motion clip, Frames for Not a scene change.
Start, End and Motion must be same size and colorspace, Should all be same length but no check on this.

dFact, Default 2.0 (greater than 1.0)
dMinim, Default 2.0, (greater than 0.1) dFact and dMinim , Control how much bigger difference between Curr->Next has to be compared to Prev->Curr
and Curr->Next to [Next->Next+1], to detect scene change.
Curr->Next has to be greater than (dFact * 'Prev->Curr' + dMinim), and also greater than (dFact * 'Next->Next+1' + dMinim)
dMinLen, Default 6 (greater than 1). Minimum frame length of current scene that qualifies for scene change detection.
ChromaW, Default 0.333333 ChromaWeight range 0.0->1.0. YUV Only, default gives twice as much weight to luma as combined chroma.
Show, Default False. Show metrics.

Metrics:
Row 3 shows difference metrics between interleaved dclip frames, and Row 2 shows "FC->CURR : CURR->BC : FC->NXT : NXT->BC".
Where row 3 metrics are:-
FC->Curr is the difference between Forward Compensated previous frame to Current frame.
Curr->BC is the difference between Current Frame and Backward Compensated Next frame.
FC->Nxt is the difference between Forward Compensated Current frame and Next frame.
Nxt->BC is the difference between Next Frame and Backward Compensated 'Next+1' frame.

Three 'T' conditions must be satified for an End Of Scene to be detected (and therefore the following Start Of Scene, matched pairs only).
T1) Current Scene must be at least dMinLen frames long.
T2) 'CURR->BC' diff must be greater than (dFact * 'FC->CURR' + dMinim).
T3) 'FC->NXT' diff must be greater than (dFact * 'NXT->BC' + dMinim).
The True/False (T/F) flags in Row 1 show result of above conditions, in left to right order T1 -> T3.
dFactHint, also shown on Row 1 is "dFactHint', assists choosing dFact arg (given current dMinim setting).
dFactHint useful to perhaps lower dFact when currently on last frame in scene that fails to detect scene change.
dFactHint shows the highest value of dFact that could pass both T2 and T3 conditions,
0.0 means no setting of dFact could pass T2 and T3 conditions. If dFactHint shows 0.0 on non-detected End of Scene frame, then
this would suggest that dMinim could be set too high.
9999.99 means any dFact setting above 1.0 would pass T2 and T3 conditions. (FC->CURR and NXT->BC are so small that
anything would work). dFactHint does not take T1 condition into account, will be set regardless of whether current Scene is
dMinLen frames long or not.
dFactHint is only meaningful on a true End Of Scene frame, whether detected as such or not.
*/
myName="SCMC: "
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
dFact=Float(Default(dFact,2.0))
dminim=Float(Default(dMinim,2.0))
dMinLen=Default(dMinLen,6)
ChromaWeight=Float(Default(ChromaW,1.0/3.0))
Show=Default(Show,False) # True = Show metrics (Default False)
Assert(dFact>1.0,myName+"dFact MUST be greater than 1.0")
Assert(dMinim>0.1,myName+"dMinim MUST be greater than 0.1")
Assert(dMinLen>1,myName+"dMinLen MUST be greater than 1")
Assert(dclip.FrameCount==3*Motion.FrameCount,myName+"dClip FrameCount Must be 3x Motion Framecount")

SIZE = Motion.Height / 8
CENT = Motion.Height / 2
Start =(Show) ? Start.SubTitle("START OF SCENE",align=5,size=SIZE,y=CENT-SIZE,Text_Color=$00FF00) : Start
End =(Show) ? End.SubTitle("END OF SCENE",align=5,size=SIZE,y=CENT+SIZE,Text_Color=$FF00FF) : End
Motion=(Show) ? Motion.SubTitle("MOTION",align=5,size=SIZE) : Motion
# Make format string only once, not at every frame
Fmt=(Show)? "%d ] %d#%-4d {\a%c%.1s\a-:\a%c%.1s\a-:\a%c%.1s\a-} dFactHint=%.2f\n" +
\ "\aRFC->CURR : CURR->BC : FC->NXT : NXT->BC\a-\n" +
\ "%6.2f : \a%c%6.2f\a- : \a%c%6.2f\a- : %6.2f"
\ : ""
FuncS="""
Function Fn@@@(clip clp,clip start,clip end,clip dclip,float dFact,float dMinim,int dMinLen,Float dChromaW,bool Show,String Fmt) {
clp
n = current_frame
If(Prev@@@!=n && Prev@@@ + 1 != n) { # User jumped about, dont you just hate users!
# We want status at current_frame-1, however to re-detect there we need fill Globals as if at current_frame-2
# You could Preroll more than 1 for some other reason but need to read Globals @ 1 prior to Preroll count.
PreRoll = 1
n = n - PreRoll - 1 # Pretend we are 2 prior to current_frame, we need to fill globals and re-detect
Global Start@@@ = (Start@@@ > n) ? n : Start@@@
Global EOS@@@=False
x = n * 3
# Get ALL Globals as if n is at correct current_frame, ie D = Curr->Next at n
# Global A@@@ = RT_FrameDifference(dclip,dclip,n=x ,n2=x+1,chromaweight=dChromaW) # NOT NEEDED
# Global B@@@ = RT_FrameDifference(dclip,dclip,n=x+1,n2=x+2,chromaweight=dChromaW) # NOT NEEDED
Global C@@@ = RT_FrameDifference(dclip,dclip,n=x+3,n2=x+4,chromaweight=dChromaW)
Global D@@@ = RT_FrameDifference(dclip,dclip,n=x+4,n2=x+5,chromaweight=dChromaW)
n = n + 1 # Now that Globals are filled, continue as if at the following iteration.
}
# A=A@@@ B=B@@@ # NOT NEEDED
C=C@@@ D=D@@@ # Get locals, As set by previous iteration
# Below loop can do a 'Rewind/Preroll' re-detect where SC status is dependent upon previous frames eg user jumped about.
# In normal (non-jump-about) use, below loop is one iteration only.
for(n=n,current_frame) {
x = n * 3 # MC Offset
SOS = EOS@@@ # Start Of Scene if Prev frame was End Of Scene
A=C B=D # Pass the parcel, from previous interation (re-use metrics)
C = RT_FrameDifference(dclip,dclip,n=x+3,n2=x+4,chromaweight=dChromaW) # New metric
D = RT_FrameDifference(dclip,dclip,n=x+4,n2=x+5,chromaweight=dChromaW) # New metric
T1 =((n - Start@@@+1) >= dMinLen) # Scene length at least dMinLen frames ?
T2 =(dFact*A+dMinim) < B # CURR->BC sufficiently bigger than 'FC->CURR' (LHS) ?
T3 =(dFact*D+dMinim) < C # FC->NXT sufficiently bigger than 'NXT->BC' (RHS) ?
Global EOS@@@=(T1&&T2&&T3) # End Of Scene detects HERE & passed on to following iteration via Global.
}
n = current_frame # Restore to current frame, where n = current_frame + 1 after for loop exit.
SOS_T=(SOS||n==0) # Take into account if start or end of clip frame.
EOS_T=(EOS@@@||n==Framecount-1)
# Below, will ONLY set Start Of Scene if SOS_T and NOT EOS_T, and End of Scene if EOS_T and NOT SOS_T
# otherwise will show Motion clip frames
Last = (SOS_T&&!EOS_T)?start:(EOS_T&&!SOS_T)?end:Last # Choose frame to show to user, Start, End, Motion
if(Show) {
# Update Start@@@ and TrimN@@@ only if needed ie Show==True
Global Start@@@ = (SOS_T&&!EOS_T)?n:Start@@@ # Update Start frame Of Current Scene Change
LFACT=((B-dMinim)<=A)?0.0:(A<0.000001)?9999.99:((B-dMinim)/A)-0.005
RFACT=((C-dMinim)<=D)?0.0:(D<0.000001)?9999.99:((C-dMinim)/D)-0.005
dFactHint=Min(LFACT,RFACT)
CC=(EOS_T)?33:45 # Hilite or Default color ? (Subtitle Last frame before Scene Change Status)
RT_Subtitle(Fmt,n,TrimN@@@,n-Start@@@,CC,T1,CC,T2,CC,T3,dFactHint,A,CC,B,CC,C,D)
RT_SubTitle("dFact=%.2f : dMinim=%.2f : dMinLen=%d", dFact,dMinim,dMinlen,align=3)

Global TrimN@@@ = (EOS_T) ? TrimN@@@+1 : TrimN@@@ # Incr for next interation (Scene number, zero rel)
}
Global Prev@@@=n # Previous frame (For next interation jump about detect)
#Global A@@@=A Global B@@@=B # NOT NEEDED
Global C@@@=C Global D@@@=D
Return Last
}
#######################################
# Unique Global Variables Initialization
#######################################
# Global A@@@=0.0 Global B@@@=0.0 # NOT NEEDED
Global C@@@=0.0 Global D@@@=0.0
Global Start@@@=0
Global EOS@@@=False Global TrimN@@@=0 Global Prev@@@= -666 # Init vars, Prev = -666 forces initalize
#######################################
# Unique Runtime Call, GScriptClip must be a one-liner:
#######################################
ARGS = "start,end,dclip,dFact,dMinim,dMinLen,ChromaWeight,Show,Fmt"
Motion.GScriptClip("Fn@@@(last, "+ARGS+")", local=true, args=ARGS)
"""
#######################################
# Unique Identifier Definition
#######################################
GIFunc ="SCMC" # Function Name, Supply unique name for your multi-instance function.
GIName =GIFunc+"_InstanceNumber" # Name of the Instance number Global
RT_IncrGlobal(GIName) # Increment Instance Global (init to 1 if not already exists)
GID = GIFunc + "_" + String(Eval(GIName))
InstS = RT_StrReplace(FuncS,"@@@","_"+GID)
# RT_WriteFile("DEBUG_"+GID+".TXT","%s",InstS)
Return GScript(InstS)
}


client

src=Avisource("D:\AVS\Flashtest.avi")

dFact = 2.0
dMinim = 2.0
dMinLen = 6

dFact2 = 1.5
dMinim2 = 6.0
dMinLen2 = 6
#
BLOCKSIZE = 8
thSCD1 = (BLOCKSIZE*BLOCKSIZE) * 64
################################
# Create pre-processed detection dClip
dclip=src.RoboCrop(WMod=4,Align=True) # Dont allow common border to 'water down' metrics or affect eg ColorYUV(AutoGain=true)
dclip=dclip.ColorYUV(AutoGain=True)
# SCMC requires Motion Compensated Interleaved clip (3 times the number of frames in source clip)
# Might want to use Prefiltered (eg Blur) Super clip (although slower).
prefiltered = dclip.blur(1.0)
superfilt = prefiltered.MSuper() # all levels for MAnalyse
super = dclip.MSuper(levels=1) # one level is enough for MCompensate
bvec = superfilt.MAnalyse(isb = true, blksize=BLOCKSIZE,search=0)
fvec = superfilt.MAnalyse(isb = false,blksize=BLOCKSIZE,search=0)
# use not-prefiltered (super) clip for motion compensation (Use MCompensate or MFlow)
#fcmp = dclip.MFlow(super, fvec, thSCD1=thSCD1, thSCD2=255)
#bcmp = dclip.MFlow(super, bvec, thSCD1=thSCD1, thSCD2=255)
fcmp = dclip.MCompensate(super, fvec, thSCD1=thSCD1, thSCD2=255)
bcmp = dclip.MCompensate(super, bvec, thSCD1=thSCD1, thSCD2=255)
dclip = interleave(fcmp, dclip, bcmp) # Make FrameCount * 3
#return dclip
################################

#return dclip.SCMC(src,src,src,dFact=dFact,dMinim=dMinim,dMinlen=dMinLen,Show=true) # Single Instance

################################

# Multi-instance with diffent args, compare results
a = dclip.SCMC(src,src,src,dFact=dFact,dMinim=dMinim,dMinlen=dMinLen,Show=true)

b = dclip.SCMC(src,src,src,dFact=dFact2,dminim=dMinim2,dMinlen=dMinLen2,Show=true)
StackHorizontal(a,b)


EDIT: I forgot to update docs for dFact, dMinim, but you get the idea. I'll do it later.
(used eg Curr->Next which we dont use for MC ver$).

EDIT: Could also use Depan stuff instead of MvT, made external so user can choose and do additional AutoGain / RemoveDirt or whatever.
EDIT: If using Depan, might want to crop about 16 pixels all way around dclip to reduce any black Depan'ed or Mirror'ed borders.

[johnmeyer]

Wow, that's a major piece of work. Thanks!

I've tried it with the current project that has proved to be so difficult, and your script seems to fail at the same place as mine. It is a really odd place to fail because it happens at a scene change where, to the eye, everything has changed (i.e., it is "obvious"). I have sent you a PM with a link to the entire 12 minute movie (I've compressed it, but it still works just the same for scene detection). I've provided two frame numbers, one that points to the scene I just described, and the other which points to an area of rapid movement that often causes spurious scene detection. With the settings you provided above, your script does not detect any spurious scenes in that rapid movement section, but it does fail at the scene change shown here:

Frame Prior to Scene Change
http://i177.photobucket.com/albums/w208/johnmeyer/Before%20Scene_zpst4ghqn89.jpg

Frame After Scene Change
http://i177.photobucket.com/albums/w208/johnmeyer/After%20Scene_zpsmdp8cgv9.jpg

The reason I have this movie is that a private collector bought the film, and then found that the plays late in the game were woefully underexposed and he asked me to brighten them. These two frames show the result of that exposure improvement (you couldn't see anything prior to the gamma tweak). Because it is private film, I don't want to post the entire clip publicly.

I have tried applying Autolevels with autogain=true, thinking that creating a high contrast clip might make the scene detection vectors get happy, but that didn't help.

I'll keep working with it ...

[johnmeyer]

I didn't realize that your description of the variables was in the SCMC script (I was looking in the calling script). Your description is extraordinarily clear, and the "hint" variable makes things very easy.

I'm working on tuning things ...

[edit]I'm finding the same things as all my other scene detection script: if I tune to detect problematic scene changes, the script then starts creating spurious scene detection points during rapid camera pans (frame 271 in the complete film I sent to you, is a good example).

[edit 2]I'm going to try using your motion compensation approach (it is a little different than what I was using) and try feeding it to my YDifference ratios which I still think is going to produce more reliable results.

[StainlessS]

John, That 1st frame will detect if you reduce dMinLen, as flagged by 'F' in T1 condition.
Shows that frame as frame 4 (0 rel) and so is only 5 frames, 1 less than dMinLen default 6.
Should NOT be a problem if you change it.

EDIT: I still think your ratios are little more than reciprocal of what I'm using, only mine has control where static scenes (dMinim)
and your does not (You will get machine gun EOS detections during static scenes).

EDIT: dFactHint show that with default dMinim of 2.0, dFact could be as high as 3.26 (default 2.0) and still be detected
(if dMinLen reduced as per T1 condition flag).

EDIT: Note for image 2, T1 condition is T(True), ie sequence is dMinLen frames long at that frame, BUT, is not an EOS frame
and so of course does not detect there, both T2 and T3 conditions fail.

[johnmeyer]

John, That 1st frame will detect if you reduce dMinLen, as flagged by 'F' in T1 condition.
Shows that frame as frame 4 (0 rel) and so is only 5 frames, 1 less than dMinim default 6.Actually, the counters in those frame grabs above are misleading. The reason is that I got to those frames by moving the cursor backwards and forwards. Your frame count within the scene does not reset when going backwards across a frame boundary, unless you do it one frame at a time (which I did not). The counter therefore is meaningless on those two frame grabs. The correct frame count within the scene at that point is 464.

[edit]There is no need to refine this behavior (which would be difficult, if not impossible). I think it is really great that you even have the scene frame counter. The user (me) just needs to understand that the only way it will get reset is if you don't skip over a scene change by moving around by using random access commands.

[StainlessS]

Yep, the problem is that some frames are cached and also, I may not have the 'Pass the Parcel' logic completely correct.
Every now and then it seems as if some frames are cached and then some not, and then some are, and the numbers then get out of whack,
not sure if my fault or not.
I am still twiddling with the logic.

Is only totally reliable if only seeking forwards. From new start, jump to about a dozen frames before target EOS and edge forward.

[StainlessS]

How would you find a similar function that auto scanned all frames (not previously visited) up to current jump to frame, so would in some instances
seem to go to sleep for a while (although somewhat faster scan than doing it with preview, as frames not displayed [saving quite a bit of time]).
Have already done such a script (somewhere), using RT_Stats DBase to store eg Scene Start. Should then be spot on for all metrics.

EDIT: Seeking to previously visited frame would be no problem.

[StainlessS]

@JohnMeyer,
I think I'll do a multi-func plugin with equivalent of SCSS, SCMC, and an SCSelect like func (unreliable frame rather than end of scene, start of scene).
The SCSelect version will be much like SCSelect but with additional dMinim type arg, and will work in all colorspaces and either luma only or Luma with weighted
chroma for YUV, and for RGB, (RGB->LumaY) or RGB average pixel difference [(sum(Rd)+sum(Gd)+sum(Bd)) / (Width*Height*3)].
For non-luma only modes (YUV or RGB) will auto detect if both frames are Grayscale and auto switch to Luma only (Chromaweight==0.0, so can cope with
movies like "A Matter Of Life And Death", B/W and Color).

Should be somewhat easier to do in C than script and should upload Beta maybe tomorrow for you to have a play with.
I'll try to implement much as the SCSS and SCMC with optional write file of scene changes, and dumb mode where will just read in file of scene changes
(perhaps user edited to override some) and choose frames based on file data.
Also something similar for the SCSelect like function (maybe SCSU [Unreliable frame], or similar).

Is there any requirement for xywh:x2y2 type coords (cant think where this would be useful), or AltScan (alternate lines scanned).
Also, do you think any need to support Interlaced YV12 chroma (where coords support and AltScan) ?, have this already implemented in next RT_Stats
RT_FrameDifference() and just a matter of enabling it.

EDIT: Scratch above :x2y2, of course that is only sensible where two different clips differenced, same x and y coords
as we only have a single clip.

[johnmeyer]

StainlessS,

I had a long response prepared, and my browser ate my homework (I lost it), so here is the short version:

I appreciate all your work on this fundamentally necessary code (the need for "perfect" scene detection is everywhere), but for me, the remaining issues feel like something that could be handled with tweaks. The failures I identified in the past were these:

1. Fast camera motion.
2. Low contrast video (causes small difference vectors which makes it harder to spot scene changes)
3. Underexposed film (same problem as #2)
4. Exposure variations both before and after the scene change (e.g., the "flash" frame from movie cameras)
5. Large object entering scene close to the camera.

#5 is the one which is missed by all techniques. My thought on that is that we need to have a secondary check on a scene change, but one that is only invoked once the main logic detects a change. Once a scene change is tentatively detected, the logic should then re-try the scene detection logic using a cropped version of the frame where "n" pixels are cropped from all edges. "n" should be large enough so that the object will not yet appear in the cropped frame. There then needs to be a second variable which is the allowable difference between the cropped and un-cropped scene change vectors.

Using this additional check, the logic would then be as follows: if a scene is detected, re-do the scene change vectors on the cropped version of the frame. If the difference between this scene change metric and the full-frame metric is almost the same (i.e., they BOTH show a scene change), then it was indeed a scene change. However, if the cropped version has a much lower metric, where "much" is determined by some secondary detection variable, then the scene change is ignored.

This obviously cannot be an AND condition, because the object is eventually going to enter the cropped version of the frame and cause the same spike in scene detection metric.

[StainlessS]

I'm only doing a set of 'simple' detectors here, just wanted something that is more reliable than script versions
and also for showing similar metrics as per the DBSC stuff (DBSC only really shows metrics after work is already done).
Despite my reluctance to call SCSelect a Scene Change Detector, it is a quite important thing to have a decent
'Unreliable Frame Detector' and that is reason for doing similar to that, but with dMinim control, dMinLen probably
either missing or allowed to be 1 and probably make LHS and RHS OR conditions rather than AND conditions to
detect unreliable frame. (EDIT: and also to use chroma in detection)
Perhaps the MC (3 frame group) version is best hope. Spent some time yesterday reading about Stab and
Depan, dont like the bouncing around thing that Depan does. Was thinking maybe do Auto crop and then some depan (mvtools
global depan) then after chopping off borders doing non global compensation, also maybe some kind of deflicker and
autogain, and then trying the MC plugin.
I was even thinking after Global depan, maybe possible (using RT_YinRangeLocate) to locate intersection of adjacent Global depans
(before black border crop) and difference that, dont think it would work with the Depan bouncy results, dont know if work with MvT version.
Anyway, started the first plug about an hour ago (been moving source about to enable easier extraction from RT), should
have at least one of the plugs partially working tomorrow.

EDIT:
5) might be solved with count of blocks (with some reasonable stdev) that remained same.

EDIT:
However, if the cropped version has a much lower metric, where "much" is determined by some secondary detection variable, then the scene change is ignored.

"secondary detection".
Correlation of say 0.55 (or perhaps even less) is not good for detecting match (quite bad really), BUT where average pixel differences
look to be a scene change and Corr >= 0.55 then is a good bet that pixel difference is in error, quite good as a false +ve override.
(The opposite can also be true).
This is just an observation in general, not specifically for any cropped version.

[videoFred]

I have sent you a PM with a link to the entire 12 minute movie

Would you mind sending me the link too? I have a scene change script myself and I would like to see how it works on this particular footage. Do not worry, I will keep this footage private.

My scene change script is based on a script made by Frustum, the man who improved Autolevels, remember? It's using MSCDetection() from MVTools. Scene detection can be tuned with threshold. And there is an option to cut out X number of frames at the end of a scene and at the beginning of a new scene.

I have modified this script: the output is not a txt logfile any more, but a working avisynth script with trim commands at every scene change. An occasional false scene detection can be easily changed in that script.

The auto generated script looks like this:
### scene cut list from file C:\01_Gerenderd_Xvid\VdP\225_VdP_Sp1.avi ###

#frames removed at the begin of a scene: 2
# frames removed at the end of a scene: 2

AviSource("C:\01_Gerenderd_Xvid\VdP\225_VdP_Sp1.avi")



trim(3,39) +\
trim(44,117) +\
trim(122,256) +\
trim(261,509) +\
trim(514,514) +\
trim(519,572) +\
trim(577,655) +\
trim(660,708)



As you can see, 514,514 is a false detection but this is easy to change. Creating a script instead of a log file saves a lot of time anyhow.

Fred.

[johnmeyer]

I have to re-upload the clip, but I'll send you a PM as soon as it is back up on Dropbox. Please understand that the reason I have this film is that the collector had already had it transferred, and the original film was extremely dark because the football game was filmed in the late fall, and it got dark before the game ended. The original transfer had not been exposure-corrected, so he asked me to "restore" it. I am providing you with the results of that exposure correction, but many of the scenes are still quite obviously under-exposed. I tell you this simply to let you know that this is not a normal film transfer.

However, because of that under-exposure, and because of the camera movement and people standing up in front of the camera, it provides many challenges for scene detection.

BTW, you might want to look at the following thread, starting here:

Converting script from mvtools to mvtools2 (http://forum.doom9.org/showthread.php?p=1729076#post1729076)

StainlessS found problems in the RemoveDirtMC script that I have now found go all the way back to the original German author back in 2006. In addition, there may be other problems that were introduced when another individual updated that work from MVtools to MVTools2. I have gone back to your two "Restoring 8mm film ..." threads and corrected the scripts I posted. So far, I have only corrected the reversed order of the Interleave statement, but I am now testing to see if the other problems that StainlessS found need to also be corrected.

[videoFred]

I have to re-upload the clip, but I'll send you a PM as soon as it is back up on Dropbox.

Thank you :)

StainlessS found problems in the RemoveDirtMC script

Thank you again, I will have a look at this.

Fred.

[StainlessS]

@Fred,
Creating a script instead of a log file saves a lot of time

If you output ranges or single frames then you can use either Prune(with audio fadeout/in of eg 1ms to remove clicks) or
FrameSel(without audio), to quickly and easily trim and join. They can also both give indication (Prune better) where joins are
by Show=True. Also, in FrameSel, can set reject=true to show cut out frames, or search SelectRanges() / RejectRanges()
which allows use of Prune() with synthesized Reject arg (I think avs exists in both Framesel and Prune zip [RejectRanges]
but FrameSel does the range command file re-write for Prune).

[videoFred]

@Fred,


If you output ranges or single frames then you can use either Prune(with audio fadeout/in of eg 1ms to remove clicks) or
FrameSel(without audio), to quickly and easily trim and join.

Thank you, I will have a closer look at this. And all your other stuff... ;)
Most of it is way beyond my knowledge but I am prepared to learn.

Fred.

[johnmeyer]

Fred,

StainlessS has been helping me with my own approach to scene detection over in this thread:

How to incorporate "Selectevery() into WriteFileIf (http://forum.doom9.org/showthread.php?t=172323)

With his help, I was able to create a really fast scene detection script (over 200 fps on SD film transfers) that is reasonably easy to tune, and which is also straightforward to understand, so you can adapt it to your needs. I have also created a version which uses YDifference ratios for the scene detection. This approach does a much better job adapting to low contrast material (common in film transfers, especially with fogged film). I haven't posted this version yet, but here is the version that uses the scaled YDifference approach that StainlessS suggested. It works great on normally-exposed film, but fails on low-contrast material.

I have not yet experimented with block size, overlap, and other MVTools2 variables, but I expect that I could further improve the scene detection quality if I spent a little more time tweaking these settings.

Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

filename = "e:\Scenes.txt"
global dFact = 2.0
global dMin = 2.0
BLOCKSIZE = 8
thSCD1 = (BLOCKSIZE*BLOCKSIZE) * 64

setMTMode(5,6)
source = AVISource("e:\fs.avi").convertTOYV12().killaudio()
setMTMode(2)

prefiltered = source.blur(1.0)
superfilt = MSuper(prefiltered,pel=2)
super = MSuper(source, pel=2)

back_vec = MAnalyse(superfilt,isb=true, blksize=BLOCKSIZE,overlap=2,search=0)
forw_vec = MAnalyse(superfilt,isb=false, blksize=BLOCKSIZE,overlap=2,search=0)

backcmp = source.MCompensate(super, back_vec, thSCD1=thSCD1, thSCD2=255)
forwcmp = source.MCompensate(super, forw_vec, thSCD1=thSCD1, thSCD2=255)

global mcomp_clip = interleave(forwcmp, source, backcmp)

#---------------------------------

#Next four lines let you view the metrics in order to set detection metrics.
#script = """Subtitle("\nScene = " + String( (dFact * YDifferenceToNext (mcomp_clip) + dMin ) ) + \
# "\nYDiffP = " + String( YDifferenceFromPrevious(mcomp_clip) ), lsp=0)"""
#final = ScriptClip(mcomp_clip,script)
#return selectevery(final,3,1)

#Evaluate every 3rd frame using Mod(3) = 1
#Comment out thse lines when viewing metrics with code above
WriteFileIf(mcomp_clip, filename, "(YDifferenceFromPrevious(mcomp_clip) > \
(dFact * YDifferenceToNext(mcomp_clip) + dMin )) && (current_frame %3 == 1)", \
"current_frame/3", append = false)

[StainlessS]

John, ripped out of DBSC


Function JMLevels(clip c,Float "Strength",Float "Gamma",Float "MinRngPerc",int "MinLen",Bool "Show") {
/*
JMLevels(), Auto Contrast Adjustment.
*/
c
myName="JMLevels: "
Strength=Max(Min(Float(Default(Strength,0.5)),1.0),0.0)
Gamma=Float(Default(Gamma,1.0))
MinRngPerc=Max(Min(Float(Default(MinRngPerc,33.3)),100.0),0.0) # If luma range less than MinRngPerc, then ignore
MinLen=Default(MinLen,25) # If less than MinLen frames, then ignore
Show=Default(Show,False)
GScript("""
if(FrameCount>=MinLen && STRENGTH>0.0) { # Auto Contrast Adjustment
Eval(RT_QueryLumaMinMax(X=8,Y=8,W=-8,H=-8)) # Ignore possible remaining borders (after RoboCrop)
if(IsRGB()) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
QLRng=QLMMMax-QLMMMin+1
CSRng=CSMax-CSMin+1
CSRngMin=CSRng*MinRngPerc/100.0
if(QLRng >= CSRngMin) { # Ignore if < MinRngPerc of CS range
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
(Show)?RT_DebugF("Levels(%3d,%.2f,%3d,%d,%d,Coring=False)",ALMin,Gamma,ALMax,CSMin,CSMax,name=myName):NOP
Levels(ALMin,Gamma,ALMax,CSMin,CSMax,Coring=False)
(Show) ?RT_Subtitle("%sLevels(%3d,%.2f,%3d,%d,%d,Coring=False)",myName,ALMin,Gamma,ALMax,CSMin,CSMax) :NOP
}
} Else if(Show){
RT_DebugF("Skip %3d,%3d Range=%d %.2f%%",QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng,name=myName)
RT_Subtitle("%sSkip %3d,%3d Range=%d %.2f%%",myName,QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng)
}
}
""")
Return Last
}

Avisource("D:\avs\StarWars.avi")

JMLevels


EDIT: Add Levels=1 in your 2nd MSuper() [should be quicker].

EDIT: I see that mcomp_clip has crept back inside your WriteFile script, it is defaulted to Last in there, remove them,
also not not need to be Global.

[johnmeyer]

Wow, my initials in a StainlessS function. I am honored!

So I have a new toy to play with. I actually have quite a few functions that could be improved by a really good auto contrast function, so I am indeed looking forward to using this. In the past, I have used ColorYUV("autogain = true") but found that the flicker it introduced made scene detection less reliable. I will be very interested to see how it works on this low contrast film transfer (which is different from the clip I sent to you).

I'll also have to go back and try one more time to remove the reference to my mcomp_clip variable inside of the WriteFileIf statement. I've tried this several times, and also tried to remove the global typing -- if for no other reason than to help Gavino sleep better :) -- but every time I do this, AVISynth throws an error. However, I don't know if ever did remove it entirely, relying instead on an implicit "last" variable. I'll try it again.

[StainlessS]

Yeh well I'm often stuck for names, most things start out being called 'Fred' (a favorite with programmers as all four keys are adjacent on Qwerty Keyboard).

Here full doc on RT_QueryLumaMinMax(), but note current 'Samples' is 32, will be 40 (as in this doc, also 'Ignore' is 0.2 will be 0.4) in next version.
Might want to add Samples or eg ScanPerCent arg or whatever and convert to Frames Samples using FrameCount.


RT_QueryLumaMinMax(clip c,int "Samples"=40,float "Ignore"=0.4,String "Prefix"="QLMM",bool "DEBUG"=false,int "X"=0,int "Y"=0,int "W"=0,int "H"=0, \
int "Matrix"=(Width>1100||Height>600?3:2),int "Start"=Undefined,int "End"=Undefined)
Prescan function to get Luma Min,Max for clip scanning Samples frames area x,y,w,h.
Returns luma min/max as string Default eg "QLMMMin=25 QLMMMax=244", use eg "Eval(RT_QueryLumaMinMax())" to set variables for use in script.

Samples:=40=frames to sample from clip, limited to Framecount.
Samples = 0 converted to FrameCount and Auto Credits skipping disabled.

Ignore:=0.4=Percentage of extreme pixels to ignore (noise) when getting luma min/max (As Threshold arg for YPlaneMin/Max).

Prefix:="QLMM"=Prefix for return string values.

DEBUG:=false= dont show. True=debug info. Need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027

X=Y=W=H=0=Full frame, as crop, area to examine.

Matrix:, For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
Default for RGB is 3(PC709) if Width > 1100 || Height > 600 Else 2(PC601) : YUV not used
For RGB, it probably does not make sense to use anything other than PC levels.

Start: Default Undefined. Start frame of scan area. The Start arg overrides Auto Intro credits skipping.

End: Default Undefined. End frame for scan area. Overrides Auto End credits skipping. 0 (or less) will be converted to Framecount - 1.


If user supplied Start and End frame numbers given then that marks the range of frames from which to select the Samples frames.
If neither Start nor End given, the function tries to avoid sampling artificial black/white in Intro and End Credits sequences,
for Auto Intro and End Credits Skipping to be set to 5% (of FrameCount for Intro Skipping) and 90% (for End Skipping), the number of
frames between them must be greater or equal to 250 frames, and MUST also be greater than Samples, otherwise Auto skipping ignored and
the Start and End frame numbers are set to 0 and Framecount - 1.
If a user supplies eg a Start frame number ONLY, then End Skipping has to comply with the same above conditions, range between End Skip
frame and user supplied Start has to be at least 250 frames and greater than Samples, otherwise End frame set to FrameCount - 1. The
same conditions apply if only a user supplied End frame only.
After either user supplied Start/End, or via Auto Credits skipping, or defaulted to 0 & FrameCount -1, we have a sample scan range.
Setting Debug=True, will send debug info including resulting sample frame range to DebugView window.

Usage: Clip global Auto levels using Levels() filter (auto contrast).
AUTOLEVEL_STRENGTH = 0.5 # 0.0 -> 1.0
Eval(RT_QueryLumaMinMax()) # using default values, sets QLMMMin and QLMMMax.
if(IsRGB()) { # Requires GScript
CSMin = 0
CSMax = 255
} else {
CSMin = 16
CSMax = 235
}
ALMin = Int(CSMin - ((CSMin - QLMMMin) * AUTOLEVEL_STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * AUTOLEVEL_STRENGTH)) # Round down
Levels(ALMin,1.0,ALMax,CSMin,CSMax,Coring=False) # DO NOT use Coring


Perhaps set Ignore high eg 2.0%

[johnmeyer]

I see that mcomp_clip has crept back inside your WriteFile script, it is defaulted to Last in there, remove them, also not not need to be Global.1. I just tried removing "global" and immediately got an on-screen error "I don't know what 'mcomp_clip" means." So, that doesn't work.

2. I then tried removing mcomp_clip from the ScriptClip call and immediately got an "invalid arguments to function ScriptClip".

So, while I'd like to follow your suggestions, they don't seem to work.

[StainlessS]

Easy Peasy

#Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

filename = "e:\Scenes.txt"


global dFact = 2.0
global dMin = 2.0
BLOCKSIZE = 8
thSCD1 = (BLOCKSIZE*BLOCKSIZE) * 64

#setMTMode(5,6)
#source = AVISource("e:\fs.avi").convertTOYV12().killaudio()
source = AVISource("D:\AVS\StarWars.avi").convertTOYV12().killaudio()
#setMTMode(2)

prefiltered = source.blur(1.0)
superfilt = MSuper(prefiltered,pel=2)
super = MSuper(source, pel=2,Levels=1)

back_vec = MAnalyse(superfilt,isb=true, blksize=BLOCKSIZE,overlap=2,search=0)
forw_vec = MAnalyse(superfilt,isb=false, blksize=BLOCKSIZE,overlap=2,search=0)

backcmp = source.MCompensate(super, back_vec, thSCD1=thSCD1, thSCD2=255)
forwcmp = source.MCompensate(super, forw_vec, thSCD1=thSCD1, thSCD2=255)

mcomp_clip = interleave(forwcmp, source, backcmp)

#---------------------------------

#Next four lines let you view the metrics in order to set detection metrics.
#script = """Subtitle("\nScene = " + String( (dFact * YDifferenceToNext + dMin ) ) + \
# "\nYDiffP = " + String( YDifferenceFromPrevious ), lsp=0)"""
#ScriptClip(mcomp_clip,script)

#Evaluate every 3rd frame using Mod(3) = 1
#Comment out thse lines when viewing metrics with code above
WriteFileIf(mcomp_clip, filename, "(YDifferenceFromPrevious > \
(dFact * YDifferenceToNext + dMin )) && (current_frame %3 == 1)", \
"current_frame/3", append = false)
#######################
return selectevery(3,1)
#############################################################


mcomp_clip removed from INSIDE WritefileIf script but still need as argument to WriteFileIf (becomes Last inside).
Added Levels=1 to 2nd MSuper.

and Auto Contrast update

Function JMLevels(clip c,Float "Strength",Float "Gamma",Float "MinRngPerc",int "MinLen",Float "ScanPerc",Float "Ignore",bool "PCLevels",
\ Int "Start",Int "End",Bool "Show") {
/*
JMLevels(), Auto Contrast Adjustment.
*/
c
myName="JMLevels: "
Strength=Max(Min(Float(Default(Strength,0.5)),1.0),0.0)
Gamma=Float(Default(Gamma,1.0))
MinRngPerc=Max(Min(Float(Default(MinRngPerc,33.3)),100.0),0.0) # If luma range less than MinRngPerc, then ignore
MinLen=Default(MinLen,25) # If less than MinLen frames, then ignore
Samples = RT_Undefined
PCLevels=Default(PCLevels,False)
GScript("""
if(ScanPerc.Defined && ScanPerc>0.0 && ScanPerc<=100.0) {
Samples = Int(FrameCount * ScanPerc / 100.0 + 0.5)
Samples = (Samples<32) ? 32 : Samples
}
Show=Default(Show,False)
if(FrameCount>=MinLen && STRENGTH>0.0) { # Auto Contrast Adjustment
Eval(RT_QueryLumaMinMax(Samples=Samples,Ignore=Ignore,X=8,Y=8,W=-8,H=-8,Start=Start,End=End,Debug=Show))
# Ignore possible remaining borders (after RoboCrop)

if(IsRGB()||PCLevels) { CSMin = 0 CSMax = 255 }
else { CSMin = 16 CSMax = 235 }
QLRng=QLMMMax-QLMMMin+1
CSRng=CSMax-CSMin+1
CSRngMin=CSRng*MinRngPerc/100.0
if(QLRng >= CSRngMin) { # Ignore if < MinRngPerc of CS range
ALMin = Int(CSMin - ((CSMin - QLMMMin) * STRENGTH) + 0.5) # Round Up
ALMax = Int(CSMax - ((CSMax - QLMMMax) * STRENGTH)) # Round down
if(ALMin!=CSMin || ALMax != CSMax) {
(Show)?RT_DebugF("Levels(%3d,%.2f,%3d,%d,%d,Coring=False)",ALMin,Gamma,ALMax,CSMin,CSMax,name=myName):NOP
Levels(ALMin,Gamma,ALMax,CSMin,CSMax,Coring=False)
(Show) ?RT_Subtitle("%sLevels(%3d,%.2f,%3d,%d,%d,Coring=False)",myName,ALMin,Gamma,ALMax,CSMin,CSMax) :NOP
}
} Else if(Show){
RT_DebugF("Skip %3d,%3d Range=%d %.2f%%",QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng,name=myName)
RT_Subtitle("%sSkip %3d,%3d Range=%d %.2f%%",myName,QLMMMin,QLMMMax,QLRng,QLRng*100.0/CSRng)
}
}
""")
Return Last
}

Avisource("D:\avs\StarWars.avi")

STRENGTH=1.0 # Heavy Duty (0.0->1.0)
IGNORE =20.0 # Ultra Heavy Duty (0.0->100.0)
SCANPERC=RT_Undefined # default (0.0<SCANPERC<=100.0)
PCLEVELS=True
MinRngPerc=0.0 # OFF
Start =0 # Start Frame to scan(override auto credits skipping) (0=start frame)
End =0 # End Frame scan (override auto credits skipping) (0=FrameCount-1)
SHOW=True # Also sends to DebugView
ADD=0 # Additional to borders cropped
Laced=False # True if interlaced for RoboCrop

RoboCrop(Start=Start,End=End,LeftAdd=ADD,RightAdd=ADD,TopAdd=ADD,botAdd=ADD,Laced=Laced,Debug=True) # Borders will upset JMLevels

Levels(0,1.0,255,48,255-48,coring=False)
BAD=Last
JMLevels(Strength=STRENGTH,MinRngPerc=MinRngPerc,ScanPerc=SCANPERC,Ignore=IGNORE,PCLevels=PCLEVELS,Start=Start,End=End,Show=SHOW)
StackVertical(BAD,Last)


EDIT: Added PCLevels
EDIT: Added, MinRngPerc=0.0 # OFF
EDIT: Added Start and End Args.
EDIT: Added Robocrop (as always want to get rid of borders before doing anything)

[johnmeyer]

John, ripped out of DBSC


Function JMLevels(clip c,Float "Strength",Float "Gamma",Float "MinRngPerc",int "MinLen",Bool "Show") {
<snip>
Well, I tried this function, but could not get the results I expected.

1. I could not get the "strength" parameter to do anything.

2. The gamma parameter appears to simply act as a global exposure control. I tried changing MinRngPerc, but it doesn't seem to limit the function until it is almost 90 (out of 100), and then it simply shuts it off for the entire video, not just those scenes that don't need it.

So, perhaps I am not understanding what this function is supposed to do. I thought it was going to be a smarter way to control contrast, but apparently that isn't what it is supposed to do.

[StainlessS]

Is a PreScan clip Global function, If you want to scan more than default 32 frames then set ScanPerc (but will of course then take more than a fraction of a second).
See Update.

You want Strength always at 1.0 for your low contrast stuff.
If you want to go overboard then set Ignore to eg 20.0.
The Function was written to scan clips that have already been separated into scene cuts (DBSC) so MinRngPerc, MinLen quite useless (EDIT: For you).

EDIT: Use BEFORE MC stuff.

EDIT: Added to previous post, MinRngPerc=0.0 # OFF
EDIT: Added Start and End Args.
EDIT: Added Robocrop (as always want to get rid of borders before doing anything)

EDIT: By the way, the "&& (current_frame %3 == 1)" part of condition does not matter when you add the SelectEvery(3,1) thing at end.
Got 3 versions of CPP plug working but not quite right as yet. Auto detects grayscale frames and switches to luma only mode detector
for those frames.

[StainlessS]

John, perhaps you were wanting to test out Beta SCSC,


Function SCSC(clip dclip,clip Motion,clip Start,clip End,
\ Float "dFact"=1.5,Float "dMin"=2.0,Int "dMinLen"=6,Float "dCW"=1.0/3.0,
\ int "x"=0,Int "y"=0,Int "w"=0,Int "h"=0,
\ Bool "AltScan"=False,Int "CDf"=4,Float "CTh"=2.0,Bool "ChromaI"=false,
\ Int "Matrix"=(dClip.Height>1100 || dClip.Width>600?3:2),
\ String "Project"="",Bool "Show"=False,Bool "Verbose"=False)

dCW = ChromaWeight for detection. [RGB, if set 0.0 then RGB->LumaY [Matrix], else is (Pixel Diff Ave for R + G + B) / 3.0].
Matrix, 2=PC601, 3=PC709.
CDf and CTh control 'Color Frames Detect', CDf=-1 to switch off and use Chromaweight only.
x,y,w,h coords detect area.
Altscan, scan only every other raster line.
ChromaI, YV12 has interlaced chroma (For Altscan/coords).
Project not yet working.

Function SCMC(clip dclip,clip Motion,clip Start,clip End,
\ Float "dFact"=1.5,Float "dMin"=2.0,Int "dMinLen"=6,Float "dCW"=1.0/3.0,
\ int "x"=0,Int "y"=0,Int "w"=0,Int "h"=0,
\ Bool "AltScan"=False,Int "CDf"=4,Float "CTh"=2.0,Bool "ChromaI"=false,
\ Int "Matrix"=(dClip.Height>1100 || dClip.Width>600?3:2),
\ String "Project"="",Bool "Show"=False,Bool "Verbose"=False)

Motion Compensated detector, as SCSC but expects dClip 3x length of Motion.


Project not currently working, was but not since changes. (writes Start/End/Mid and Range files).
Will add Override text file read in before scanning start and preset overrides + or -.
Above Doc is a bit spartan.

MC version script for SCMC

#Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

Project = "Scenes"

global dFact = 1.5
global dMin = 2.0
BLOCKSIZE = 8
thSCD1 = (BLOCKSIZE*BLOCKSIZE) * 64

source = AVISource("1955 Lions @ 49ers 4th Quarter_07 (23.976).mp4.AVI").convertTOYV12().killaudio()
#setMTMode(2)

prefiltered = source.blur(1.0)
superfilt = MSuper(prefiltered,pel=2)

fvec = MAnalyse(superfilt,isb=false, blksize=BLOCKSIZE,overlap=2,search=0)
bvec = MAnalyse(superfilt,isb=true, blksize=BLOCKSIZE,overlap=2,search=0)

super = MSuper(source, pel=2,Levels=1)

fcmp = source.MCompensate(super, fvec, thSCD1=thSCD1, thSCD2=255)
bcmp = source.MCompensate(super, bvec, thSCD1=thSCD1, thSCD2=255)
#fcmp = source.MFlow(super, fvec, thSCD1=thSCD1, thSCD2=255)
#bcmp = source.MFlow(super, bvec, thSCD1=thSCD1, thSCD2=255)


mcomp_clip = interleave(fcmp, source, bcmp)

SCMC(mcomp_clip,source ,source ,source ,dFact=dfact,dMin=dMin,Project=Project,Show=True)

#SCMCUF(mcomp_clip,source ,source ,source ,Project=Project)


Here, http://www.mediafire.com/download/6bj6jt00jpb33h7/SCSC_v0.0Beta.zip

Dont need bug reports, significant changes yet to occur.

[johnmeyer]

I'm in the middle of a 1000+ feet of film transfer. None of these scene detection scripts are working very well. I get "double hits" on many scenes (two scene detects in a row, often one or two frames away from the actual scene change), and many other obvious scene changes are missed. The dFact/dMin just isn't working very well, but my YDiffNext/YdiffPrev doesn't work any better.

I think I'm just going to give up on scene detection. I've spent too much time chasing this, and while all the (five) different methods I've tried can be made to work on short clips, none of them seem to work very reliably -- even after tuning the variables -- with most real-life material.

This is no reflection on your work or talent, but rather a reflection of the difficulty of the challenge. I just can't spend any more time on this, and therefore am giving up. Thanks for all your help!

[StainlessS]

OK john.

Here MC script to help visualize what the plugin is seeing


#Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

Project = "Scenes"

global dFact = 2.5
global dMin = 1.5
BLOCKSIZE = 8
thSCD1 = (BLOCKSIZE*BLOCKSIZE) * 64

source = AVISource("1955 Lions @ 49ers 4th Quarter_07 (23.976).mp4.AVI").convertTOYV12().killaudio()
#source = AVISource("D:\avs\Starwars.AVI").convertTOYV12().killaudio()
#setMTMode(2)

prefiltered = source.blur(1.0)
superfilt = MSuper(prefiltered,pel=2)

fvec = MAnalyse(superfilt,isb=false, blksize=BLOCKSIZE,overlap=2,search=0)
bvec = MAnalyse(superfilt,isb=true, blksize=BLOCKSIZE,overlap=2,search=0)

super = MSuper(source, pel=2,Levels=1)

fcmp = source.MCompensate(super, fvec, thSCD1=thSCD1, thSCD2=255)
bcmp = source.MCompensate(super, bvec, thSCD1=thSCD1, thSCD2=255)
#fcmp = source.MFlow(super, fvec, thSCD1=thSCD1, thSCD2=255)
#bcmp = source.MFlow(super, bvec, thSCD1=thSCD1, thSCD2=255)

mc = interleave(fcmp, source, bcmp)

A=fcmp.ScriptClip("""Subtitle(String(current_frame)+"F",align=5,size=80)""")
B=source.ScriptClip("""Subtitle(String(current_frame),align=5,size=80)""")
C=bcmp.ScriptClip("""Subtitle(String(current_frame)+"B",align=5,size=80)""")
D=A.SelectEvery(1,1)
E=B.SelectEvery(1,1)
F=C.SelectEvery(1,1)

WW=source.width
HH=((source.Height+12)/24)*4

X=StackHorizontal(A,B,C,D,E,F).BilinearResize(WW,HH)

MAP=StackVertical(Source,X)
SCMC(mc,MAP ,MAP ,MAP ,dFact=dfact,dMin=dMin,Project=Project,Show=True)

https://s20.postimg.org/42q62w1od/49_zpswiakotgs.jpg (https://postimg.org/image/sj8bxd2ex/)

[AlonP]

I was wondering if when you are done with it, you could suggest various ways in which this script could be used? Right now I can think of Like, some anime-dvds I have whenever there's a scene change there's like a "tear" on the lower part, so I freeze-framed them manually and replaced those, so this scene-change-detect could prolly automate it... But like, a bunch of "examples" would be nice - for those who are less comfortable with using such things... If you have the time and feel like it - naturally ;).

[johnmeyer]

Automating anything having to do with scene changes is a really, really bad idea, IMHO.

As one example, I was hoping to delete the first frame at the beginning of each scene change in amateur film transfers because most of these are over-exposed. This happens because the film camera rotating shutter doesn't get up to speed instantly and therefore almost always over-exposes the first frame. However, if the scene change fails by mis-identifying a scene change, you end up deleting a perfectly good frame right in the middle of a scene. This means that you have made the film worse, not better. Not good.

In addition, even if the scene change is perfect (something I've never achieved, except on really short test clips, even with StainlessS' excellent scripts), sometimes there isn't a problem at the scene change, and an automated process will delete frames that you don't need to remove.

It is my strong belief that scene detect logic (which I've given up on, after eight years of trying to get this to work), should be used as a way to quickly get to (most) scene changes while working within your NLE. With this workflow, you don't have to go step-by-step at each scene change to find it, and then do something. This is a huge time saver, and is usually worth the time it takes to do the scene detection. This means that the speed of the scene detection script is really important, and is why, when I use these less than perfect scripts, I usually use the ones I created, because they usually achieve almost 10x real-time speed when dealing with 16 fps films.

[edit]My workflow is to export the scene change frame numbers, and then import these into my NLE (Vegas) as markers. I can move to these markers with one keypress. I have then written Vegas scripts which let me delete the first frame of a new scene with one keypress. So, while the editing is repetitively tedious, it is also very straightforward, and usually only takes 10-15 minutes, even for a very long film.

[StainlessS]

@Lyris,

Further to your PM requesting guide on DBSC_ProcessByDAR(), this is as good as you are going to get (me no likee docs nor guides).

Make a test clip with several different DAR's

# MakeProcessByDARTest.avs

Avisource("E:\V\Avatar.avi") # 720x576@25-16:9, 232627 frames (full frames-no borders)

# Pick 9 x 1000 frames, making sure there will be a scene change at each (we step 2000 at a time).
Trim(100000,-1000)++Trim(102000,-1000)++Trim(104000,-1000)++Trim(106000,-1000)++Trim(108000,-1000)++
\ Trim(110000,-1000)++Trim(112000,-1000)++Trim(114000,-1000)++Trim(116000,-1000)

Clip169=Last # source

LR = (720/2) - (540/2)
Clip43 = LetterBox(0,0,LR,LR) # 4:3

TB = (576/2) - (432/2)
Clip237 = LetterBox(TB,TB) # 2.37:1

NICK=""" # Nicknames, use names instead of clip number in SCMD
C169 = 0 # ClipClop source clip
C43 = 1
C237 = 2
"""

SCMD="""
C169 0, -1000
C43 1000, -1000
C169 2000, -1000
C237 3000, -1000
C169 4000, -1000
C43 5000, -1000
C169 6000, -1000
C237 7000, -1000
C169 8000, -1000
"""

SHOW=False

ClipClop(Clip169,Clip43,Clip237,scmd=SCMD,NickName=NICK,show=SHOW,dv=5)

# Save Results as "AvatarTest.avi"


Create ClipClop command file

# DBSC_ProcessByDAR_Client.AVS

Import("DBSC.avs")

VideoFileName = "AvatarTest.avi"

InfoDB = VideoFileName+"_Info.DB" # INFO DBase
ScanDB = VideoFileName+"_Scan.DB" # SCAN DBase
LogFn = VideoFileName+"_User.Log"
SpliceDB = VideoFileName+"_Splice.DB"

RECYCLE_INFO_DB= True
RECYCLE_SCAN_DB= True

Avisource(VideoFileName)

DBSC_CreateInfo(InfoDB,RecycleDB=RECYCLE_INFO_DB) # Create INFO DB (REQUIRED), for fast scanning
DBSC_Scan(InfoDB,ScanDB,RecycleDB=RECYCLE_SCAN_DB) # Detect Scene Changes and store in ScanDB

### Below, unlikely to be necessary for DAR detection.
#DBSC_Override(ScanDB,"Override.txt") # Overrides, add/remove detections in Override.txt
#Return DBSC_SubsClip(InfoDB,ScanDB,Subs=True,Metrics=true,Metrics2=true) # View detections
#Return DBSC_StackSceneChanges(Last,InfoDB,ScanDB) # Check detections START/END of scene STACKED.
###

# DBSC_DynaCrop MUST have been called (at some time) before DBSC_DynaSplice (then comment it out)
RLBT=$F # Left, Right, top and bottom CROPPING
CropLimit_Left = 90 # You choose, normally avoids eg overcropping on credits
CropLimit_Top = 72 # We chose these numbers because we know what borders we added
CropLimit_Right = 90 # The defaults are 16
CropLimit_Bot = 72 # (was not originally intended to have more than 1 DAR)
DynaCrop_Show = True
DBSC_DynaCrop(ScanDB,SpliceDB=SpliceDB,rlbt=RLBT,
\ CropLimit_Left=CropLimit_Left,CropLimit_Top=CropLimit_Top,CropLimit_Right=CropLimit_Right,CropLimit_Bot=CropLimit_Bot,
\ LeftAdd=0,RightAdd=0,TopAdd=0,BotAdd=0,LogFn=LogFn,Show=DynaCrop_Show,Debug=True)
#Return Last # Results will be all border cropped and resized to original frame size, ie for variable DAR some will be distorted.

###
# When DBSC_DynaCrop() has already created Splice DBase.
# Source full Frame DAR, ClipClop source DAR, ClipClop Replacement clips 1, & 2.
# We set just the DAR's we are wanting, and nullify the others (avoid mistakes)
# Will create "ClipClopCmd.txt".
FUNC = "DBSC_ProcessByDAR(SpliceDB=SpliceDB,SceneNo=SceneNo,Show=True," +
\ "FDAR=16.0/9.0," +
\ "AR1=16.0/9.0," +
\ "AR2=4.0/3.0," +
\ "AR3=2.37," +
\ "AR4=0.0,AR5=0.0,AR6=0.0,AR7=0.0,AR8=0.0)"

DBSC_DynaSplice(SpliceDB,func=FUNC) # ID DAR and create ClipClop command file
Return Last


Created ClipClop command file

1 0,37 # SceneNo=1 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 38,92 # SceneNo=2 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 93,178 # SceneNo=3 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 179,239 # SceneNo=4 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 240,296 # SceneNo=5 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 297,421 # SceneNo=6 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 422,486 # SceneNo=7 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 487,559 # SceneNo=8 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 560,617 # SceneNo=9 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 618,626 # SceneNo=10 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 627,637 # SceneNo=11 Crop(0,0,716,574) CropAR=1.77 BestAR=1.78
1 638,654 # SceneNo=12 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 655,681 # SceneNo=13 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 682,781 # SceneNo=14 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 782,848 # SceneNo=15 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 849,909 # SceneNo=16 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 910,957 # SceneNo=17 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 958,999 # SceneNo=18 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
2 1000,1090 # SceneNo=19 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1091,1132 # SceneNo=20 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 1133,1160 # SceneNo=21 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 1161,1229 # SceneNo=22 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1230,1287 # SceneNo=23 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1288,1336 # SceneNo=24 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1337,1401 # SceneNo=25 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1402,1461 # SceneNo=26 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1462,1510 # SceneNo=27 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 1511,1564 # SceneNo=28 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1565,1730 # SceneNo=29 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1731,1813 # SceneNo=30 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 1814,1850 # SceneNo=31 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1851,1964 # SceneNo=32 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 1965,1999 # SceneNo=33 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
1 2000,2050 # SceneNo=34 Crop(0,16,720,558) CropAR=1.84 BestAR=1.78
1 2051,2265 # SceneNo=35 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 2266,2353 # SceneNo=36 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 2354,2400 # SceneNo=37 Crop(70,0,648,574) CropAR=1.61 BestAR=1.78
1 2401,2457 # SceneNo=38 Crop(0,0,716,574) CropAR=1.77 BestAR=1.78
1 2458,2611 # SceneNo=39 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 2612,2698 # SceneNo=40 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 2699,2790 # SceneNo=41 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 2791,2971 # SceneNo=42 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 2972,2999 # SceneNo=43 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
3 3000,3033 # SceneNo=44 Crop(2,72,716,432) CropAR=2.36 BestAR=2.37
3 3034,3146 # SceneNo=45 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3147,3244 # SceneNo=46 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3245,3320 # SceneNo=47 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3321,3369 # SceneNo=48 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3370,3418 # SceneNo=49 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3419,3447 # SceneNo=50 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3448,3472 # SceneNo=51 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3473,3569 # SceneNo=52 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3570,3713 # SceneNo=53 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3714,3764 # SceneNo=54 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3765,3865 # SceneNo=55 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 3866,3999 # SceneNo=56 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
1 4000,4092 # SceneNo=57 Crop(0,8,640,566) CropAR=1.61 BestAR=1.78
1 4093,4156 # SceneNo=58 Crop(0,0,720,504) CropAR=2.03 BestAR=1.78
1 4157,4345 # SceneNo=59 Crop(0,0,716,574) CropAR=1.77 BestAR=1.78
1 4346,4504 # SceneNo=60 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 4505,4580 # SceneNo=61 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 4581,4608 # SceneNo=62 Crop(0,0,716,574) CropAR=1.77 BestAR=1.78
1 4609,4624 # SceneNo=63 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 4625,4677 # SceneNo=64 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 4678,4734 # SceneNo=65 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 4735,4813 # SceneNo=66 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 4814,4947 # SceneNo=67 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 4948,4999 # SceneNo=68 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
2 5000,5043 # SceneNo=69 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 5044,5092 # SceneNo=70 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 5093,5141 # SceneNo=71 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 5142,5180 # SceneNo=72 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 5181,5239 # SceneNo=73 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 5240,5677 # SceneNo=74 Crop(90,0,540,576) CropAR=1.33 BestAR=1.33
2 5678,5728 # SceneNo=75 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 5729,5857 # SceneNo=76 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 5858,5966 # SceneNo=77 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
2 5967,5999 # SceneNo=78 Crop(90,0,540,574) CropAR=1.34 BestAR=1.33
1 6000,6007 # SceneNo=79 Crop(0,0,720,576) CropAR=1.78 BestAR=1.78
1 6008,6277 # SceneNo=80 Crop(36,0,684,504) CropAR=1.93 BestAR=1.78
1 6278,6592 # SceneNo=81 Crop(38,0,644,554) CropAR=1.65 BestAR=1.78
1 6593,6940 # SceneNo=82 Crop(36,0,684,514) CropAR=1.89 BestAR=1.78
1 6941,6999 # SceneNo=83 Crop(0,0,720,504) CropAR=2.03 BestAR=1.78
3 7000,7275 # SceneNo=84 Crop(0,72,720,432) CropAR=2.37 BestAR=2.37
3 7276,7427 # SceneNo=85 Crop(10,72,672,432) CropAR=2.21 BestAR=2.37
3 7428,7668 # SceneNo=86 Crop(80,72,638,432) CropAR=2.10 BestAR=2.37
3 7669,7790 # SceneNo=87 Crop(0,72,640,432) CropAR=2.11 BestAR=2.37
3 7791,7999 # SceneNo=88 Crop(0,72,716,432) CropAR=2.36 BestAR=2.37
1 8000,8085 # SceneNo=89 Crop(0,6,720,568) CropAR=1.80 BestAR=1.78
1 8086,8162 # SceneNo=90 Crop(0,0,716,574) CropAR=1.77 BestAR=1.78
1 8163,8363 # SceneNo=91 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 8364,8501 # SceneNo=92 Crop(0,0,720,504) CropAR=2.03 BestAR=1.78
1 8502,8586 # SceneNo=93 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 8587,8691 # SceneNo=94 Crop(0,0,720,564) CropAR=1.82 BestAR=1.78
1 8692,8773 # SceneNo=95 Crop(80,0,640,574) CropAR=1.59 BestAR=1.78
1 8774,8830 # SceneNo=96 Crop(0,0,716,574) CropAR=1.77 BestAR=1.78
1 8831,8936 # SceneNo=97 Crop(0,0,720,574) CropAR=1.78 BestAR=1.78
1 8937,8998 # SceneNo=98 Crop(2,0,716,504) CropAR=2.02 BestAR=1.78
1 8999,8999 # SceneNo=99 Crop(44,2,668,502) CropAR=1.89 BestAR=1.78


Final ClipClop script

# ProcessByDAR_ClipClop_Client.AVS

Avisource("AvatarTest.avi")
ORG=Last

Clip169 = Subtitle("16:9",size=40,align=5) # Whatever you want to do with each
Clip43 = Subtitle("4:3",size=40,align=5)
Clip237 = Subtitle("2.37:1",size=40,align=5)

SHOW=True
ClipClop(ORG,Clip169,Clip43,Clip237,cmd="ClipClopCmd.txt",show=SHOW)
# Above clips same order as AR1, AR2, and AR3, DBSC_ProcessByDAR().

/*
FUNC = "DBSC_ProcessByDAR(SpliceDB=SpliceDB,SceneNo=SceneNo,Show=True," +
\ "FDAR=16.0/9.0," +
\ "AR1=16.0/9.0," +
\ "AR2=4.0/3.0," +
\ "AR3=2.37," +
\ "AR4=0.0,AR5=0.0,AR6=0.0,AR7=0.0,AR8=0.0)"

*/

[Lyris]

Thanks Stainless,
That clears things up. There's only one problem I'm having now.

When I run the DBSC_ProcessByDAR_Client script, it works perfectly and resizes clips with letterboxing.

But on the ClipClopCmd.txt output, every clip is identified as "1" (first character of each line) so I can't do much with it, unless I manually change the appropriate lines to read "2". Then I can process away to my heart's content.

How do I fix that?

[StainlessS]

Show me the ClipClopCmd.txt file!

[StainlessS]

Also wrap up DBSC_ProcessByDAR_Client.AVS, and ProcessByDAR_ClipClop_Client.AVS and output from DebugView in zip, thanks.

EDIT: The CropLimits., are maximum inner allowed limit, ALL DAR sections are limited to this, suspect you have done
limit on outermost and not innermost.
If auto cropping exceeds these limits, it assumes a mistake and re-does with different parameters, but still limited to the limits,
ie looking for LESS cropping than was detected.

You want to set the MAX cropped LEFT for any DAR, etc.

[StainlessS]

This only expands (never contracts) to maximum PC Levels range 0->255.


Function MaxContrast(clip c,int "Samples",Float "Ignore",Float "Gamma",Bool "Show") { # PC Levels Max Contrast
c Gamma=Float(Default(Gamma,1.0)) Show=Default(Show,False)
Eval(RT_QueryLumaMinMax(Samples=Samples,Ignore=Ignore,X=4,Y=4,W=-4,H=-4,Debug=Show))
Doit = (QLMMMin!=0 || QLMMMax != 255)
(Doit && Show) ? RT_DebugF("Levels(%3d,%.2f,%3d,%d,%d,Coring=False)",QLMMMin,Gamma,QLMMMax,0,255,name="MaxContrast: ") : NOP
(Doit && Show) ? RT_Subtitle("%sLevels(%3d,%.2f,%3d,%d,%d,Coring=False)","MaxContrast: ",QLMMMin,Gamma,QLMMMax,0,255) : NOP
(Doit) ? Levels(QLMMMin,Gamma,QLMMMax,0,255,Coring=False) : NOP
Return Last
}


Samples 'Samples' frames and uses minimum and maximum pixel value (after ignoring up to 'Ignore' [default 0.2%] extremes per frame ie noise)
and expands entire clip range using Levels. Usually takes less than a second to scan default samples (32) before frame serving begins.
Faster than eg ColorYUV(AutoGain=true) as Lut built before frame serving, and not at every frame.
EDIT: Also does not produce fluctuating levels as large objects enter or leave frame, as ColorYUV(Autogain=true) does.

NOTE, results are at PC Levels.

Useful during eg scene change detection or during security camera motion detection utils.

RoboCrop beforehand may be useful.

EDIT: Post #61 above outlines RT_QueryLumaMinMax() usage.
EDIT: MaxContrast() intended for use on detection clip only.

[Dogway]

The script works! despite not using runtime filters, what kind of wizardry did you do? :)

So far for temporal stable auto-contrast I've been using AutoAdjust, since it's a plugin it runs fast with accelerated CPU instructions, although the output differs a lot from yours.

AutoAdjust(temporal_radius=10,auto_balance=true,auto_gain=true,use_interp=true,\
avg_safety=0.25,dark_limit=10,bright_limit=10,gamma_limit=1.0,dark_exclude=0.05,bright_exclude=0.05,\
chroma_process=true,scd_threshold=16,input_tv=false,output_tv=false,high_bitdepth=false,debug_view=false)

RT_QueryLumaMinMax can come handy for mvtools2 at modulating the thSAD derived from the auto-contrast of the prefilter clip to be fed for motion search. So thSAD can be as dynamic as contrast (not sure if they cancel each other until I test)

Also wanted to ask, I would have said there was a post here describing how to output scenes to files? But I wonder how that could be possible in avisynth if not for a avspmod macro...

[StainlessS]

The script works! despite not using runtime filters, what kind of wizardry did you do?

RT_QueryLumaMinMax() Samples the frame luma itself, DBSC uses RT_YStats to store this info in DBSC DBase


# Info for Frame Number n ie record number
# INT YStats Flags Full name Local Var Range
# 0 1($01) RT_YPlaneMin() "yMin" (0->255)
# 1 2($02) RT_YPlaneMax() "yMax" (0->255)
# 2 4($04) RT_YPlaneMinMaxDifference() "yMinMaxDiff" (0->255)
# 3 8($08) RT_YPlaneMedian() "yMed" (0->255)
# FLOAT
# 4 16($10) RT_AverageLuma() "yAve" (0.0->255.0)
# 5 32($20) RT_YPlaneStdev() "yStdev" (0.0->255.0)
# -
# 6 RT_FrameDifference(n <-> n+1) (0.0->255.0)


So far for temporal stable auto-contrast I've been using AutoAdjust, since it's a plugin it runs fast with accelerated CPU instructions, although the output differs a lot from yours.


The Auto Contrast funcs including MaxContrast() posted a few posts earlier, just sample a number of frames before frame serving starts,
and calls Leves() to pre-calculate the Lut, I would imagine that it is somewhat faster than AutoAdjust which does the auto stuff in real time.
EDIT: Correction, the DBSC auto contrast stuff does the sampling when calling DBSC_Levels() on a scene change, so is done once on each scene.

Also wanted to ask, I would have said there was a post here describing how to output scenes to files? But I wonder how that could be possible in avisynth if not for a avspmod macro...


I dont use AvsPMod, dont know what it can do.

Below writes Frames Files, if they are of use.

Function DBSC_WriteFiles(String ScanDB,String "Start_Fn",String "End_Fn",String "Range_Fn",String "Mid_Fn",String "ClipClopCmd_Fn",
\ Bool "Comment",Bool "Comma",String "VideoFileName") {
/*
Write Scene info to output files:- Start, End, Mid Frame and Range.
Also writes a ClipClop command file for Start (clip 1) and End (clip 2) frames, used by DBSC_Select().
ScanDB, SCAN DBase file created via DBSC_Scan.
Comment, Default True writes additional Hash (#) separated info to file. False, omits extra info.
Comma, Default True, use Comma (,) range separator, Else False uses SPACE range separator.
VideoFilename, can supply clip name, added to Comment Header.
*/


For Video Only, could perhaps knock up a DBSC_DynaSplice User function to call TWriteAVI() to write AVI Video (TWriteAVI does not support audio, yet).

EDIT: I think there is a plugin that also writes audio, maybe SoundOut or something like that, perhaps call something via eg CallCmd()
to mux Video and Audio together.

[Matt Kirby]

At first: Thank you for this script.
I tried your (older) scene detection with "SC_SceneSelect.avs" and "DynaCrop.AVS". The results are quite good.

Now a gave DBSC a chance. But the results are worse in my opinion.

Import("c:\Program Files (x86)\AviSynth+\plugins\DBSC.avs")
VideoFileName="Fanto.mkv"
LWLibavVideoSource(VideoFileName)
InfoDB = VideoFileName+"_Info.DB" # INFO DBase
ScanDB = VideoFileName+"_Scan.DB" # SCAN DBase
LogFn = VideoFileName+"_User.Log"
SpliceDB = VideoFileName+"_Splice.DB"
RECYCLE_INFO_DB= True
RECYCLE_SCAN_DB= True
DBSC_CreateInfo(InfoDB,RecycleDB=RECYCLE_INFO_DB) # Create INFO DB (REQUIRED), for fast scanning
DBSC_Scan(InfoDB,ScanDB,RecycleDB=RECYCLE_SCAN_DB,dFact=1.1, dMinim=2.0, dMinLen =2) # Detect Scene Changes and store in ScanDB
DBSC_WriteFiles(ScanDB,"Start.txt","End.txt",Comment=false)

I have a source with a "complicated" scene change that isn't detected by the DBSC, but the older "SC_SceneSelect" does it.
I must go down to dFact 1.1 and dMinim=2.0 with my source-material.
The older scene detection work here good, the newer (DBSC) one doesn't

What can I do (which settings have I to choose), that the DBSC detects this scene change?

https://www.dropbox.com/sh/djgmzylsnlbgz26/AAAodymYFkPErht-Us7kPsHra?dl=0

[StainlessS]

Why are you using DBSC ?
It is intended to deal with source which has many changes in borders.
(The clip you gave with only one scene change looks a bit "cloudy midnight in the country", ie all dark, with no border change).

[Matt Kirby]

I need a scene detection tool because I want a framelist of all scene changes. My videos have no dublicate frames. I want to cut the videos in parts at scene change given by a txt-file.

So I thought DBSC ist the best way for it. If not, I'll stay with SC_SceneSelect.avs and the SC_MakeTrimFile function.

[StainlessS]

You need both start and end frames or just start frames ?
(I've been up all night and am knackered, somehow got confused between dupes detector and scene change detector :( I deleted that stuff before you posted)

[Matt Kirby]

actually it doesn't matter if the textfile contains start and endframes or only startframes. I need just the mark information between the scenes. My further way will get along with any information where the scene change is.

My question was: why BDSC doesn't recognize this scene change in the video but SC_MakeTrimFile does...

[StainlessS]

Make dfact=4.0 and dminim=4.0. [same as the other one).

DBSC defaults are deliberately a little bit shy on detect, as can set manual overrides.

[Matt Kirby]

I have a source with a "complicated" scene change that isn't detected by the DBSC, but the older "SC_SceneSelect" does it.
I must go down to dFact 1.1 and dMinim=2.0 with my source-material.
The older scene detection work here good, the newer (DBSC) one doesn't

What can I do (which settings have I to choose), that the DBSC detects this scene change?

I already went down to "1.1" and "2.0"

DBSC doesn't work at this scene, see at video

[Matt Kirby]

I mean your "SC_MakeTrimFile" function is the best scene detection for me.
I made a GUI for that. The user must just install avisynth+ and can run a scene detection without any knowledge of scripts.
Here is how the tool looks like. I hope you don't mind.

[StainlessS]

DBSC is way better. Current version [not prep'ed for publishing] is at SendSpace below this post, for 30 days only.

Requires RT_Stats v2.0Beta12, AVS+, Gscript [EDIT: wrapper] stuff removed, but would still run under v2.60 Std with AvsInit() if DBSC.avsi added to AvsInit GIMPORT folder and Gscript in plugins.

This the one I mostly use

DBSC_Cropper.avst [ an Avisynthesizer_Mod Template ] # EDIT: Batch mode template, if multiple avi files sent to Avisynththesizer_mod, then creates multuiple avs files, one for each source.

#ASYNTHER DBSC_Cropper

VideoFileName ="___FILE___"
VideoFileName = RT_GetFullPathName(VideoFileName)
ScanDB = VideoFileName+"_Scan.DB"
LogFn = VideoFileName+"_User.Log"
SpliceDB = VideoFileName+"_Splice.DB"

Avisource(VideoFileName)

dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
RoboCrop(WMod=2,HMod=2,Laced=False,Ignore=0.04,Align=True,debug=true,LeftSkip=0,TopSkip=0,RightSkip=0,botskip=0) # EDIT: Make sure these are 0, or as required (non zero in zip)
ORG=Last
OUTPUT_WID=Width OUTPUT_HIT=Height # EDIT: Output Re-size after cropping.
CropLimit_Left=16 CropLimit_Top=4 CropLimit_Right=16 CropLimit_Bot=4 # EDIT: The dynamic border cropping max's (also see RLBT)

# Allows set CropLimit_Left etc, then comment out
Return DBSC_ShowCrop(Lft=CropLimit_Left,Top=CropLimit_Top,Rgt=CropLimit_Right,Bot=CropLimit_Bot)


# Scan for scene changes. EDIT: Takes long time, watch DebugView
DBSC_Scan(ORG,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh)


/*
DBSC_Override(), Using SCAN DBase and Overrides.txt, allows user to set additional Scene Change or remove erroneous ones
from the SCAN DBase.

ScanDB, created by DBSC_Scan.
Override_fn filename: If first character == '*' then uses text in the Override_fn string and does not load file.
When using Override_fn as string, then each override should be on separate line (same format as file).
FORMAT: One frame numer command on each line.
Any +ve frame number sets START Of Scene Change frame.
Any -ve frame number REMOVES a scene change at that frame (START or END).
Set any overrides to detected scene changes, only call once to change DBase, then comment out.
*/
#DBSC_Override(ScanDB,"Override.txt") return Last

#Return DBSC_StackSceneChanges(ORG,ScanDB) # Check detections START/END of scene STACKED.


SHOW_DYNACROP_METRICS=false
RLBT=$F # 1=Top, 2=Bot, 4=Left, 8=Right. (1+2+4+8=15 = all sides)
BLANK=False # If True, DONT NOT CROP, Blank Border ONLY
# DBSC_DynaCrop MUST have been called (at some time) before DBSC_DynaSplice
DC_Clip=DBSC_DynaCrop(ORG,ScanDB,SpliceDB,W=OUTPUT_WID,H=OUTPUT_HIT,
\ rlbt=RLBT,CropThresh=-40.0,
\ CropLimit_Left=CropLimit_Left,CropLimit_Top=CropLimit_Top,CropLimit_Right=CropLimit_Right,CropLimit_Bot=CropLimit_Bot,
\ LeftAdd=0,TopAdd=0,RightAdd=0,BotAdd=0,
\ LogFn=LogFn,Debug=True,DissolveSkip=8,Blank=BLANK,Show=SHOW_DYNACROP_METRICS)

FUNC=""

DBSC_DynaSplice(ORG,SpliceDB,func=Func,W=OUTPUT_WID,H=OUTPUT_HIT)

Return Last # DO NOT USE PREFETCH


DBSC_ShowCrop() probably has not been shown before, uses RoboCrop(Show=True) to show cursors to set max possilble croppings.

Suggest dont touch Scan detect args, and only use Overrides where necessary to remove or add scene changes. DONT WORRY about a single scene fail, use override instead.

EDIT: Dont forget to start DebugView before running script, so you can watch progress.

[Dogway]

Hello StainlessS, where can I find DBSC_ShowCrop()? I need to crop random scenes on all sides dynamically, sometimes uncentered. Robocrop doesn't seem to do the job. :(

[StainlessS]

Hi Doggy :)
OK, this what I've last used [not for some time], showcrop in DBSC.avsi, Its on SendSpace, for 30 days.
Good luck [I really should get back to that at some point].

Also, I think you did some work on stab script, see here:- https://forum.doom9.org/showthread.php?p=1938351#post1938351
EDIT: Also see post #58 in that thread.

[Dogway]

Thanks for the head up, for some reason it didn't work for me, the cropping wasn't dynamic, does it need an analysis pass?
In any case I looked back to what I did and drafted a script that seems to do the job. It requires ResizersPack which I did long ago for some functions, one of them I had to update since it had a little bug.


function DogCrop(clip c, int "width", int "height", float "thr", bool "CropMore", int "ScanW", int "ScanH", int "mode") {

c
w = width()
h = height()

nw = Default(width,w)
nh = Default(height,h)
addw = Default(ScanW,round((width()/3)))
addh = Default(ScanH,round((height()/3)))
thr = Default(thr, 16.0) # Threshold, pixel values above this will be considered borders
CM = Default(CropMore, True) # In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
mode = Default(mode, 2) # 0: center+pad 1: crop to minimum 2: resize to maximum

sisphbd = AvsPlusVersionNumber > 2294
contoy = sisphbd ? !isy() : !isy8()

ScriptClip("""
contoy ? sisphbd ? converttoy() : converttoy8() : last

for (li=2, addw, 2) {
L1=crop(0,0,-w+li,0)
avgLL=AverageLuma(L1)
if (avgLL>=thr) {
x1=li-2
li=addw
}
}

for (ri=2, addw, 2) {
R1=crop(w-ri,0,0 ,0)
avgLR=AverageLuma(R1)
if (avgLR>=thr) {
x2=ri-2
ri=addw
}
}

for (ti=2, addh, 2) {
T1=crop(0,0,0,-h+ti)
avgLT=AverageLuma(T1)
if (avgLT>=thr) {
y1=ti-2
ti=addh
}
}

for (bi=2, addh, 2) {
B1=crop(0,h-bi,0 ,0)
avgLB=AverageLuma(B1)
if (avgLB>=thr) {
y2=bi-2
bi=addh
}
}

fx1 = CM ? x1+x1%2 : x1-x1%2
fx2 = CM ? x2+x2%2 : x2-x2%2
fy1 = CM ? y1+y1%2 : y1-y1%2
fy2 = CM ? y2+y2%2 : y2-y2%2

crop(c,fx1,fy1,-fx2,-fy2)
mode == 0 ? padresize(w,h) : \
mode == 1 ? padresize(w,h) : \
RatioResize(float(nw),"adjust2w").padresize(w,h)

""",args="c,nw,nh,addw,addh,w,h,thr,CM,sisphbd,contoy,mode")
padresize(nw,nh)
}

mode = 1 is not working ATM which aims to crop to maximum found border (might need an analysis pass for this)
Surely it can be improved/optimized as this is my first function in a long time.

[StainlessS]

What is does [roughly].

Firstly, Does a Robocrop() which finds outermost crop coords for entire clip. [border common thoughput entire clip].
EDIT: This border will be already cropped when using lines in script afterwards.

Then, using DBSC_ShowCrop() you set CropLimit_Left, CropLimit_top etc.
You must scan though the file in eg Vdub2 looking for greatest amount of border, and set the relevant coord [CropLimit_Left etc]
to just a little more than that [It sets the maximum allowed cropping, if more than that is detected, it is taken to be a mistake].
When you adjust CropLimit_Left [or other] then presss refresh in VD2 to update coord cursors [provided by current RoboCrop].
When all 4 coords limits set, then comment out DBSC showCrop line.

The use DBSC_Scan() line [takes a long time to prescan entire clip for detect scene changes], creates a DBASE of scenes.
[Can view output in DebugView. Then comment line out when done]

Then use DBSC_StackSceneChanges() to show detected scene changes. [first and last frames of scene changes, stacked].
[Then comment line out when done]

If you need to Override some detections or add new ones, then use DBSC_Override(), which updates the DBase, then comment DBSC_Override line out.

Can again check detected scene changes using DBSC_StackSceneChanges(), then again comment out.

Then, we get to dynamic borders.

Use DBSC_DynaCrop() to scan for changing borders, using the CropLeft_Limit style settings previously set using the DBSC_ShowCrop thingy.
Again, this DBSC_DynaCrop() scans each scene looking for borders, creating another DBase.
[Can view output in DebugView. Comment out when complete].

Finally, DBSC_DynaSplice(ORG,SpliceDB,func=Func,W=OUTPUT_WID,H=OUTPUT_HIT)
will crop and resize entire scenes at a time.
[and can perform other functions on the cropped resized clips using the Func thingy, but we will not be doing that here].

Save result.

Also, see a couple of posts earlier in BLUE [comments].

Your script will likely [if clip is any any great length] chop off chunks of image haphazardly, and instead of changing borders you will get changing image size, which may or may not be any kind of improvement :)

DBSC is flying by the seat of your pants, not for the faint hearted.

Good Luck.

[Dogway]

Thanks for the help, maybe I shouldn't name it *crop*, what it does is recenter the frame within the borders (in mode=0) so the image size won't change and use width and height to crop to a fixed value, unless you use mode=2 in which case it will resize to maximize width, and here you can see the frame (matte?) opening and closing on a shot by shot basis depending on title, this is very similar to what happens on some IMAX titles, the matte opens up for some shots.
I updated ResizersPack on my github (https://github.com/Dogway/Avisynth-Scripts).

[StainlessS]

OK thanks, I did not look at your script too closely at all.

[kedautinh12]

I seen many functions duplicates between DBSC, duplicity2, dropdeadgorgeous. I wonder functions duplicates will make errors when put them in same folder autoload??? Btw, are your new ver dbsc is 0.08???

[StainlessS]

So long as the functions are the same, then does not really matter, although I suppose I should give them different names,
or put them in some kind of common library file.
This is one of the duplicated functions.

Function StartOfSceneClip(clip c,Int "thSCD1",Int "thSCD2") { # All Luma Samples set 255 at SOS, else 0
thSCD1=Default(thSCD1,400) thSCD2=Default(thSCD2,130)
sup=c.MSuper(pel=1,sharp=0,rfilter=2,hpad=16, vpad=16)
fv=sup.MAnalyse(isb=false,delta=1,blksize=16)
Return c.MSCDetection(fv,thSCD1=thSCD1,thSCD2=thSCD2)
}

The last one to be loaded will be the one that is used.
Sorry, I aint been back to DBSC for quite some time, I have source to a dll plugin that I wrote a couple of years back,
which did fairly similar to the DBSC scan whotsit, I will probably adapt that dll to be used in DBSC [at some point].

EDIT: I've used DBSC many times since then, but not changed it.

[kedautinh12]

Thanks

[Boulder]

Would it be possible to have an option to create an x264/x265 QP file out of the scene change list? I've seen x265 miss a lot of easy scene changes so why not help the poor bugger (the encoder parameters don't help a thing).

The syntax is very simple, just frame number and frame type (K for keyframe so it's either I or i in the encode) like this:
10952 K
15243 K
22533 K
28734 K
36745 K

[StainlessS]

So the K frames are the Start of Scene Frames, Yes ?


Function DBSC_WriteFiles(String ScanDB,String "Start_Fn",String "End_Fn",String "Range_Fn",String "Mid_Fn",String "ClipClopCmd_Fn",
\ Bool "Comment",Bool "Comma",String "VideoFileName") {
/*
Write Scene info to output files:- Start, End, Mid Frame and Range.
Also writes a ClipClop command file for Start (clip 1) and End (clip 2) frames, used by DBSC_Select().
ScanDB, SCAN DBase file created via DBSC_Scan.
Comment, Default True writes additional Hash (#) separated info to file. False, omits extra info.
Comma, Default True, use Comma (,) range separator, Else False uses SPACE range separator.
VideoFilename, can supply clip name, added to Comment Header.
*/


I could hack above to just output the SOS frames and append a K.

[Boulder]

Yes, that's how it works :)

[StainlessS]

OK, try this, untested.

EDIT: SCRIPT REMOVED

EDIT: Changed it, added assert on QP_Fn, and NOT Optional.

[StainlessS]

Edit: Script removed

[Boulder]

OK, I think I've finished changing it.

Thanks, I'll give it a go as soon as I'm back from all the Christmas spiriting ;)

[StainlessS]

I've made a few more changes, kept finding mistakes.

[Boulder]

Hmm, there's still something there. Complains about this line:

RT_WriteFile(QP_Fn,"%d K", Start,Append=True) : NOP

I also had to change this line a bit since it's DBSC_ScanValidate (params needed slight changing as well).
DBSC_Validate(ScanDB,"SCAN",MustExist=True,name=myName)

After editing, I was able to run the process but it produces multiple lines per keyframe entry (probably something in the for loop).

[StainlessS]

Oops, sorry Boulder, below tested and working. [EDIT: I somehow missed your above post]


Function DBSC_WriteQPFile(String ScanDB,String QP_Fn) {
myName="DBSC_WriteQPFile: "
Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB)
DBSC_Validate(ScanDB,"SCAN",MustExist=True,name=myName)
Exist(QP_Fn) ? RT_FileDelete(QP_Fn) : NOP
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start = RT_DBaseGetField(ScanDB,n,0) # Start of scene
End = RT_DBaseGetField(ScanDB,n,1) # End of Scene
RT_WriteFile(QP_Fn,"%d K", Start,Append=True)
n=End # next time around, n is End + 1
}
return 0
}


EDIT: I would have thought that x264 scene change detection is better than DBSC. [However, you can add (or remove) scene change detect with Override.txt file" if required]

EDIT: My entire test script


#ASYNTHER DBSC_Cropper

VideoFileName ="D:\DVD\z.avi"
VideoFileName = RT_GetFullPathName(VideoFileName)
ScanDB = VideoFileName+"_Scan.DB"
LogFn = VideoFileName+"_User.Log"
SpliceDB = VideoFileName+"_Splice.DB"
TmpCropLog = RT_GetFullPathName(".\~CropTmp_"+RT_LocalTimeString+".log")

Avisource(VideoFileName)
limiter(16,235,16,240)

ORG=Last

######## Temp OUTER RoboCrop args (to remove outermost/common border)
RC_SAMPLES = Max(Round(FrameCount/(FrameRate*10.0)),100) # Sample 1 Frame every 10 seconds, or at least 100 frames
RC_IGNORE = 0.04
RC_WMOD = 2 # Width multiple of this
RC_HMOD = 2 # Height multiple of this
RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 2
RC_RIGHTSKIP = 0
RC_BOTSKIP = 2

OCROP = ORG.RoboCrop(SAMPLES=RC_SAMPLES,ignore=RC_IGNORE,WMod=RC_WMOD,HMod=RC_HMOD,
\ LeftSkip=RC_LEFTSKIP,TopSkip=RC_TOPSKIP,RightSkip=RC_RIGHTSKIP,botskip=RC_BOTSKIP,
\ LogFn=TmpCropLog,LogAppend=false,Align=True,laced=False)

ts=RT_ReadTxtFromFile(TmpCropLog) RT_FileDelete(TmpCropLog)
RC_X=RT_NumberValue(ts) ts=MidStr(ts,FindStr(ts," ")+1)
RC_Y=RT_NumberValue(ts)
RC_WR=(RC_X+OCROP.Width) -ORG.Width
RC_HR=(RC_Y+OCROP.Height)-ORG.Height
RT_DebugF("RoboCrop Outermost crop Coords = %d,%d,%d,%d (%dx%d)",RC_X,RC_Y,RC_WR,RC_HR,OCROP.Width,OCROP.Height)

# Dynamic cropped will be resized to this [or set your own]
OUTPUT_WID=OCROP.Width OUTPUT_HIT=OCROP.Height

### Visual set limits for dynamic cropping (additional to outermost cropping)
CropLimit_Left=12 CropLimit_Top=2 CropLimit_Right=12 CropLimit_Bot=2
#Return OCROP.DBSC_ShowCrop(CropLimit_Left,CropLimit_Top,CropLimit_Right,CropLimit_Bot)

### Scan for scene changes
dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
DBSC_Scan(ORG,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh,ocx=RC_X,ocy=RC_Y,ocw=OCROP.Width,och=OCROP.Height)

#DBSC_Override(ScanDB,"Override.txt") return Last

QP_FN=".\QP_fn.txt"
DBSC_WriteQPFile(ScanDB,QP_Fn)
Return Last

Function DBSC_WriteQPFile(String ScanDB,String QP_Fn) {
myName="DBSC_WriteQPFile: "
Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB)
DBSC_Validate(ScanDB,"SCAN",MustExist=True,name=myName)
Exist(QP_Fn) ? RT_FileDelete(QP_Fn) : NOP
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start = RT_DBaseGetField(ScanDB,n,0) # Start of scene
End = RT_DBaseGetField(ScanDB,n,1) # End of Scene
RT_WriteFile(QP_Fn,"%d K", Start,Append=True)
n=End # next time around, n is End + 1 # EDIT: ie End+1 = next start frame or record
}
return 0
}

You might want to change/remove those in BLUE.

[Boulder]

Thanks, got it working now. However, I believe this line "DBSC_Validate(ScanDB,"SCAN",MustExist=True,name=myName)" is from your non-released version of the script? Had to change it to match the released one.

Actually it would be better to put I instead of K to make sure the references are ended there (did that change in the script myself already). If the SCs are falsely detected, it won't do much harm anyway.

And unfortunately the scene change detection in x26x is quite flaky. It misses some very clear scene changes and the parameters don't seem to do a thing, or if they do, the results only go worse. Funny thing is that it seems that it is not possible to make it more sensitive towards scene changes :confused:

[StainlessS]

is from your non-released version
Yep, more than possible, changed quite a bit since publish.

Glad you got it working.

[Boulder]

The detection is working very well. Some notes from my tests:

Using RequestLinear after DGSource is a no-no, it seems to confuse the detection. I've checked the DebugView log but cannot find the reason why, so I've just commented it out. It's part of my standard script to avoid an MVTools issue I seem to have at times.
For HDR, I've also added DGHDRToSDR() to the chain as it improves the accuracy.
For HD/UHD sources, BilinearResizeMT(width/3, height/3) improves performance without affecting the results noticably.
Dither_luma_rebuild(s0=3.0) from Dogway's toolset also improves detection in dark frames so it's applied to all sources as the last item.
Using Prefetch doesn't seem to improve performance so it's also commented out to be sure that multithreading doesn't cause any issues. This is why I prefer BilinearResizeMT in downscaling as it gives a little boost in this case.

One more request: would it be possible to add support for using the DGIndexNV produced QP file in DBSC_Override? I tried simply concatenating the two QP files but x265 doesn't seem to respect the file contents if the list is not sorted.
It would simply mean reading the line until a space or EOL is found as the entries look like this:

0 I -1
12896 I -1
23083 I -1
31478 I -1
40279 I -1
51121 I -1
57040 I -1
75859 I -1
86462 I -1
100643 I -1
112318 I -1
124988 I -1
134089 I -1
145346 I -1
156093 I -1
169714 I -1

[StainlessS]

OOps again boulder, I did not see or forgot about your request. [too much XMAS/New Year cheer, maybe]

What does the DGIndexNV '-1' indicate [remove a scene change] ?

I'm about to go feed the ducks, but will find a solution later.

[Boulder]

OOps again boulder, I did not see or forgot about your request. [too much XMAS/New Year cheer, maybe]

What does the DGIndexNV '-1' indicate [remove a scene change] ?

I'm about to go feed the ducks, but will find a solution later.
It's like a placeholder for QP if you want to override whatever the encoder decides. It's not mandatory in the QP file and -1 is ignored since it's outside the valid range.

[StainlessS]

So, I read an integer frame number [always without + or - prefix] , and ignore everything after that,
all the entries are ADDING first frame of scenechange.
Is that it ?

If so, I can read in your QP thing file, and parse the string [via some function] producing a multi-line text string
with each line setting a new start of scene.
The file will be prefixed by "*" + NewLine, flagging to DBSC_Override that it is a command string rather than a filename of file commands.
(EDIT: Or I might actually create an Alternative Override File, without the "*\n" prefix)

Then you just call the DBSC_Override() whatsit with that string rather than the filename string.
You might call DBSC_Override() with the filename first, after processing, then call with command string [or vice versa].

Does that sound like a plan ?

EDIT: Then you call the DBSC QP file creator thing to produce the QP file as required.

EDIT: And Post you final version of the DBSC_WriteQPFile(ScanDB,QP_Fn) script function [I think you made some changes].

[Boulder]

So, I read an integer frame number [always without + or - prefix] , and ignore everything after that,
all the entries are ADDING first frame of scenechange.
Is that it ?

If so, I can read in your QP thing file, and parse the string [via some function] producing a multi-line text string
with each line setting a new start of scene.
The file will be prefixed by "*" + NewLine, flagging to DBSC_Override that it is a command string rather than a filename of file commands.
(EDIT: Or I might actually create an Alternative Override File, without the "*\n" prefix)

Then you just call the DBSC_Override() whatsit with that string rather than the filename string.
You might call DBSC_Override() with the filename first, after processing, then call with command string [or vice versa].

Does that sound like a plan ?

EDIT: Then you call the DBSC QP file creator thing to produce the QP file as required.

EDIT: And Post you final version of the DBSC_WriteQPFile(ScanDB,QP_Fn) script function [I think you made some changes].

Yes, I think that would work. I usually save the DGIndexNV-created QP files in one folder along with the project files and actual scripts. I could then use that path as a variable in the scenechange detection script and so I only need to change the base filename depending on the actual script name.

This is the default script I've used when testing:
workdir = "f:\temp\captures\"
workfil = "rearwindow"
Import(workdir+workfil+".avs")
BilinearResizeMT(width/4, height/4)
DGHDRToSDR()
dither_luma_rebuild(s0=3.0)

# Start DebugView before script to view output

VideoFileName = workfil
Start_Fn = "Start_Frames.txt"
End_Fn = "End_Frames.txt"
Range_Fn = "Range_Frames.txt"
Mid_Fn = "Mid_Frames.txt"
ClipClop_Fn = "ClipClop_Frames.txt"
InfoDB = VideoFileName+"_Info.DB" # INFO DBase
ScanDB = VideoFileName+"_Scan.DB" # SCAN DBase
RECYCLE_INFO_DB= True
RECYCLE_SCAN_DB= True

DBSC_CreateInfo(InfoDB,RecycleDB=RECYCLE_INFO_DB)
DBSC_Scan(InfoDB,ScanDB,RecycleDB=RECYCLE_SCAN_DB)
DBSC_WriteQPFile(ScanDB,VideoFileName+".txt")

Return DBSC_StackSceneChanges(Last,InfoDB,ScanDB)



The function now looks like this:
Function DBSC_WriteQPFile(String ScanDB,String QP_Fn) {
myName="DBSC_WriteQPFile: "
Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB)
DBSC_ScanValidate(ScanDB,MustExist=True,name=myName)
Exist(QP_Fn) ? RT_FileDelete(QP_Fn) : NOP
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start = RT_DBaseGetField(ScanDB,n,0) # Start of scene
End = RT_DBaseGetField(ScanDB,n,1) # End of Scene
RT_WriteFile(QP_Fn,"%d I", Start,Append=True)
n=End # next time around, n is End + 1
}
return 0
}

[StainlessS]

Thanks B, now go take a nap for a while.

[StainlessS]

OK, spent some time looking in DGIndexNV docs looking for the QP whatsit stuff, not much found.
Anyways, I just did it without much clue what it is all about and hey presto, no probs, think it works ok.
I've left in RT_Stats docs for funcs used, for your reference.
Say if you need any mods.

Function ConvDgiNvQp2Override(String DgNv_Fn,String Ovr_Fn) {
myName="ConvDgiNvQp2Override: "
Assert(DgNv_Fn!="",myName+"DgNv_Fn cannot be ''")
Assert(Ovr_Fn!="",myName+"Ovr_Fn cannot be ''")
DgNv_Fn = DgNv_Fn.RT_GetFullPathname
Ovr_Fn = Ovr_Fn.RT_GetFullPathname
Assert(DgNv_Fn.Exist,myName+"input DgNv_Fn does Not Exist")
RT_FileDelete(Ovr_Fn) # Force delete of existing output, specifiy how you want it to deal with that (if different)
IN_LINES = DgNv_Fn.RT_FileQueryLines
for(i=0,IN_LINES-1) {
iS = RT_ReadTxtFromFile(DgNv_Fn ,Lines=1,Start=i).RT_TxtGetLine # Read the n/l terminated string, and strip trailing n/l.
while(iS.RT_Ord==32 || (9 <= iS.RT_Ord <= 13)) {iS=iS.MidStr(2)} # Eat leading white
if(48 <= iS.RT_Ord <= 57) { # If remaining lead is digit, else skip line
frm = iS.RT_NumberValue
RT_WriteFile(Ovr_Fn,"%d",frm,Append=True) # write frame number of scene change start + n/l
}
}
Return 0 # Whatever
}

ConvDgiNvQp2Override(".\DgNv_Fn.txt",".\Ovr_Fn.txt")
return blankclip


__END__

RT_FileQueryLines(String)
Non-clip function.
String, Filename, Name of text file to query.
Returns number of lines in a text file. Returns 0 for a zero length file.
You can use Exist() to inquire whether a file exists or not, beforehand.


RT_ReadTxtFromFile(String ,Int "Lines"=0,Int "Start"=0)
Non-clip function.
String Filename, Name of text file to load into a string.
Lines=0=unlimited. Set to number of leading lines in text file to load, eg 1 = load only the first line of text file.
The return string is n/l ie Chr(10) separated, and carriage returns are removed from the returned string.
If source file was missing newline on very last line, it will append a newline so that all lines are similarly formatted.
v1.03, Added Start arg default=0=very first line (relative 0). Would have been nice to have start and lines in reverse
order but implemented as above to not break scripts.
Throws an error if your requested Start is >= to the number of lines in the file, or zero len file.
To fetch the last line of a text file, use eg Start = RT_FileQueryLines(Filename) - 1 (Start is zero relative).
You could eg get the last line of a d2v file which might look like this:- "FINISHED 100.00% VIDEO"


RT_TxtGetLine(String, Int "Line"=0)
Non-clip function.
Extract a single line from a multiline Newline[Chr(10)] separated string. Default=0 == first line.
The Line index is Zero Relative like frame number versus FrameCount.
The returned string has trailing Newlines and carriage returns stripped from it.
Throws an error if your requested line is >= to the number of lines in the multiline string.

RT_Ord(string,int "pos"=1)
Returns same as 'Ord' in v2.6, the ASCII code for the 1st letter in string (0 if empty string "").
v1.17, Added pos arg, default 1, ie 1st character in string. Returns 0 if pos < 1 or greater than string length.
Could possibly use a string as a sort of BYTE sized int array containing non zero values.

RT_NumberValue(String,int "base"=10,int "pos"=1)
Returns an int conversion of the supplied number base string.
Base, (10, 2 -> 36) is the number base that the string uses.
To convert a Decimal string to a Float, use Avisynth native "Value()" func.
Conversion will cease at the first non legal number base digit, without producing an error.
eg RT_NumberValue("1100",2) returns 12 from the binary string.
v1.14, Added optional pos arg default=1, start position in string of the number string, 1 denotes the string beginning. Will return 0
if error in 'pos' ie if pos is less than 1 or greater than string length.


For DgNv_Fn.txt I used in file

0 I -1
12896 I -1
23083 I -1
31478 I -1
40279 I -1
51121 I -1
57040 I -1
75859 I -1
86462 I -1
100643 I -1
112318 I -1
124988 I -1
134089 I -1
145346 I -1
156093 I -1
169714 I -1


result in Ovr_Fn.txt


0
12896
23083
31478
40279
51121
57040
75859
86462
100643
112318
124988
134089
145346
156093
169714


EDIT: Once you've processed the output file, you can probably just RT_FileDelete() it.

[Boulder]

Thanks, I'll give it a go later today. Looks like exactly what I need to create the big QP file without any manual copy stuff :)

[leoenc]

@StainlessS
Where can we find the latest full version with the QP file write function? I couldn't find it in your MediaFire folder.

[leoenc]

Ok, found it here:
https://forum.doom9.org/showthread.php?p=1963345#post1963345

[Boulder]

@StainlessS, any tips for getting the detection to work properly on a clip, which spits out a very small amount of scene changes even with these crazy settings? I got about 40 while a normal amount for this material (good old Star Trek TOS) should probably be around 300-500.

DBSC_CreateInfo(InfoDB,RecycleDB=RECYCLE_INFO_DB,Thresh=0.01)
DBSC_Scan(InfoDB,ScanDB,RecycleDB=RECYCLE_SCAN_DB,dFact=1.01,dMinim=1.0)

The DebugView log has mostly these messages.
[20056] DBSC_Scan: 72391 ] n-1->n Luma Pixel Diff Count OVERRIDE(too low) = 18.54%
[20056] DBSC_Scan: 72393 ] n->n+1 Luma Pixel Diff Count OVERRIDE(too low) = 7.72%

[StainlessS]

Chuck up a bit of a sample, maybe 250MB max.
Are the non detects blended scene changes ?

[Boulder]

Chuck up a bit of a sample, maybe 250MB max.
Are the non detects blended scene changes ?

Sure, I'll get back to this when I'm home from work. I did check some scene changes by visual inspection and they are clear and sharp ones without any of those blends familiar to old TV shows. Probably it's something that is not so easily detectable by human eyes.

[StainlessS]

My standard use args normally same for everything (easier to do manual override than screw with settings for a few mistakes).


### Scan for scene changes
dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
DBSC_Scan(ORG,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh,ocx=RC_X,ocy=RC_Y,ocw=OCROP.Width,och=OCROP.Height)

ocx etc are cropped coords (returned via RoboCrop outermost common border).

However, dont offhand know if args are all same for posted DBSC,
I was working on it a few days back, is not really in a state for publication, I'm a bit stuck on how best to do something.
Also, a border crop is good idea before hand if border.

[Boulder]

https://drive.google.com/file/d/1ZPsArfwoUW-VRR7W7Eu-ozVoRNPqzMC3/view?usp=share_link is a 2-minute sample, about 250MB. The script didn't detect a single scene change with the default settings (+ DupTh=0.01 which always works best for me).

[StainlessS]

You might wait a while for some response, Amazon just delivered a bunch of stuff for my new (2nd hand) Lenovo ThinkCentre M70q Tiny Gen 2
machine and I'm gonna wanna play with that after I get some sleep, aint been to bed last night, and its getting dark right now.

I'm a bit pissed off that M70q Tiny Gen 2 (CPU i5-11400T, pci-e 4.0 capable) is only PCI-e gen 3.0 motherboard, wasted my time/cash buying
Crucial P5 Plus PCI-e gen 4.0 nvme for it. But the SATA ssd I just had delivered should be ok.
Got a small 4 port VGA KVM with USB switching for mouse/keyboard/printer for my 2nd VGA monitor (debugging and convenience) so as to use with my
main setup, I can switch input to main monitor via the monitor. Also got a separate USB switch so as to enable flipping machine attached to external
hard drive dock, dont wanna be plugging and unplugging stuff all day long. [EDIT: Heavy duty printer style USB 3.x connector switch]

When checking out PCI-e on it, I found Speccy (from CCleaner people) was not so good with more recent machine, nor was another hardware info
prog I had so I tried this (HWINFO), I might have much older version on my main machine (I'm on the new one right now), great hardware inf prog for recent-ish machine.
https://www.hwinfo.com/ [ An overwhelming amount of info from it ]

[StainlessS]

Boulder,
I'm gettin' a HTTP 405 from your link [method not allowed] on download
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/405

[Boulder]

Boulder,
I'm gettin' a HTTP 405 from your link [method not allowed] on download
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/405

That's odd, I can download it without being logged in so it's shared..

Try this one for the direct download link. It should just say that the file is too big to do a scan for viruses.
https://drive.google.com/uc?id=1ZPsArfwoUW-VRR7W7Eu-ozVoRNPqzMC3&export=download

[StainlessS]

OK, its downloading now.
It was FreeDownloadManager that was being blocked.

[StainlessS]

Boulder, with my current DBSC, flags 16 scene changes in your clip,
havta use Trim(5,0) at script start to get rid of gray frames [due to file cutting],
False +ve for frame 1272 (bit of a pan where James T Kirk stands up after picking up some grass (@ about 52 secs).
Detects all true scene cuts, no other false +ve's.

My args in my current script.

### Scan for scene changes
dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
DBSC_Scan(ORG,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh,ocx=RC_X,ocy=RC_Y,ocw=OCROP.Width,och=OCROP.Height)


Client code used

#ASYNTHER DBSC_Cropper

#Import(".\DBSC.avs")
#Import(".\DBSC_DEMO.avs")
#Import(".\DBSC_OTHER.avs")


VideoFileName ="D:\___BLURAY\Boulder\tos_s01e01.sample.mkv.AVI"
VideoFileName = RT_GetFullPathName(VideoFileName)
ScanDB = VideoFileName+"_Scan.DB"
LogFn = VideoFileName+"_User.Log"
SpliceDB = VideoFileName+"_Splice.DB"
TmpCropLog = RT_GetFullPathName(".\~CropTmp_"+RT_LocalTimeString+".log")

Avisource(VideoFileName,Pixel_Type="YV12")
trim(5,0)
ORGORG=Last
limiter(16,235,16,240)
ORG=Last

######## Temp OUTER RoboCrop args (to remove outermost/common border)
RC_SAMPLES = Max(Round(FrameCount/(FrameRate*10.0)),100) # Sample 1 Frame every 10 seconds, or at least 100 frames
RC_IGNORE = 0.04
RC_WMOD = 2 # Width multiple of this
RC_HMOD = 2 # Height multiple of this
RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 0
RC_RIGHTSKIP = 0
RC_BOTSKIP = 0

OCROP = ORG.RoboCrop(SAMPLES=RC_SAMPLES,ignore=RC_IGNORE,WMod=RC_WMOD,HMod=RC_HMOD,
\ LeftSkip=RC_LEFTSKIP,TopSkip=RC_TOPSKIP,RightSkip=RC_RIGHTSKIP,botskip=RC_BOTSKIP,
\ LogFn=TmpCropLog,LogAppend=false,Align=True,laced=False)

ts=RT_ReadTxtFromFile(TmpCropLog) RT_FileDelete(TmpCropLog)
RC_X=RT_NumberValue(ts) ts=MidStr(ts,FindStr(ts," ")+1)
RC_Y=RT_NumberValue(ts)
RC_WR=(RC_X+OCROP.Width) -ORG.Width
RC_HR=(RC_Y+OCROP.Height)-ORG.Height
RT_DebugF("RoboCrop Outermost crop Coords = %d,%d,%d,%d (%dx%d)",RC_X,RC_Y,RC_WR,RC_HR,OCROP.Width,OCROP.Height)

# Dynamic cropped will be resized to this [or set your own]
OUTPUT_WID=OCROP.Width OUTPUT_HIT=OCROP.Height

### Visual set limits for dynamic cropping (additional to outermost cropping)
CropLimit_Left=0 CropLimit_Top=0 CropLimit_Right=0 CropLimit_Bot=0
#Return OCROP.DBSC_ShowCrop(CropLimit_Left,CropLimit_Top,CropLimit_Right,CropLimit_Bot)

### Scan for scene changes
dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
DBSC_Scan(ORG,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh,ocx=RC_X,ocy=RC_Y,ocw=OCROP.Width,och=OCROP.Height)

#Added Override to remove false +ve
OVR = """*
-1272
"""

DBSC_Override(ScanDB,OVR)

#Return DBSC_StackSceneChanges(ORG,ScanDB) # Check detections START/END of scene STACKED.

SHOW_DYNACROP_METRICS=true
RLBT=$F # 1=Top, 2=Bot, 4=Left, 8=Right. (1+2+4+8=15 = all sides)
BLANK=False # If True, DONT NOT CROP, Blank Border ONLY
# DBSC_DynaCrop MUST have been called (at some time) before DBSC_DynaSplice
DC_Clip=DBSC_DynaCrop(ORGORG,ScanDB,SpliceDB,W=OUTPUT_WID,H=OUTPUT_HIT,
\ rlbt=RLBT,CropThresh=-40.0,
\ CropLimit_Left=RC_X+CropLimit_Left,CropLimit_Top=RC_Y+CropLimit_Top,CropLimit_Right=CropLimit_Right-RC_WR,CropLimit_Bot=CropLimit_Bot-RC_HR,
\ LeftAdd=0,TopAdd=0,RightAdd=0,BotAdd=0,
\ LogFn=LogFn,Debug=True,DissolveSkip=8,Blank=BLANK,Show=SHOW_DYNACROP_METRICS)

return DC_Clip


EDIT: Added Override in Blue

EDIT: Oops, Override 1272 should have been -1272 for delete.

[Boulder]

You must have a more recent version than the v0.051, mine doesn't at least have the *corr parameters..
Is the most recent version in your Mediafire folder? Couldn't find it by a quick peek.

[StainlessS]

Nope, not there.
Can use this, but watch out for problems, think DBSC_SubsClip broken. [maybe others]
https://www.mediafire.com/file/9ez26uli200y0bo/DBSC.7z/file

EDIT:
Added the ORGORG stuff in RED, avoid return of Limiter() modded clip.
(limiter to aid in detection)

EDIT: Changed this lot to 0

CropLimit_Left=0 CropLimit_Top=0 CropLimit_Right=0 CropLimit_Bot=0


WARNING,
SHOW_DYNACROP_METRICS=true
will show metrics [if gonna take long time for scan]

[Boulder]

Cheers, this version seems to work fine. The changes were so big compared to the version I had that Beyond Compare couldn't show me anything clear, but probably it's generally a better working version nonetheless :)

I just copy-pasted these to the end to include them since I use them with the detection all the time as well.

Function DBSC_WriteQPFile(String ScanDB,String QP_Fn) {
myName="DBSC_WriteQPFile: "
Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB)
Exist(QP_Fn) ? RT_FileDelete(QP_Fn) : NOP
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start = RT_DBaseGetField(ScanDB,n,0) # Start of scene
End = RT_DBaseGetField(ScanDB,n,1) # End of Scene
RT_WriteFile(QP_Fn,"%d I", Start,Append=True)
n=End # next time around, n is End + 1
}
return 0
}

Function ConvDgiNvQp2Override(String DgNv_Fn,String Ovr_Fn) {
myName="ConvDgiNvQp2Override: "
Assert(DgNv_Fn!="",myName+"DgNv_Fn cannot be ''")
Assert(Ovr_Fn!="",myName+"Ovr_Fn cannot be ''")
DgNv_Fn = DgNv_Fn.RT_GetFullPathname
Ovr_Fn = Ovr_Fn.RT_GetFullPathname
Assert(DgNv_Fn.Exist,myName+"input DgNv_Fn does Not Exist")
RT_FileDelete(Ovr_Fn) # Force delete of existing output, specifiy how you want it to deal with that (if different)
IN_LINES = DgNv_Fn.RT_FileQueryLines
for(i=0,IN_LINES-1) {
iS = RT_ReadTxtFromFile(DgNv_Fn ,Lines=1,Start=i).RT_TxtGetLine # Read the n/l terminated string, and strip trailing n/l.
while(iS.RT_Ord==32 || (9 <= iS.RT_Ord <= 13)) {iS=iS.MidStr(2)} # Eat leading white
if(48 <= iS.RT_Ord <= 57) { # If remaining lead is digit, else skip line
frm = iS.RT_NumberValue
RT_WriteFile(Ovr_Fn,"%d",frm,Append=True) # write frame number of scene change start + n/l
}
}
Return 0 # Whatever
}

[leoenc]

@StainlessS, can this script run multithreaded?
BTW great work!

[tormento]

Cheers, this version seems to work fine
Is possible to create a similar script to find the fps changings in a video? I am not talking about variable frame rate but discrete parts, each with its own fps, such as 23.976p, 29.97p, 60i.

[StainlessS]

@StainlessS, can this script run multithreaded?
Nope, not a chance.

[StainlessS]

Is possible to create a similar script to find the fps changings in a video? I am not talking about variable frame rate but discrete parts, each with its own fps, such as 23.976p, 29.97p, 60i.

ApparentFPS is the only thing I'm aware of and that aint gonna please you.

Mobile:

EDIT: Currenly battling with this script.

[Boulder]

Is possible to create a similar script to find the fps changings in a video? I am not talking about variable frame rate but discrete parts, each with its own fps, such as 23.976p, 29.97p, 60i.

The variations in the frame rate are in the container, not in the video itself. The best bet is to use TIVTC's hybrid mode and use the log it creates in the first pass.

[johnmeyer]

Is possible to create a similar script to find the fps changings in a video? I am not talking about variable frame rate but discrete parts, each with its own fps, such as 23.976p, 29.97p, 60i.The is a tough one, although detecting the interlaced footage is trivial, since there is no temporal difference between fields in progressive footage. I think you can just bob, do a Ydiff of some sort, and when that metric goes from near zero to a much larger value for more than a few frames, you've got interlace footage.

For the 24 and 30 fps progressive, you might be able to come up with something using the logic I created in these two threads:

Automatically fix dups followed (eventually) by drops (https://forum.doom9.org/showthread.php?t=161758)

Need help "perfecting" script to delete drops and dups (https://forum.doom9.org/showthread.php?t=183351)

I never quite got the dups to balance the drops (tough problem, it turns out, so my scripts failed to achieve their goal) but the ability to detect drops (which is somewhat tough) worked pretty well.

You might be able to adapt this by using the logic to "measure" the temporal gap between frames, and then when it changes, take some action to change the frame rate.

However, having now proposed it, I don't think it could ever be made reliable enough to "set it and forget it." However you might be able to use it to output frame numbers where the script suspected a frame rate change had taken place, and then let you go investigate.

[StainlessS]

@Boulder,

Make these changes in DBSC_Scan()

CorB=RT_LumaCorrelation(Last,Last,n=n ,n2=n+1,x=ocx,y=ocy,w=ocw,h=och) # cor n <-> n+1
if(dCorr<1.0) { # Detect Blended Frame SC
BldNxt = (1.0 - CorB) * 255 # distance n <-> n+1
BldPrv = (1.0 - RT_LumaCorrelation(Last,Last,n=n ,n2=n-1,x=ocx,y=ocy,w=ocw,h=och)) *255 # distance n-1 <-> n
BldSum = bldPrv + BldNxt
Bld_Z = BldSum * 0.9 + 40.0
BldAlt = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n+1,x=ocx,y=ocy,w=ocw,h=och)) * 255.0 # Distance, Prev <-> Next, either side of current
BldpAl = (1 - RT_LumaCorrelation(Last,Last,n=n-2,n2=n ,x=ocx,y=ocy,w=ocw,h=och)) * 255.0 # Distance, either side of previous
BldnAl = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+2,x=ocx,y=ocy,w=ocw,h=och)) * 255.0 # Distance, either side of next
ZOK = (Bld_Z < BldAlt)
# distance 127.5 = corr 0.5 : distance 76.5.0 = corr 0.7
AltOK = (BldAlt > 127.5 && (BldpAl < 76.5.0 || BldnAl < 76.5.0)) # ALT must look like scenechange and at least one of prv/nxt ALTs like NOT Scene change
# AltOK helps avoid false +ve's, and nearly detects single frame blends OK all on its own.
if(ZOK && AltOK) {
SS=RT_String("%d ] BLENDED SCENE CHANGE detected, Alt=%.3f BldpAl=%3f BldnAl=%3f",n,BldAlt,BldpAl,BldnAl)
RT_DebugF("%s",SS,name=myName)
RFLAGS=$0004 # RFLGS reason to BLEND DETECT
OFLAGS=$0004 # OFLGS to BLEND DETECT OVERRIDE
RT_DBaseSetField(ScanDB,Start,1,n-1) # Set Start of scene frame/Record EOSix(End OF Scene) to previous frame.
RT_DBaseSetField(ScanDB,Start,4,RFLAGS)
RT_DBaseSetField(ScanDB,Start,5,OFLAGS)
Start = n
RT_DBaseSetField(ScanDB,Start,1,n) # Set SINGLE FRAME scene Start of scene frame/Record EOSix(End OF Scene) to current frame.
RT_DBaseSetField(ScanDB,Start,4,RFLAGS)
RT_DBaseSetField(ScanDB,Start,5,OFLAGS)
Start = n + 1 # Next frame, SOS follows this EOS
NextPossEnd = n + dMinLen # ie Next Start + dMinLen - 1
}
}

Better blended frame scenechange detect.

EDIT: Remove RED error[76.5.0 to 76.5]

[Boulder]

Thanks, will do that.

About high bitdepth - should the material be 8-bit or is the analysis ok for 10 or 16 bits as well?

[StainlessS]

If you're just generating frame numbers, suggest convert to 8 bit [RT_Stats only does 8 bit at the moment].
HBD extra bits of no real use in detecting scene change.

[StainlessS]

@Boulder, remove RED error in prev post #136.
Bad online edit. [76.5.0 to 76.5]

[StainlessS]

@StainlessS, can this script run multithreaded?

Nope, not a chance.
Prefetch(...) only applies when frameserving,
The script is mostly working during the 'compile' stage, ie before frameserving starts and that is pretty much single core and done only in scripting sequence.
Prefetch() will only apply once the clip appears in eg Vdub2, where the only stuff it will apply to are Trim(), Crop(), Spline36Resize(), and AlignedSplice(), non of which would benefit greatly from Prefetch() anyway (I dont think).

Maybe at some stage it could be turned onto a plugin, but nothing would change there, would just be scanning single threaded but in C/C++ rather than AVS script.
(script path is dependent upon results already obtained, and not really parallelizable, leastwise not in any way that I am capable of)

Some things must be done in sequencial order, eg writing AVI file must be done in correct sequence order, cannot just write in any old higgledy piggledy order or the resulting file would be corrupt.

[Boulder]

What about the fade out detection? When I was testing, I noticed that it will slow down the process greatly since the source is processed twice.

[StainlessS]

Yeah, well it needs to know the start and stop points of a scene before it can re-scan for the "Fade to black" scan changes between those points.
Its easier to do in separate pass (maybe at some point could be done in single pass after first establishing End Of Scene).

[Boulder]

Yeah, well it needs to know the start and stop points of a scene before it can re-scan for the "Fade to black" scan changes between those points.
Its easier to do in separate pass (maybe at some point could be done in single pass after first establishing End Of Scene).

Yes, in terms of caching I believe it could work much better performance-wise. It's a bit of a pain to let the thing read 80 GB of data twice. Off a USB 2.0 external HDD :devil::D

[StainlessS]

You can turn off Fade-To-Black SC detection, dFade=False.

Cheers, this version seems to work fine. The changes were so big compared to the version I had that Beyond Compare couldn't show me anything clear, but probably it's generally a better working version nonetheless :)

I just copy-pasted these to the end to include them since I use them with the detection all the time as well.

Function DBSC_WriteQPFile(String ScanDB,String QP_Fn) {
myName="DBSC_WriteQPFile: "
Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB)
Exist(QP_Fn) ? RT_FileDelete(QP_Fn) : NOP
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start = RT_DBaseGetField(ScanDB,n,0) # Start of scene
End = RT_DBaseGetField(ScanDB,n,1) # End of Scene
RT_WriteFile(QP_Fn,"%d I", Start,Append=True)
n=End # next time around, n is End + 1
}
return 0
}

Function ConvDgiNvQp2Override(String DgNv_Fn,String Ovr_Fn) {
myName="ConvDgiNvQp2Override: "
Assert(DgNv_Fn!="",myName+"DgNv_Fn cannot be ''")
Assert(Ovr_Fn!="",myName+"Ovr_Fn cannot be ''")
DgNv_Fn = DgNv_Fn.RT_GetFullPathname
Ovr_Fn = Ovr_Fn.RT_GetFullPathname
Assert(DgNv_Fn.Exist,myName+"input DgNv_Fn does Not Exist")
RT_FileDelete(Ovr_Fn) # Force delete of existing output, specifiy how you want it to deal with that (if different)
IN_LINES = DgNv_Fn.RT_FileQueryLines
for(i=0,IN_LINES-1) {
iS = RT_ReadTxtFromFile(DgNv_Fn ,Lines=1,Start=i).RT_TxtGetLine # Read the n/l terminated string, and strip trailing n/l.
while(iS.RT_Ord==32 || (9 <= iS.RT_Ord <= 13)) {iS=iS.MidStr(2)} # Eat leading white
if(48 <= iS.RT_Ord <= 57) { # If remaining lead is digit, else skip line
frm = iS.RT_NumberValue
RT_WriteFile(Ovr_Fn,"%d",frm,Append=True) # write frame number of scene change start + n/l
}
}
Return 0 # Whatever
}

I can Add DBSC_WriteQPFile(String ScanDB,String QP_Fn) to DBSC as is sorta related, however I'll leave out the DGIndexNV related one.
OR more likely, I can just add it as another option for the DBSC_WriteFiles() thingy. [ <<<<<<======== Doing this One ]

EDIT: What does QP Stand for again [I'll just call file QPFile_Fn for now].

[Boulder]

EDIT: What does QP Stand for again [I'll just call file QP_Fn for now].
The result is a QP file for x264 or x265, I guess QP stands for "Quantization Parameter".

[StainlessS]

QPFile_Fn OK ?
Or you want "QuantizationParameter_Fn" as arg name ?

Your machine not got USB 3.x ?

EDIT: I think this should be OK

Function DBSC_WriteFiles(String ScanDB,String "Start_Fn",String "End_Fn",String "Range_Fn",String "Mid_Fn",String "SingleFrameScene_Fn",String "ClipClopCmd_Fn",String "QPFile_Fn",
\ Bool "Comment",Bool "Comma",String "VideoFileName") {
/*
Write Scene info to output files:- Start, End, Mid Frame and Range, and frame numbers of single frame scenes.
Also writes a ClipClop command file for Start (clip 1) and End (clip 2) frames, used by DBSC_Select().
QPFile_Fn outputs QPFile for x264,in format "%d I", where %d is framenumber of StartOfScene. No Comment is written to QPFile_Fn file.
ScanDB, SCAN DBase file created via DBSC_Scan.
Comment, Default True writes additional Hash (#) separated info to file. False, omits extra info.
Comma, Default True, use Comma (,) range separator, Else False uses SPACE range separator.
VideoFilename, can supply clip name, added to Comment Header.
*/
/*
Field number
0 ) SOSix, Start Of Scene frame(record)
1 ) EOSix, End Of Scene frame(record)
2 ) Type(1=SOS,2=EOS,3=SOS and EOS(single frame scene),0=none)
3 ) SceneNo(Zero Rel)
4 ) RFLGS, Detection Reason code
5 ) OFLGS, Detection Overridden flags

ID's
0) DBSC_Str2ID("DBSC")
1) DBSC_Str2ID("SCAN")
2) Rand (non zero)
3) Outer_Crop_X
4) Outer_Crop_Y
5) Outer_Crop_W Width Relative
6) Outer_Crop_H Height Relative
7) Input Width
8) Input Height
9) DBSC_ColorSpaceIx
10) Framecount
11) Matrix
12) Thresh
13) ChromaWeight
14) dFact
15) dMinim
16) dMinLen
17) oCorr
18) dcorr
19) fCorr
20) fFrames
21) DupTh
22) dblend (true=1,false=0)
23) dFade (true=1,false=0)

User Attribs
-1) Number of Scene changes
-2) Number of Scene Crop changes
-3) Output clip Width (same for all DB).
-4) utput clip Height (same for all DB).

StrAttrib,
-1 Resizer String (256 chars], Same for all DB

*/
myName="DBSC_WriteFiles: " Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB) DBSC_Validate(ScanDB,"SCAN",MustExist=True,name=myName)
START_TIME = RT_TimerHP
Start_Fn=Default(Start_Fn,"") End_Fn=Default(End_Fn,"") Range_Fn=Default(Range_Fn,"")
Mid_Fn=Default(Mid_Fn,"") Single_Fn=Default(SingleFrameScene_Fn,"") QPFile_Fn=Default(QPFile_Fn,"")
ClipClopCmd_Fn=Default(ClipClopCmd_Fn,"")
Comment=Default(Comment,True) Comma=Default(Comma,True)
VideoFileName=Default(VideoFileName,"")
VideoFileName=(VideoFileName!="") ? RT_FilenameSplit(VideoFileName,12) : ""
SEP = RT_Ord((Comma) ? "," : " ")
(Start_Fn !="" && Exist(Start_Fn)) ? RT_FileDelete(Start_Fn) : NOP
(End_Fn !="" && Exist(End_Fn)) ? RT_FileDelete(End_Fn) : NOP
(Range_Fn !="" && Exist(Range_Fn)) ? RT_FileDelete(Range_Fn) : NOP
(Mid_Fn !="" && Exist(Mid_Fn)) ? RT_FileDelete(Mid_Fn) : NOP
(Single_Fn !="" && Exist(Single_Fn)) ? RT_FileDelete(Single_Fn) : NOP
(ClipClopCmd_Fn !="" && Exist(ClipClopCmd_Fn)) ? RT_FileDelete(ClipClopCmd_Fn) : NOP
(QPFile_Fn !="" && Exist(QPFile_Fn)) ? RT_FileDelete(QPFile_Fn) : NOP

(Start_Fn !="") ? RT_DebugF("Writing Start_Fn File '%s'",Start_Fn,name=myName) : NOP
(End_Fn !="") ? RT_DebugF("Writing End_Fn File '%s'", End_Fn, name=myName) : NOP
(Range_Fn !="") ? RT_DebugF("Writing Range_Fn File '%s'",Range_Fn,name=myName) : NOP
(Mid_Fn !="") ? RT_DebugF("Writing Mid_Fn File '%s'", Mid_Fn,name=myName) : NOP
(Single_Fn !="") ? RT_DebugF("Writing Single_Fn File '%s'", Single_Fn,name=myName) : NOP
(ClipClopCmd_Fn !="") ? RT_DebugF("Writing ClipClopCmd_Fn File '%s'", ClipClopCmd_Fn,name=myName) : NOP
(QPFile_Fn !="") ? RT_DebugF("Writing QPFile_Fn File '%s'",QPFile_Fn,name=myName) : NOP

if(Comment) {
Matrix = RT_DBaseGetID(ScanDB,11) Thresh = RT_DBaseGetID(ScanDB,12) ChromaWeight = RT_DBaseGetID(ScanDB,13)
dFact = RT_DBaseGetID(ScanDB,14) dMinim = RT_DBaseGetID(ScanDB,15) dMinLen = RT_DBaseGetID(ScanDB,16)
oCorr = RT_DBaseGetID(ScanDB,17) dCorr = RT_DBaseGetID(ScanDB,18)
fCorr = RT_DBaseGetID(ScanDB,19) fFrames = RT_DBaseGetID(ScanDB,20)
DupTh = RT_DBaseGetID(ScanDB,21)
dBlend = (RT_DBaseGetID(ScanDB,22)!=0)
dFade = (RT_DBaseGetID(ScanDB,23)!=0)
nAttribs= RT_DBaseGetAttrib(ScanDB,-1) OvrCnt = RT_DBaseGetAttrib(ScanDB,nAttribs-1)

BAR =RT_StrPad("",86,"#")+Chr(10)
HEAD=RT_String("#\n# DBSC: Scene Change @@@ Frames file : @ %s\n#\n",RT_LocalTimeString(File=False))
VSTR=(VideoFileName=="") ? "" : RT_String("# SourceFile:- '%s'\n#\n",VideoFileName)

ARG =RT_String(
\ "# dFact =%.2f : dMinim =%.2f : dMinLen =%d\n" +
\ "# oCorr =%.2f : dCorr =%.2f : fCorr =%.2f : fFrames = %d\n" +
\ "# DupTh =%.2f : dBlend =%s : dFade =%s\n" +
\ "# Matrix=%d : Thresh =%.2f : ChromaW=%.2f\n" +
\ "# User Overides=%d\n#\n",
\ dFact,dMinim,dMinLen,
\ oCorr,dCorr,
\ fCorr,fFrames,
\ DupTh,dBlend,dFade,
\ Matrix,Thresh,ChromaWeight,
\ OvrCnt)
LAB=RT_StrPad("# # SceneNo : Start : Mid : End : Length",85)+"#"
Z=RT_String("%s%s%s%s%s%s",BAR,HEAD,VSTR,ARG,BAR,LAB)
Z=RT_StrReplace(Z,"@@@","%s")
(Start_Fn !="") ? RT_WriteFile(Start_Fn, Z,"START" ,Append=True) : NOP
(End_Fn !="") ? RT_WriteFile(End_Fn, Z,"END" ,Append=True) : NOP
(Range_Fn !="") ? RT_WriteFile(Range_Fn, Z,"RANGE" ,Append=True) : NOP
(Mid_Fn !="") ? RT_WriteFile(Mid_Fn, Z,"MID" ,Append=True) : NOP
(Single_Fn !="") ? RT_WriteFile(Single_Fn, Z,"SINGLE" ,Append=True) : NOP
(ClipClopCmd_Fn !="") ? RT_WriteFile(ClipClopCmd_Fn,Z,"CLIPCLOP_CMD",Append=True) : NOP
}
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start=RT_DBaseGetField(ScanDB,n,0) End =RT_DBaseGetField(ScanDB,n,1) SceneNo=RT_DBaseGetField(ScanDB,n,3)+1
Mid = (Start + End)/ 2 Len = End-Start + 1
if(!Comment) {
(Start_Fn!="") ? RT_WriteFile(Start_Fn, "%d" ,Start ,Append=True) : NOP
(End_Fn!="") ? RT_WriteFile(End_Fn, "%d" ,End ,Append=True) : NOP
(Range_Fn!="") ? RT_WriteFile(Range_Fn, "%d%c%d" ,Start,SEP,End,Append=True) : NOP
(Mid_Fn!="") ? RT_WriteFile(Mid_Fn, "%d" ,Mid ,Append=True) : NOP
(Single_Fn!="" && Len==1) ? RT_WriteFile(Single_Fn, "%d" ,Start ,Append=True) : NOP
if(ClipClopCmd_Fn!="") {
RT_WriteFile(ClipClopCmd_Fn, "%d%c%d",(Start==End)?3:1,SEP,Start,Append=True)
(Start!=End) ? RT_WriteFile(ClipClopCmd_Fn, "2%c%d",SEP,End,Append=True) : NOP
}
} Else {
Com = RT_String(" # %6d : %6d : %6d : %6d : %6d",SceneNo,Start,Mid,End,Len)
(Start_Fn!="") ? RT_WriteFile(Start_Fn ,"%-13d%s", Start,Com,Append=True) : NOP
(End_Fn!="") ? RT_WriteFile(End_Fn ,"%-13d%s", End, Com,Append=True) : NOP
(Range_Fn!="") ? RT_WriteFile(Range_Fn ,"%s%s",RT_StrPad(RT_String("%d%c%d",Start,SEP,End),13),Com,Append=True) : NOP
(Mid_Fn!="") ? RT_WriteFile(Mid_Fn ,"%-13d%s", Mid, Com,Append=True) : NOP
(Single_Fn!="" && Len==1) ? RT_WriteFile(Single_Fn ,"%-13d%s", Len, Com,Append=True) : NOP
if(ClipClopCmd_Fn!="") {
RT_WriteFile(ClipClopCmd_Fn,RT_String("%s%s",RT_StrPad(RT_String("%d%c%d",(Start==End)?3:1,SEP,Start),13),com),Append=True)
(Start!=End)?RT_WriteFile(ClipClopCmd_Fn,RT_String("%s%s",RT_StrPad(RT_String("%d%c%d",2,SEP,End),13),com),Append=True):NOP
}
}
(QPFile_Fn!="") ? RT_WriteFile(QPFile_Fn, "%d I" ,Start ,Append=True) : NOP
n = End # Next iteration n = End + 1, ie Start of Next Scene
}
T = RT_TimerHP - START_TIME
RT_DebugF("Total Time taken = %.2f Seconds (%.2f Mins)",T,T/60.0,name=myName)
return 0
}

[Boulder]

QPFile_Fn OK ?
Or you want "QuantizationParameter_Fn" as arg name ?
QPFile_Fn is a good name.

Your machine not got USB 3.x ?
It does, I just have this ancient 1TB WD drive which refuses to die :D

EDIT: I think this should be OK

Function DBSC_WriteFiles(String ScanDB,String "Start_Fn",String "End_Fn",String "Range_Fn",String "Mid_Fn",String "SingleFrameScene_Fn",String "ClipClopCmd_Fn",String "QPFile_Fn",
\ Bool "Comment",Bool "Comma",String "VideoFileName") {
/*
Write Scene info to output files:- Start, End, Mid Frame and Range, and frame numbers of single frame scenes.
Also writes a ClipClop command file for Start (clip 1) and End (clip 2) frames, used by DBSC_Select().
QPFile_Fn outputs QPFile for x264,in format "%d I", where %d is framenumber of StartOfScene. No Comment is written to QPFile_Fn file.
ScanDB, SCAN DBase file created via DBSC_Scan.
Comment, Default True writes additional Hash (#) separated info to file. False, omits extra info.
Comma, Default True, use Comma (,) range separator, Else False uses SPACE range separator.
VideoFilename, can supply clip name, added to Comment Header.
*/
/*
Field number
0 ) SOSix, Start Of Scene frame(record)
1 ) EOSix, End Of Scene frame(record)
2 ) Type(1=SOS,2=EOS,3=SOS and EOS(single frame scene),0=none)
3 ) SceneNo(Zero Rel)
4 ) RFLGS, Detection Reason code
5 ) OFLGS, Detection Overridden flags

ID's
0) DBSC_Str2ID("DBSC")
1) DBSC_Str2ID("SCAN")
2) Rand (non zero)
3) Outer_Crop_X
4) Outer_Crop_Y
5) Outer_Crop_W Width Relative
6) Outer_Crop_H Height Relative
7) Input Width
8) Input Height
9) DBSC_ColorSpaceIx
10) Framecount
11) Matrix
12) Thresh
13) ChromaWeight
14) dFact
15) dMinim
16) dMinLen
17) oCorr
18) dcorr
19) fCorr
20) fFrames
21) DupTh
22) dblend (true=1,false=0)
23) dFade (true=1,false=0)

User Attribs
-1) Number of Scene changes
-2) Number of Scene Crop changes
-3) Output clip Width (same for all DB).
-4) utput clip Height (same for all DB).

StrAttrib,
-1 Resizer String (256 chars], Same for all DB

*/
myName="DBSC_WriteFiles: " Assert(ScanDB!="",RT_String("%sMUST Supply ScanDB",myName))
ScanDB = RT_GetFullPathName(ScanDB) DBSC_Validate(ScanDB,"SCAN",MustExist=True,name=myName)
START_TIME = RT_TimerHP
Start_Fn=Default(Start_Fn,"") End_Fn=Default(End_Fn,"") Range_Fn=Default(Range_Fn,"")
Mid_Fn=Default(Mid_Fn,"") Single_Fn=Default(SingleFrameScene_Fn,"") QPFile_Fn=Default(QPFile_Fn,"")
ClipClopCmd_Fn=Default(ClipClopCmd_Fn,"")
Comment=Default(Comment,True) Comma=Default(Comma,True)
VideoFileName=Default(VideoFileName,"")
VideoFileName=(VideoFileName!="") ? RT_FilenameSplit(VideoFileName,12) : ""
SEP = RT_Ord((Comma) ? "," : " ")
(Start_Fn !="" && Exist(Start_Fn)) ? RT_FileDelete(Start_Fn) : NOP
(End_Fn !="" && Exist(End_Fn)) ? RT_FileDelete(End_Fn) : NOP
(Range_Fn !="" && Exist(Range_Fn)) ? RT_FileDelete(Range_Fn) : NOP
(Mid_Fn !="" && Exist(Mid_Fn)) ? RT_FileDelete(Mid_Fn) : NOP
(Single_Fn !="" && Exist(Single_Fn)) ? RT_FileDelete(Single_Fn) : NOP
(ClipClopCmd_Fn !="" && Exist(ClipClopCmd_Fn)) ? RT_FileDelete(ClipClopCmd_Fn) : NOP
(QPFile_Fn !="" && Exist(QPFile_Fn)) ? RT_FileDelete(QPFile_Fn) : NOP

(Start_Fn !="") ? RT_DebugF("Writing Start_Fn File '%s'",Start_Fn,name=myName) : NOP
(End_Fn !="") ? RT_DebugF("Writing End_Fn File '%s'", End_Fn, name=myName) : NOP
(Range_Fn !="") ? RT_DebugF("Writing Range_Fn File '%s'",Range_Fn,name=myName) : NOP
(Mid_Fn !="") ? RT_DebugF("Writing Mid_Fn File '%s'", Mid_Fn,name=myName) : NOP
(Single_Fn !="") ? RT_DebugF("Writing Single_Fn File '%s'", Single_Fn,name=myName) : NOP
(ClipClopCmd_Fn !="") ? RT_DebugF("Writing ClipClopCmd_Fn File '%s'", ClipClopCmd_Fn,name=myName) : NOP
(QPFile_Fn !="") ? RT_DebugF("Writing QPFile_Fn File '%s'",QPFile_Fn,name=myName) : NOP

if(Comment) {
Matrix = RT_DBaseGetID(ScanDB,11) Thresh = RT_DBaseGetID(ScanDB,12) ChromaWeight = RT_DBaseGetID(ScanDB,13)
dFact = RT_DBaseGetID(ScanDB,14) dMinim = RT_DBaseGetID(ScanDB,15) dMinLen = RT_DBaseGetID(ScanDB,16)
oCorr = RT_DBaseGetID(ScanDB,17) dCorr = RT_DBaseGetID(ScanDB,18)
fCorr = RT_DBaseGetID(ScanDB,19) fFrames = RT_DBaseGetID(ScanDB,20)
DupTh = RT_DBaseGetID(ScanDB,21)
dBlend = (RT_DBaseGetID(ScanDB,22)!=0)
dFade = (RT_DBaseGetID(ScanDB,23)!=0)
nAttribs= RT_DBaseGetAttrib(ScanDB,-1) OvrCnt = RT_DBaseGetAttrib(ScanDB,nAttribs-1)

BAR =RT_StrPad("",86,"#")+Chr(10)
HEAD=RT_String("#\n# DBSC: Scene Change @@@ Frames file : @ %s\n#\n",RT_LocalTimeString(File=False))
VSTR=(VideoFileName=="") ? "" : RT_String("# SourceFile:- '%s'\n#\n",VideoFileName)

ARG =RT_String(
\ "# dFact =%.2f : dMinim =%.2f : dMinLen =%d\n" +
\ "# oCorr =%.2f : dCorr =%.2f : fCorr =%.2f : fFrames = %d\n" +
\ "# DupTh =%.2f : dBlend =%s : dFade =%s\n" +
\ "# Matrix=%d : Thresh =%.2f : ChromaW=%.2f\n" +
\ "# User Overides=%d\n#\n",
\ dFact,dMinim,dMinLen,
\ oCorr,dCorr,
\ fCorr,fFrames,
\ DupTh,dBlend,dFade,
\ Matrix,Thresh,ChromaWeight,
\ OvrCnt)
LAB=RT_StrPad("# # SceneNo : Start : Mid : End : Length",85)+"#"
Z=RT_String("%s%s%s%s%s%s",BAR,HEAD,VSTR,ARG,BAR,LAB)
Z=RT_StrReplace(Z,"@@@","%s")
(Start_Fn !="") ? RT_WriteFile(Start_Fn, Z,"START" ,Append=True) : NOP
(End_Fn !="") ? RT_WriteFile(End_Fn, Z,"END" ,Append=True) : NOP
(Range_Fn !="") ? RT_WriteFile(Range_Fn, Z,"RANGE" ,Append=True) : NOP
(Mid_Fn !="") ? RT_WriteFile(Mid_Fn, Z,"MID" ,Append=True) : NOP
(Single_Fn !="") ? RT_WriteFile(Single_Fn, Z,"SINGLE" ,Append=True) : NOP
(ClipClopCmd_Fn !="") ? RT_WriteFile(ClipClopCmd_Fn,Z,"CLIPCLOP_CMD",Append=True) : NOP
}
for(n=0,RT_DBaseRecords(ScanDB)-1) {
Start=RT_DBaseGetField(ScanDB,n,0) End =RT_DBaseGetField(ScanDB,n,1) SceneNo=RT_DBaseGetField(ScanDB,n,3)+1
Mid = (Start + End)/ 2 Len = End-Start + 1
if(!Comment) {
(Start_Fn!="") ? RT_WriteFile(Start_Fn, "%d" ,Start ,Append=True) : NOP
(End_Fn!="") ? RT_WriteFile(End_Fn, "%d" ,End ,Append=True) : NOP
(Range_Fn!="") ? RT_WriteFile(Range_Fn, "%d%c%d" ,Start,SEP,End,Append=True) : NOP
(Mid_Fn!="") ? RT_WriteFile(Mid_Fn, "%d" ,Mid ,Append=True) : NOP
(Single_Fn!="" && Len==1) ? RT_WriteFile(Single_Fn, "%d" ,Start ,Append=True) : NOP
if(ClipClopCmd_Fn!="") {
RT_WriteFile(ClipClopCmd_Fn, "%d%c%d",(Start==End)?3:1,SEP,Start,Append=True)
(Start!=End) ? RT_WriteFile(ClipClopCmd_Fn, "2%c%d",SEP,End,Append=True) : NOP
}
} Else {
Com = RT_String(" # %6d : %6d : %6d : %6d : %6d",SceneNo,Start,Mid,End,Len)
(Start_Fn!="") ? RT_WriteFile(Start_Fn ,"%-13d%s", Start,Com,Append=True) : NOP
(End_Fn!="") ? RT_WriteFile(End_Fn ,"%-13d%s", End, Com,Append=True) : NOP
(Range_Fn!="") ? RT_WriteFile(Range_Fn ,"%s%s",RT_StrPad(RT_String("%d%c%d",Start,SEP,End),13),Com,Append=True) : NOP
(Mid_Fn!="") ? RT_WriteFile(Mid_Fn ,"%-13d%s", Mid, Com,Append=True) : NOP
(Single_Fn!="" && Len==1) ? RT_WriteFile(Single_Fn ,"%-13d%s", Len, Com,Append=True) : NOP
if(ClipClopCmd_Fn!="") {
RT_WriteFile(ClipClopCmd_Fn,RT_String("%s%s",RT_StrPad(RT_String("%d%c%d",(Start==End)?3:1,SEP,Start),13),com),Append=True)
(Start!=End)?RT_WriteFile(ClipClopCmd_Fn,RT_String("%s%s",RT_StrPad(RT_String("%d%c%d",2,SEP,End),13),com),Append=True):NOP
}
}
(QPFile_Fn!="") ? RT_WriteFile(QPFile_Fn, "%d I" ,Start ,Append=True) : NOP
n = End # Next iteration n = End + 1, ie Start of Next Scene
}
T = RT_TimerHP - START_TIME
RT_DebugF("Total Time taken = %.2f Seconds (%.2f Mins)",T,T/60.0,name=myName)
return 0
}

Thanks, I'll test it soonish :)

[Boulder]

Does the dFact parameter work properly? I've now tested a couple of runs with Star Trek season 1 episode 2, and setting it to 2.0 or 5.0 had no effect, setting to 20.0 created three(!) scene changes more while I thought it would reduce them. Parameters for testing were "ScanDB,RecycleDB=RECYCLE_SCAN_DB,DEBUG=true,dFact=20.0,dMinim=8.0,dMinLen=5,oCorr=0.5,dCorr=0.55,fCorr=0.6,fFrames=4,DupTh=0.0,dFade=False,dBlend=True". The low corr values are for testing, in this case it skipped only two scene changes for the whole ~50 min video.

[StainlessS]

Does the dFact parameter work properly?

Dont know, maybe :)
I may have made some embedded changes. I suggest just use defaults and Overrides if necessary.
Much easier to set Override than screw around forever trying to tweak for gazillions of different scene changes.
(Also, if eg 4K stuff, then maybe best to downsize to eg 720x576, UHD not necessary for SC detect. HBD definitely not necessary for SC detect.)

Anyways, I've decided to post this, I've separated into separate DBSC_SCAN.avs, for scan ONLY, no Dynamic crop stuff.
Any Metrics shown, is not worth taking notice of, much changed since implemented, but at least it dont throw error now.
(EDIT: Any changes in scan args or methods, and I gotta change stuff in multiple other places in script too, a real pain for 2000 {well 1859} line script, especially for metrics)

https://www.mediafire.com/file/ppatohs09e5r6nj/DBSC_SCAN_v0.09_18Nov2022.7z/file

Included test script

Import(".\DBSC_SCAN.avs") # SCAN FOR SCENE CHANGES ONLY : NO DYNAMIC BORDER CROPPING STUFF.

VideoFileName =".\TEST.avi" # EDIT: OOoops fixed.
VideoFileName = RT_GetFullPathName(VideoFileName)
ScanDB = VideoFileName+"_Scan.DB"
TmpCropLog = RT_GetFullPathName(".\~CropTmp_"+RT_LocalTimeString+".log")

Avisource(VideoFileName)
ORG=Last

######## Temp OUTER RoboCrop args (to remove outermost/common border)
RC_SAMPLES = Max(Round(FrameCount/(FrameRate*10.0)),100) # Sample 1 Frame every 10 seconds, or at least 100 frames
RC_IGNORE = 0.04
RC_WMOD = 2 # Width multiple of this
RC_HMOD = 2 # Height multiple of this
RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 0
RC_RIGHTSKIP = 0
RC_BOTSKIP = 0

OCROP = ORG.RoboCrop(SAMPLES=RC_SAMPLES,ignore=RC_IGNORE,WMod=RC_WMOD,HMod=RC_HMOD,
\ LeftSkip=RC_LEFTSKIP,TopSkip=RC_TOPSKIP,RightSkip=RC_RIGHTSKIP,botskip=RC_BOTSKIP,
\ LogFn=TmpCropLog,LogAppend=false,Align=True,laced=False)

ts=RT_ReadTxtFromFile(TmpCropLog) RT_FileDelete(TmpCropLog)

RC_X=RT_NumberValue(ts) ts=MidStr(ts,FindStr(ts," ")+1)
RC_Y=RT_NumberValue(ts)
RC_WR=(RC_X+OCROP.Width) -ORG.Width # Right relative Width
RC_HR=(RC_Y+OCROP.Height)-ORG.Height # Bot relative Height
RT_DebugF("RoboCrop Outermost crop Coords = %d,%d,%d,%d (%dx%d)",RC_X,RC_Y,RC_WR,RC_HR,OCROP.Width,OCROP.Height)


### Scan for scene changes
dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
DBSC_Scan(ORG,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh,ocx=RC_X,ocy=RC_Y,ocw=OCROP.Width,och=OCROP.Height,debug=True)

# Overrides in String (first char is '*') rather than a filename. (StartOfSCene FrameNos, -frameNo for remove StartOfScene).
OVR="""*
"""

#DBSC_Override(ScanDB,OVR)

OCROP

START_FN = "START_FN.TXT"
END_FN = ""
RANGE_FN = ""
MID_FN = ""
SINGLEFRAMESCENE_FN = ""
CLIPCLOPCMD_FN = ""
QPFILE_FN = "QPFILE_FN.TXT"
COMMENT = True
COMMA = True

#DBSC_WriteFiles(ScanDB,Start_Fn=Start_fn,End_Fn=End_Fn,Range_Fn=Range_Fn,Mid_Fn=Mid_Fn,SingleFrameScene_Fn=SingleFrameScene_Fn,ClipClopCmd_Fn=ClipClopCmd_Fn,QPFile_Fn=QPFile_Fn,Comment=Comment,Comma=Comma,VideoFileName=VideoFileName)


SZ=64 # Subtitle Size
#Return DBSC_Select(Last,ScanDB,Subtitle("START",Size=SZ,Text_Color=$FFFF00,Align=5,Y=Height/2 - SZ),Subtitle("END",Size=SZ,Text_Color=$0000FF,Align=5,Y=Height/2 + SZ),Subtitle("SINGLE",Size=SZ*2,Text_Color=$FF0000,Y=Height/2+SZ/2,Align=5))

#Return DBSC_StackSceneChanges(ORG,ScanDB,Metrics=False) # Check detections START/END of scene STACKED.
#Return DBSC_StackSceneChanges(ORG,ScanDB,Metrics=True) # Check detections START/END of scene STACKED.


SUBS = True
METRICS = True
METRICS2 = True # Only with METRICS=TRUE Else Does nuttin.

Return DBSC_SubsClip(ORG,ScanDB,Subs=SUBS,Metrics=METRICS,Metrics2=METRICS2)


Return Last


EDIT: I'll try to do FadeToBlack together in single pass.

EDIT: And header from DBSC_SCAN

# Use GImport("DBSC_SCAN.avs")

/*
DBSC v0.09, 18 Nov 2022 by StainlessS @ Doom9

A Scene Change Detection Tool set/kit/box. (Work In Progress - guinea pigs required)

Currently Implemented: (to some degree):-

Scenechange detection:- IMPLEMENTED IN DBSC_SCAN.avs

DBSC_Scan(), Creates a SCAN DBase containing Scene Change detection data. Returns Int 0.
Single/Multiple Flash frame tolerant. Fade-To-Black and single Blended frame Scene cuts detected.

DBSC_Override(), Using SCAN DBase and Overrides.txt, allows user to set additional Scene Change or remove erroneous ones
from the SCAN DBase. Additional overrides set/unset without re-creating ScanDB. Returns Int 0.

DBSC_WriteFiles(), Using SCAN DBase, Writes Start, End, Mid, And/Or Range Files for Scene Changes.
Also writes a ClipClop command file for Start (ClipClop clip 1) and End (clip 2) frames. Also writes single frame scene cut frames.
Used by DBSC_Select().

DBSC_SubsClip(), Returns clip optionally subtitled (metrics And/Or START OF SCENE / END OF SCENE / SINGLE FRAME SCENE subtitles)

DBSC_StackSceneChanges(), Returns clip, End frame of a scene together with Stacked Start frame of following scene. (Check detections).
Uses DBSC_WriteFiles() and DBSC_SubsClip().

DBSC_Select(), Similar-ish to SCSelect.
Uses DBSC_WriteFiles.


Dynamically Changing Borders:- IMPLEMENTED IN DBSC_DYNAMIC.avs

DBSC_ShowCrop(clip c,Int "Lft",Int "Top",Int "Rgt", Int "Bot"), GUI to best set CropLimit_Left/Right/Top/Bot.
Call with commom border already cropped by RoboCrop(), and Args for CropLimit_Left/Right/Top/Bot.


DBSC_DynaCrop(), Detect Dyanamically changing borders using ScanDB and create a SpliceDB for DBSC_DynaSplice().
Returns trimmed/cropped/resized/spliced clip, with optional metrics.

DBSC_DynaSplice(), Trim, Crop, Resize and Splice, dynamically changing borders. Option to call user defined function
on each trim, after it has been cropped, and before resized and splicing back together again.


Dynamically UTILITY:- IMPLEMENTED IN DBSC_UTILITY.avs
DBSC_AutoContrast(), Function to DEMO a User Function as called by DBSC_DynaSplice().
DBSC_ProcessByDAR, User Function to select processing of a scene based on Image area DAR, Requires ClipClop.

TODO:-
Improve detection.
+ whatver else I (or you) can think of.

Required:-
Frame Accurate seekable source clip (Ideally via AviSource, and NOT XVid/DivX).
RT_Stats v2.0, http://forum.doom9.org/showthread.php?t=165479
GSCript, http://forum.doom9.org/showthread.php?t=147846 (if not AVS+)
Grunt, http://forum.doom9.org/showthread.php?t=139337
FrameSel, http://forum.doom9.org/showthread.php?t=167971
ClipClop, http://forum.doom9.org/showthread.php?t=162266
RoboCrop http://forum.doom9.org/showthread.php?t=168053
DebugView http://technet.microsoft.com/en-gb/sysinternals/bb896647.aspx (viewing Debug Output in real time)
*/

[StainlessS]

(Also, if eg 4K stuff, then maybe best to downsize to eg 720x576, UHD not necessary for SC detect. HBD definitely not necessary for SC detect.)

Example how to use when DownScale detect clip. [REPOSTED]


Import(".\DBSC_SCAN.avs") # SCAN FOR SCENE CHANGES ONLY : NO DYNAMIC BORDER CROPPING STUFF.

VideoFileName =".\TEST.avi"
VideoFileName = RT_GetFullPathName(VideoFileName)
ScanDB = VideoFileName+"_Scan.DB"
TmpCropLog = RT_GetFullPathName(".\~CropTmp_"+RT_LocalTimeString+".log")

AviSource(VideoFileName)
SRC = Last

DOWNSIZE = True # Demo use with downsized detect clip.
DOWNSIZE_W = 480
DOWNSIZE_H = 360

(DOWNSIZE) ? SRC.BilinearResize(DOWNSIZE_W,DOWNSIZE_H) : SRC # Maybe avoid sharpening/ringing downsizer.

DC = Last # Detect Clip

######## Temp OUTER RoboCrop args (to remove outermost/common border)
RC_SAMPLES = Max(Round(FrameCount/(FrameRate*10.0)),100) # Sample 1 Frame every 10 seconds, or at least 100 frames
RC_IGNORE = (100.0 / 2048) # Ignore at most 1 in 2048 noisy outlier pixels. About 0.048%
RC_WMOD = 2 # Width multiple of this
RC_HMOD = 2 # Height multiple of this
RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 0
RC_RIGHTSKIP = 0
RC_BOTSKIP = 0


PRECROP_DC = DC # REMEMBER Original Pre-Cropped Size
DC = DC.RoboCrop(SAMPLES=RC_SAMPLES,ignore=RC_IGNORE,WMod=RC_WMOD,HMod=RC_HMOD,
\ LeftSkip=RC_LEFTSKIP,TopSkip=RC_TOPSKIP,RightSkip=RC_RIGHTSKIP,botskip=RC_BOTSKIP,
\ LogFn=TmpCropLog,LogAppend=false,Align=True,laced=False)

ts=RT_ReadTxtFromFile(TmpCropLog) RT_FileDelete(TmpCropLog)

RC_X=RT_NumberValue(ts) ts=MidStr(ts,FindStr(ts," ")+1)
RC_Y=RT_NumberValue(ts)
RC_WR=(RC_X+DC.Width) -PRECROP_DC.Width # Right relative Width
RC_HR=(RC_Y+DC.Height)-PRECROP_DC.Height # Bot relative Height
RT_DebugF("RoboCrop Outermost crop Coords = %d,%d,%d,%d (%dx%d) {Original=%dx%d}",RC_X,RC_Y,RC_WR,RC_HR,DC.Width,DC.Height,PRECROP_DC.Width,PRECROP_DC.Height)


### Scan for scene changes
dFact=1.5 dMinim=8.0 dMinLen=12 oCorr=0.85 dCorr=0.60 fCorr=0.75 fFrames=8 DupTh=0.5 dFade=True
DBSC_Scan(DC,ScanDB,dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,oCorr=oCorr,dCorr=dCorr,fCorr=fCorr,fFrames=fFrames,DupTh=DupTh,ocx=RC_X,ocy=RC_Y,ocw=DC.Width,och=DC.Height,debug=True)

# Overrides in String (first char is '*') rather than a filename. (StartOfSCene FrameNos, -frameNo for remove StartOfScene).
OVR="""*

"""

#DBSC_Override(ScanDB,OVR)



START_FN = "START_FN.TXT"
END_FN = ""
RANGE_FN = ""
MID_FN = ""
SINGLEFRAMESCENE_FN = ""
CLIPCLOPCMD_FN = ""
QPFILE_FN = "QPFILE_FN.TXT"
COMMENT = True
COMMA = True

#DBSC_WriteFiles(ScanDB,Start_Fn=Start_fn,End_Fn=End_Fn,Range_Fn=Range_Fn,Mid_Fn=Mid_Fn,SingleFrameScene_Fn=SingleFrameScene_Fn,ClipClopCmd_Fn=ClipClopCmd_Fn,QPFile_Fn=QPFile_Fn,Comment=Comment,Comma=Comma,VideoFileName=VideoFileName)



#Return DC.DBSC_StackSceneChanges(ScanDB,Metrics=False) # Check detections START/END of scene STACKED. Currently, must Use Detect Clip.
#Return DC.DBSC_StackSceneChanges(ScanDB,Metrics=True) # Check detections START/END of scene STACKED. Currently, Must Use Detect Clip.



SUBS = True
METRICS = True
METRICS2 = True # Only with METRICS=TRUE Else Does nuttin.
Return DC.DBSC_SubsClip(ScanDB,dClip=DC,Subs=SUBS,Metrics=METRICS,Metrics2=METRICS2) # Input clip either SRC or DC.


SRC # Allow use of Implicit Last, for input clip, Subtitle(), and Height.

# DBSC_Select(), Input clip FrameCount MUST be same as DC detect clip, no other similarities enforced.
SZ=64 # Subtitle Size
#Return Last.DBSC_Select(ScanDB,Subtitle("START",Size=SZ,Text_Color=$FFFF00,Align=5,Y=Height/2 - SZ),Subtitle("END",Size=SZ,Text_Color=$0000FF,Align=5,Y=Height/2 + SZ),Subtitle("SINGLE",Size=SZ*2,Text_Color=$FF0000,Y=Height/2+SZ/2,Align=5))

Return Last

Seems not too bad downsizing PAL DVD size down to 480x360. Could do same for HBD -> 8 Bit. [Only supports 8 bit]

If you use VDub2 as viewer, when in AVS script editor {Tools Menu} can use CTRL/P to insert current frameNo into the OVR string to add SceneChange Override.
I guess that there is similar in AvsPMod.

[Boulder]

I ran into another strange problem..

If I run this sample clip with cropping like this, I get the error "RT_DBaseSetField: Error, DB(-1:1) : Invalid record -1(0 -> 1175)". Commenting the cropping part out, it works fine. I'd just like to clean the borders because the area is nasty, the DVDs of the show were mastered from very old tapes.

DGSource("C:\Temp\hammer_snip.dgi")
Crop(16,16,-16,-16)
Bob()

https://drive.google.com/file/d/1XK9aUgf26om3MQvd6Z9irBdqDUTTeRfn/view?usp=share_link

EDIT: a longer clip seems to crash even without the cropping.

[StainlessS]

Invalid record -1(0 -> 1175)
Looks to me to accessing record(for frameNo) -1 (where is 1176 frames, 0 - 1175).

I'm actually doing a re-write-ish kinda thing at them moment, we'll check if it still does that when done.

Also, probably a moderate boost in detection, as I have implemented an additional outer 'unsafe region', inside of which
we scan for scene change [additional to border crop stuff], to avoid scanning area with a tiny bit of border, or them nasty little
lines on TV style caps [half line of black at top and bottom, especially bad after BOB()], they mess badly with correlation detection
and also blended scene change detect.

EDIT: Below how it is currently working client (although I'm changing DBSC_Scan stuff and might havta change below too].
It will not (the client) work with any script you have but you can see how it will roughly be implemented.

Import(".\DBSC_SCAN.avs") # SCAN FOR SCENE CHANGES ONLY : NO DYNAMIC BORDER CROPPING STUFF.

VideoFileName =".\TEST.avi"

VideoFileName = RT_GetFullPathName(VideoFileName)
ScanDB = VideoFileName+"_Scan.DB"
TmpCropLog = RT_GetFullPathName(".\~CropTmp_"+RT_LocalTimeString+".log")

AviSource(VideoFileName,Pixel_type="YV12")
Trim(38840-20,-10000)
#KillAudio
SRC = Last

DOWNSIZE = False # Demo use with downsized detect clip.
DOWNSIZE_W = 480
DOWNSIZE_H = 360

#(DOWNSIZE) ? BilinearResize(DOWNSIZE_W,DOWNSIZE_H) : SRC # Maybe avoid sharpening/ringing downsizer.
(DOWNSIZE) ? Spline36Resize(DOWNSIZE_W,DOWNSIZE_H) : SRC # Maybe avoid sharpening/ringing downsizer.

DC = Last # Detect Clip

######## Temp OUTER QueryBorderCrop args (to find outermost/common border)

QBC_SAMPLES = Max(Round(FrameCount/(FrameRate*10.0)),100) # [Max(Round(FrameCount/(FrameRate*10.0)),100)] # Sample 1 Frame every 10 seconds, or at least 100 frames {Function default is 40}
QBC_IGNORE = (100.0 / 2048) # [(100.0 / 2048)] # Ignore at most 1 in 2048 noisy outlier pixels. About 0.048%. {Function Default is 0.4].
QBC_WMOD = 2 # [ 2] Width multiple of this
QBC_HMOD = 2 # [ 2] Height multiple of this
QBC_LEFTSKIP = 0 # [ 0] Borders at least these (maybe ignore crud around edges)
QBC_TOPSKIP = 0 # [ 0]
QBC_RIGHTSKIP = 0 # [ 0]
QBC_BOTSKIP = 0 # [ 0]
QBC_MODE = 2 # [ 2] 0=CropExact=Dont Use : 1=CropLess=Dont Usec(crop a bit Less to comply with CS X/Y/W/H-Mod)
# 2=CropMore=DO USE{crop a bit MORE to comply with CS X/Y/W/H-Mod} : 3=CropPlus=Maybe Use{CropExact+2.ThenAsCropMore}

SAFE_X = 12 # # [ 12] # Dont Scan near border when trying to find Scene Changes. Should be at least thickness of Dynamic border (in addition to common outer border).
SAFE_Y = 12 # # [ 12] # Suggest these all at least 4 (for clean inner image, and more if changing border widths)
SAFE_W = 12 # # [ 12] # Even a tiny bit of edge crud makes Single frame Blend scene changes much harder to detect, also craps up other detects too.
SAFE_H = 12 # # [ 12] # You want *** TOTALLY CLEAN IMAGE, NO BORDERS AT ALL ***

### Get common (ie Outermost) crop coords (but dont actually crop) ###
### Results in QBCROPX, QBCROPY, QBCROPW, QBCROPH, variables. ###
### Some RT_QueryBorderCrop arguments default. ###
DC.RT_QueryBorderCrop(SAMPLES=QBC_SAMPLES,ignore=QBC_IGNORE,
\ WMod=QBC_WMOD,HMod=QBC_HMOD,
\ LeftSkip=QBC_LEFTSKIP,TopSkip=QBC_TOPSKIP,RightSkip=QBC_RIGHTSKIP,botskip=QBC_BOTSKIP,
\ laced=False,Relative=True)
\ .RT_TxtGetLine(QBC_MODE).RT_StrReplace(Select(QBC_MODE,"","L","M","P"),"").Eval # Set variables QBCropX, QBCropY, QBCropW, QBCropH, cropped coords according to QBC_MODE chosen.

### Use DebugView to see realtime debug info:- https://learn.microsoft.com/en-us/sysinternals/downloads/debugview ###
RT_DebugF("QbCropX=%d QbCropY=%d QbCropW=%d QbCropH=%d (%dx%d) {Original=%dx%d}",QbCropX,QbCropY,QbCropW,QbCropH,DC.Width-QbCropX+QbCropW,DC.Height-QbCropY+QbCropH,DC.Width,DC.Height)
#Return DC.Crop(QBCropX+SAFE_X,QBCropY+SAFE_Y,QBCropW-SAFE_W,QBCropH-SAFE_H) # CHECK clean IMAGE without any borders AT ALL.


### Scan for scene changes #### ( defaults in [...] )
dFact = 1.5 # [ 1.5]
dMinim = 8.0 # [ 8.0]
dMinLen = 6 # [ 6]
oCorr = 0.85 # [ 0.85]
dCorr = 0.60 # [ 0.60]
fCorr = 0.75 # [ 0.75]
fFrames = 8 # [ 8]
DupTh = 0.0 # [ 0.0]
dBlend = True # [ TRUE]
dFade = True # [ TRUE]
Matrix = (Width>1100||Height>600)?3:2 # [(Width>1100||Height>600)?3:2] # 2=PC.601, 3=PC.709
Thresh = 0.1 # [ 0.1]
ChromaWeight = 1.0/6 # [1.0/6] # About 1.0/5 OR 1.0/6, Seems better than RT_FrameDifference default of 1.0/3. [Use 0.0 on GreyScale]
ProjectName = RT_FilenameSplit(VideoFileName,get=4) # [RT_FilenameSplit(VideoFileName,get=4)] Get=4 Name Only, 12=Name+Ext # "DBSC_Project" Is function default.
RecycleDB = True # [ TRUE]
DEBUG = True # [FALSE]
VERB = False # [FALSE] # Verbose (only if DEBUG)

DC.DBSC_Scan(
\ ScanDB,
\ ocx=QbCropX,ocy=QbCropY,ocw=QbCropW,och=QbCropH,
\ dFact=dFact,dMinim=dMinim,dMinLen=dMinLen,
\ oCorr=oCorr,dCorr=dCorr,
\ fCorr=fCorr,fFrames=fframes,
\ DupTh=DupTh,dBlend=dBlend,dFade=dFade,
\ matrix=Matrix,thresh=Thresh,chromaWeight=ChromaWeight,
\ projectName=ProjectName,
\ safe_X=SAFE_X,safe_y=SAFE_Y,safe_w=SAFE_W,safe_h=SAFE_H,
\ RecycleDB=RecycleDB,
\ debug=DEBUG,
\ verb=Verb)

return last
### Overrides in String (first char is '*') rather than a filename. (StartOfSCene FrameNos, -frameNo for remove StartOfScene).
OVR="""*

"""

#DBSC_Override(ScanDB,OVR)

### WriteFiles ###
START_FN = "START_FN.TXT"
END_FN = ""
RANGE_FN = ""
MID_FN = ""
SINGLEFRAMESCENE_FN = ""
CLIPCLOPCMD_FN = ""
QPFILE_FN = "QPFILE_FN.TXT"
COMMENT = True
COMMA = True

#DBSC_WriteFiles(ScanDB,Start_Fn=Start_fn,End_Fn=End_Fn,Range_Fn=Range_Fn,Mid_Fn=Mid_Fn,SingleFrameScene_Fn=SingleFrameScene_Fn,ClipClopCmd_Fn=ClipClopCmd_Fn,QPFile_Fn=QPFile_Fn,Comment=Comment,Comma=Comma,VideoFileName=VideoFileName)



#Return DC.DBSC_StackSceneChanges(ScanDB,Metrics=False) # Check detections START/END of scene STACKED. Currently, must Use Detect Clip.
#Return DC.DBSC_StackSceneChanges(ScanDB,Metrics=True) # Check detections START/END of scene STACKED. Currently, Must Use Detect Clip.


SUBS = True
METRICS = True
METRICS2 = True # Only with METRICS=TRUE Else Does nuttin.
Return DC.DBSC_SubsClip(ScanDB,dClip=DC,Subs=SUBS,Metrics=METRICS,Metrics2=METRICS2) # Input clip either SRC or DC.


SRC # Allow use of Implicit Last, for input clip, Subtitle(), and Height.

### DBSC_Select(), Input clip FrameCount MUST be same as DC detect clip, no other similarities enforced.
SZ=64 # Subtitle Size
#Return Last.DBSC_Select(ScanDB,Subtitle("START",Size=SZ,Text_Color=$FFFF00,Align=5,Y=Height/2 - SZ),Subtitle("END",Size=SZ,Text_Color=$0000FF,Align=5,Y=Height/2 + SZ),Subtitle("SINGLE",Size=SZ*2,Text_Color=$FF0000,Y=Height/2+SZ/2,Align=5))

Return Last


RoboCrop is gone in the client, replaced by RT_QueryBorderCrop().