DetSub - W.I.P

[StainlessS]

v0.00
Work In Progress.

Post #1 of 6.

DetSub, purpose to detect hard coded subtitles with halo.

Here, import script to set common arguments.

DetSub_Args.avs

# DetSub_Args.avs
# This just allows set exact same args eg coords in the different scripts


###
# DetSub.
# Req Avs+, RT_Stats v2.00 Beta 02, GScript, Grunt, MaskTools v2.0. RemoveGain or RGTools.
# RT_Stats v2.00 Beta2 first posted here:- http://forum.doom9.org/showthread.php?p=1782028#post1782028
# If CallCmd plugin installed then will auto delete Temp DBase's on clip closure.
###


# Sample from here:- https://forum.doom9.org/showthread.php?p=1949702#post1949702
HARDFN="hard sub - 01 WEBdlRip 720p, 23.976.mkv.AVI" # converted to AVI via ffmpeg [or Vdub2]
HARDFN=HARDFN.RT_GetFullPathName

AVISource(HARDFN)
killaudio

# Subtitle coords # Need some border around area for allow expand of masks.
X = 44
Y = Height-132
W = -X
H = -2
# Text, Halo color

TEXTCOL = $F3F3F3 # Approx RGB color of text
TEXTTOL = $0E0E0E # Per channel text tolerance
TEXTTOLT = 0 # Additional text Tweak Tolerance, added to each individual channel of TEXTTOL [+/-].

HALOCOL = $102A37 # Approx RGB color of halo
HALOTOL = $121212 # Per channel halo tolerance
HALOTOLT = 0 # Additional halo Tweak Tolerance, added to each individual channel of TEXTTOL [+/-].

# Text pixel width,height
TEXT_W = 4 # minimum text pixel 'thickness' Horizontal (horizontal thickness of vertical strokes)
TEXT_H = 4 # minimum text pixel 'thickness' Vertical (vertical thickness of horizontal strokes)
# Halo pixel width,height
HALO_L = 3 # Left Halo width in pixels
HALO_R = 3 # Right Halo width in pixels
HALO_T = 3 # Top Halo height in pixels
HALO_B = 3 # Bot Halo height in pixels

MATRIX = "Rec601"
##################
FrameDB = "FrameDB.DB" # If Named, then FrameDB not deleted on exit. (Maybe DBase used by other scripts later)
OverRide = "" # ""=Not used, "*..." multiline string of ranges, "OverRide.txt"=Text file of ranges.
#OverRide="OverRide.txt"

/*
OverRide="""* # <<=== NOTE Asterisk (STAR). Example String OverRide (1st char is '*'). Ranges OverRidden to NO SUBTITLE
10 100 # A comment, SPACE or COMMA Separator
200,300 # 200 to 300. NOTE -ve frame count NOT supported.
3000,0 # 3000 till last frame.
"""
*/


DetSub as 7z:- https://www.mediafire.com/file/wsqsho09oiu8fxw/DetSub_WIP_v0.0.7z/file
All scripts in folder and also IMAGE folder included inside.
All you need is the sample in above script, resaved as avi via ffmpeg or Vdub2, and away you go.

[StainlessS]

Post #2 of 6

some additional routines

DetSub_Sundry.avs

# DetSub_Sundry.avs

Function Hit_Marker(int W, Int H, Bool "Hit", Int "BW", Int "BH",Int "PW",Int "PH",Bool "YV12",Bool "Mod2") {
/*
Req RT_Stats & mt_tools v2. http://forum.doom9.org/showthread.php?t=174527
Creates a marker mask WxH, for use with Overlay as Mask arg.
Intended for use via some kind of detector to show detection Hit, or Miss.
Returns Single frame clip with null audio, the marker top left hand side is aligned to 0,0.
Default return clip colorspace is Y8 for Avisynth v2.6 and above, or YV12 if defunct version.
Args:-
W,H, Size of marker mask (can be odd), return clip dimensions are rounded up to next multiple of 2 if YV12 return clip.
Bare minimum usable size is 4x4 with defaulted BW, BH, PW, PH.
HIT, Default False, returns marker as angled corners (Thickness set by BW and BH).
True, returns marker as angled corners + outer perimeter where perimeter thickness set by PW and PH.
BW, Default Max(W/8,1). Horizontal thickness of the marker corners.
BH, Default Max(H/8,1). Vertical thickness of the marker corners.
PW, Default Max(BW/8,1). Horizontal thickness of the vertical perimeter.
PH, Default Max(BH/8,1). Vertical thickness of the horizontal perimeter.
YV12, Default False if Avisynth version 2.6 or greater(Y8), else true(YV12). Selects return clip colorspace.
Mod2, Default True. If Y8 then round odd dimensions up modulo 2 (the mask is still WxH).

Example:-
Import("Hit_Marker.avs")
WW=128 HH=WW HIT=True
BlankClip
Yell=Last.BlankClip(Width=WW,Height=HH,Length=1,Color=$FFFF00)
Return OverLay(Yell,x=(Width-WW)/2,y=(Height-HH)/2,mask=Hit_Marker(WW,HH,HIT))
*/
Hit=Default(hit,False) BW=Max(Default(BW,W/8),1) BH=Max(Default(BH,H/8),1)
BW2=(W>4)?BW*2:BW BH2=(H>4)?BH*2:BH
PW=Max(Default(PW,BW/8),1) PH=Max(Default(PH,BH/8),1) is26 = VersionNumber>=2.6
YV12=(!is26) ? True : Default(YV12,False) Mod2=Default(Mod2,True) Mod = (YV12||Mod2) ? 2 : 1
CanvasW=(W+Mod-1)/Mod*Mod CanvasH=(H+Mod-1)/Mod*Mod
Rpn= RT_String("x %d < x %d >= | y %d < | y %d >= | x %d < x %d >= | & y %d < y %d >= | & ",BW,W-BW,BH,H-BH,BW2,W-BW2,BH2,H-BH2)
Rpn= (Hit) ? Rpn + RT_String("x %d < x %d >= | y %d < | y %d >= | | ",PW,W-PW,PH,H-PH) : Rpn
Rpn= Rpn + RT_String("x %d < y %d < & & 255 0 ?",W,H)
Blankclip(width=CanvasW,height=CanvasH,Length=1,pixel_type=YV12?"YV12":"Y8").Killaudio
return mt_lutspa(relative = false,yExpr=Rpn, chroma = "-128")
}


Function ChrIsStar(String S) {return RT_Ord(S)==42}
Function ChrIsNul(String S) {return RT_Ord(S)== 0} # End of String
Function ChrIsHash(String S) {return RT_Ord(S)==35} # #
Function ChrIsDigit(String S) {C=RT_Ord(S) return C>=48&&C<=57}
Function ChrEatWhite(String S) {i=1 C=RT_Ord(S,i) While(C==32||C>=8&&C<=13) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}
Function ChrEatDigits(String S) {i=1 C=RT_Ord(S,i) While(C>=48&&C<=57) {i=i+1 C=RT_Ord(S,i)} return i>1?MidStr(S,i):S}

Function DetSub_Override(string Override,string DB) {
if(ChrIsStar(Override)) {OverRide=MidStr(OverRide,2)
} else {
Assert(Exist(OverRide),RT_String("%sOverRide File does not exist",myName))
OverRide=RT_ReadTxtFromFile(Override)
}
LINES=RT_TxtQueryLines(OverRide)
for(i=0,LINES-1) {
SS=ChrEatWhite(RT_TxtGetLine(Override,i))
S=SS
if(!ChrIsNul(S) && !ChrIsHash(S)) {
if(ChrIsDigit(S)) {
StartF = RT_NumberValue(S)
EndF=StartF
S=ChrEatWhite(ChrEatDigits(S))
if(ChrIsComma(S)){S=MidStr(S,2) S=ChrEatWhite(S)}
if(ChrIsDigit(S)) {
EndF=RT_NumberValue(S)
S=ChrEatDigits(S)
if(EndF==0) { EndF=FrameCount-1}
Assert(StartF<=EndF,RT_String("%sError StartFrame(%d) > EndFrame(%d)",myName,StartF,EndF))
}
for(n=StartF,EndF) {RT_DBaseSetField(DB,n,0,3)}
}
S=ChrEatWhite(S)
if(ChrIsHash(S)) {S=""}
Assert(ChrIsNul(S),RT_String("%s *** NON-PARSE *** Error in Override Line %d\n'%s'",myName,i+1,SS))
}
}
return 0
}


Function DetSub_AvgAll2(clip c) { c.FrameCount() <= 1 ? c : Merge(DetSub_AvgAll2(c.SelectEven()), DetSub_AvgAll2(c.SelectOdd())) }

Function DetSub_AvgAll(clip c,int "Th",int "ExpandCnt") {
Th = Min(Max(Default(Th,254),0),254)
ExpandCnt = Min(Max(Default(ExpandCnt,0),0),16)
c2=c.DetSub_AvgAll2.mt_binarize(threshold=Th)
if(ExpandCnt>0) { c2=c2.mt_Expand(mode=mt_circle(ExpandCnt)) }
return c2
}

Function Frame2SMPTE(int Frameno,float Fps,int "digits",bool "File") { # ssS Mod of fn by tsp:- https://forum.doom9.org/showthread.php?p=551000#post551000
ch = Default(File,false) ?"_":":"
digits = Default(digits, int(log(max(Fps,1))/log(10))+1) # Log10 v2.60+, [Log10(n) = Log(n) / Log(10)]
hh=int(Frameno/3600/Fps) mm=int((Frameno-hh*3600*Fps)/60/Fps)
ss=int((Frameno-hh*3600*Fps-mm*60*Fps)/Fps) ff=int((Frameno-hh*3600*Fps-mm*60*Fps-ss*Fps))
# return String(hh,"%02.0f")+ch+string(mm,"%02.0f")+ch+string(ss,"%02.0f")+ch+string(ff,"%0"+string(digits)+".0f")
return RT_String("%02d%s%02d%s%02d%s%0*d",hh,ch,mm,ch,ss,ch,digits,ff)
}

[StainlessS]

Post #3 of 6

Initial detector script,

# DetSub_MaskClip.avs

/*
Function Prototype:

DetSub_MaskClip(clip c,Int "X"=-10,Int "Y"=Height-120,Int "W"=-X,Int "H"=-8,
\ Int "Text_w"=4,Int "Text_h"=4,Int "TextCol"=$FFFFFF,Int "TextTol"=$0F0F0F,Int "TextTolT"=0,
\ Int "Halo_L"=3,Int "Halo_T"=3,Int "Halo_R"=3,Int "Halo_B"=3,Int "HaloCol"=$000000,Int "HaloTol"=$0F0F0F,Int "HaloTolT"=0,
\ Int "ClipIx"=-1,string "Matrix"="rec601",string "Override"="",String "FrameDB"="",Bool "DB_ReUse"=False)

Args:-

X,Y,W,H, [defaults X=-10, Y=Height-120, W=-X. H=-8]. Coords to search for subtitles, subtitle Detect Area.

Text_W, default 4, Width of text verticals.
Text_H, default 4, Height of text horizontals.

TextCol, default $FFFFFF, Color of Subtitle Text in RGB.
TextTol, default $0F0F0F, RGB per channel tolerance for text.
TextTolT, default 0, Addtional text tolerance tweaker. Added to each of the TextTol R, G, or B channel tolerances. [+/-]

Halo_L, default 3, Width of Left halo verticals.
Halo_T, default 3, Height of Top halo horizontals.
Halo_R, default 3, Width of Right halo verticals.
Halo_B, default 3, Height of Bottom halo horizontals.

HaloCol, default $000000, Color of Subtitle Halo in RGB.
HaloTol, default $0F0F0F, RGB per channel tolerance for Halo.
HaloTolT, default 0, Addtional Halo tolerance tweaker. Added to each of the HaloTol R, G, or B channel tolerances. [+/-]

ClipIx, Default -1 [SubsArea, full detect info clip]. Selects [and returns] from 16 subtitle masks, -1 -> 14.
-1) = Full detect Info clip, same width as input clip with all Masks stacked vertically [color/tolerance tweaking].
0->14) = Other mask clip, dimensions X,Y,W,H.
ClipIx select which sub clip to return, [DetSub, output always RGB32 with Alpha = known state $FF (opaque)]
Use ClipX=-1, to aid in selecting coords, colors, and tolerances.

Matrix, default "rec601", for src conversion to RGB32.
Override, default "" [Not used]. Allows for ignoring ranges where subtitles detected but not existing.
""=Not used, "*..." multiline string of ranges, "OverRide.txt"=Text file of ranges.

OverRide="""* # <<=== NOTE Asterisk (STAR). Example String OverRide (1st char is '*'). Ranges OverRidden to NO SUBTITLE
10 100 # A comment, SPACE or COMMA Separator
200,300 # 200 to 300. NOTE -ve frame count NOT supported.
3000,0 # 3000 till last frame.
"""
FrameDB, default "". If "" then the detect subtitles per frame DBase is private, and deleted on Function Exit.
If provided with a DBase filename, then DBase will be created and will be available to calling routine on return.
FrameDB DBase Fields:-
0) Status, 0=Unknown (unvisited), 1=Sub, 2 Not sub, 3 User OverRide to NoSub
1) X # Coords relative the Subtitles X,Y,W,H Detect Area
2) Y
3) W
4) H
DB_ReUse, default False. If True, AND FrameDB supplied, then will use the already created DBase [avoid recreation], otherwise creates new DBase new.

*/
Function DetSub_MaskClip(clip c,Int "X",Int "Y",Int "W",Int "H",
\ Int "Text_w",Int "Text_h",Int "TextCol",Int "TextTol",Int "TextTolT",
\ Int "Halo_L",Int "Halo_T",Int "Halo_R",Int "Halo_B",Int "HaloCol",Int "HaloTol",Int "HaloTolT",
\ Int "ClipIx",string "Matrix",string "Override",String "FrameDB",Bool "DB_reUse") {
myName="DetSub_MaskClip: "
c
IsPlus=FindStr(VersionString,"AviSynth+")!=0
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(IsPlus||RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
Assert(RT_FunctionExist("mt_expand"),myName+"Essential MaskTools v2 plugin installed, http://forum.doom9.org/showthread.php?t=98985")
Assert(RT_FunctionExist("RemoveGrain"),myName+"Essential RemoveGrain installed.")
X=Default(X,10) Y=Default(Y,Height-120) W=Default(W,-X) H=Default(H,-8)
W=(W<=0)?Width-X+W:W H=(H<=0)?Height-Y+H:H
Text_w=Default(Text_w,3) Text_h=Default(Text_h,3)
TextCol=Default(TextCol,$FCFCFC) TextTol=Default(TextTol,$030303) TextTolT=Default(TextTolT,0)
Halo_L=Default(Halo_L,3) Halo_T=Default(Halo_T,3) Halo_R=Default(Halo_R,3) Halo_B=Default(Halo_B,3)
HaloCol=Default(HaloCol,$030303) HaloTol=Default(HaloTol,$030303) HaloTolT=Default(HaloTolT,0)
Text_Tol_B=Min(Max(0,RT_BitAnd(TextTol,$FF)+TextTolT),255) TextTol=RT_BitLSR(TextTol,8)
Text_Tol_G=Min(Max(0,RT_BitAnd(TextTol,$FF)+TextTolT),255) TextTol=RT_BitLSR(TextTol,8)
Text_Tol_R=Min(Max(0,RT_BitAnd(TextTol,$FF)+TextTolT),255)
Halo_Tol_B=Min(Max(0,RT_BitAnd(HaloTol,$FF)+HaloTolT),255) HaloTol=RT_BitLSR(HaloTol,8)
Halo_Tol_G=Min(Max(0,RT_BitAnd(HaloTol,$FF)+HaloTolT),255) HaloTol=RT_BitLSR(HaloTol,8)
Halo_Tol_R=Min(Max(0,RT_BitAnd(HaloTol,$FF)+HaloTolT),255)
ClipIx = Default(ClipIx,-1)
Assert(-1 <= ClipIx <=14,MyName+"-1 <= ClipIx <=14")
Matrix = Default(Matrix,"Rec601")
Override=Default(OverRide,"")
FrameDB=Default(FrameDB,"") UserDB=(FrameDB!="")
FrameDB=(!UserDB) ? "~DetSub_Mask_FrameDB_"+RT_LocalTimeString+".DB" : FrameDB
FrameDB=RT_GetFullPathName(FrameDB)
DB_ReUse = (UserDB) ? Default(DB_ReUse,False) : False
HasCallCmd = RT_FunctionExist("CallCmd")

###
# Mt_Hysteresis (reduce stray pixels):- http://forum.doom9.org/showthread.php?p=1780589#post1780589
# Text Mask Mt_Hysteresis inpand count (prior to mt_hysteresis), Dont Mt_Hysteresis if 0
THYSCNT = Min(TEXT_W,TEXT_H)/2 - 1

# Halo Mask Mt_Hysteresis inpand count (prior to mt_hysteresis), Dont Mt_Hysteresis if 0
HHYSCNT = Min(HALO_L,HALO_R,HALO_T,HALO_B)/2 - 1

# Detect clip Mt_Hysteresis inpand count (prior to mt_hysteresis), Dont Mt_Hysteresis if 0
DHYSCNT = Min(TEXT_W+HALO_L+HALO_R,TEXT_H+HALO_T+HALO_B)/2 - 1

###### Subs detect position
THRESH_W = -(Text_w+(Halo_L+Halo_R)/2) # -ve, specifies Abs(Pixel width) {Else +ve = Percent width 0.0->100.0%) {0.0=any pixel in range)
THRESH_H = -(Text_h+(Halo_T+Halo_B)/2) # -ve, specifies Abs(Pixel Height) {Else +ve = Percent width 0.0->100.0%) {0.0=any pixel in range)
BAFFLE_W = Text_w+Halo_L+Halo_R # Width of outermost Vertical edges.
BAFFLE_H = Text_h+Halo_T+Halo_B # Height of outermost Horizontal edges.
RESCAN = true # Switching on might get rid of the odd stray pixel (most likely not).
###
VW=Width VH=Height
/* DBase Fields
0) Status, 0=Unknown (unvisited), 1=Sub, 2 Not sub, 3 User OverRide
1) X # Coords relative the Subtitles X,Y,W,H Detect Area
2) Y
3) W
4) H
*/
if(!DB_ReUse) {
RT_DBaseAlloc(FrameDB,FrameCount,"iiiii")
RT_DBaseSetID(FrameDB,0,c.FrameCount,c.FrameRate,VW,VH,X,Y,W,H)
if(OverRide!="") {DetSet_Override(Override,FrameDB)}
}

FullWidSubC=ConvertToRGB32(matrix=Matrix).crop(0,Y,0,H) # Full width Subs area

Area=FullWidSubC.Crop(X,0,W,0).ResetMask # Valid Subs Area only, Reset Alpha to White
Text1=Area.ColorKeyMask(TextCol,Text_Tol_R,Text_Tol_G,Text_Tol_B).ShowAlpha(Pixel_Type="Y8") # Set TextCol in range pixels to black
Halo1=Area.ColorKeyMask(HaloCol,Halo_Tol_R,Halo_Tol_G,Halo_Tol_B).ShowAlpha(Pixel_Type="Y8") # Set HaloCol in range pixels to black
Text1=Text1.Invert # Text In-range pixels white, else black
Halo1=Halo1.Invert # Halo In-range pixels white, else black

Text2 = Text1.RemoveGrain(2).RemoveGrain(1)
Halo2 = Halo1.RemoveGrain(2).RemoveGrain(1)

Text3 = Text2
# Swell Text by Halo thickness [Including Halo Drop Shadow]
HALO_MAX=Max(HALO_L,HALO_T,HALO_R,HALO_B)
for(i=1,HALO_MAX) {
HorFlg=(i<=HALO_L?1:0)+(i<=HALO_R?2:0) VerFlg=(i<=HALO_T?1:0)+(i<=HALO_B?2:0)
if(HorFlg+VerFlg==6) {Text3=Text3.Mt_Expand(mode="square")
} else {
if(HorFlg==3) {Text3=Text3.Mt_Expand(mode="horizontal") HorFlg=0}
else if(VerFlg==3) {Text3=Text3.Mt_Expand(mode="vertical") VerFlg=0}
if(HorFlg!=0) {Text3=Text3.Mt_Expand(mode=HorFlg==1? " 1 0 0 0" : " -1 0 0 0")} # ? Expand_Left : Expand_Right
if(VerFlg!=0) {Text3=Text3.Mt_Expand(mode=VerFlg==1? " 0 1 0 0" : " 0 -1 0 0")} # ? Expand_Up : Expand_Down
}
}

Text4=Text2
Text5=Text3
# reduce Text strays a little
if(THysCNT>0) {
For(i=1,THysCNT) {Text4=Text4.MT_Inpand(mode="both")}
Text5=Text4.MT_Hysteresis(Text3)
}

Halo3 = Halo2
# Swell/close Halo by Text thickness
TEXT_MAX=Max(TEXT_W,TEXT_H)
for(i=1,TEXT_MAX) {
HorFlg=(i<=TEXT_W?1:0) VerFlg=(i<=TEXT_H?1:0)
if(HorFlg+VerFlg==2) {Halo3=Halo3.Mt_Expand(mode="square")
} else {
if(HorFlg!=0) {Halo3=Halo3.Mt_Expand(mode="horizontal") }
if(VerFlg!=0) {Halo3=Halo3.Mt_Expand(mode="vertical") }
}
}
Halo3=Halo3.RemoveGrain(1)
# contract again, we dont want halo outside of real halo
for(i=1,TEXT_MAX) {
HorFlg=(i<=TEXT_W?1:0) VerFlg=(i<=TEXT_H?1:0)
if(HorFlg+VerFlg==2) {Halo3=Halo3.Mt_Inpand(mode="square")
} else {
if(HorFlg!=0) {Halo3=Halo3.Mt_Inpand(mode="horizontal") }
if(VerFlg!=0) {Halo3=Halo3.Mt_Inpand(mode="vertical") }
}
}


Halo4=Halo2
Halo5=Halo3
# reduce Halo strays a little
if(HHysCNT>0) {
For(i=1,HHysCNT) {Halo4=Halo4.MT_Inpand(mode="both")}
Halo5=Halo4.MT_Hysteresis(Halo3)
}

Detect1=Mt_Logic(Text5,Halo5,"and")
Detect2=Detect1
Detect3=Detect2
Ocr1=Text2
if(DHysCNT>0) {
For(i=1,DHysCNT) {Detect2=Detect2.MT_Inpand(mode="both")}
Detect3=Detect2.MT_Hysteresis(Detect1)
Ocr1=Detect2.MT_Hysteresis(Mt_Logic(Text2,Detect3,"and"))
}

Grn=Detect3.BlankClip(Pixel_Type="RGB32",color=$00FF00)
Ocr2=Grn.Overlay(Grn.BlankClip(color=$000000),Mask=Detect3.RemoveGrain(2)).Overlay(Ocr1,Mask=Ocr1)

SSS1="""
n = current_frame
Status = RT_DBaseGetField(FrameDB,n,0)
if(Status==0) { # Unknown Status
Status = (Detect3.RT_YInRangeLocate(Baffle_w=Baffle_w,Baffle_h=Baffle_h,lo=255,hi=255,Thresh_w=Thresh_w,Thresh_h=Thresh_h,ReScan=ReScan)) ? 1 : 2
if(Status==1) { RT_DBaseSet(FrameDB,n,Status,YIRL_X,YIRL_Y,YIRL_W,YIRL_H) }
else { RT_DBaseSetField(FrameDB,n,0,Status) }
}
if(Status==1 && ClipIx<0) { # Overlay Detected area in purple, only for ClipIx -1
SUB_X = RT_DBaseGetField(FrameDB,n,1) SUB_Y = RT_DBaseGetField(FrameDB,n,2)
SUB_W = RT_DBaseGetField(FrameDB,n,3) SUB_H = RT_DBaseGetField(FrameDB,n,4)
OverLay(Last.BlankClip(color=$FF00FF,width=SUB_W,height=SUB_H),x=X+SUB_X,y=SUB_Y,opacity=0.25)
}
Return Last
"""
if(ClipIx<0) {
FullWidSubC=FullWidSubC.GScriptClip(SSS1, local=true, args="Detect3,FrameDB,Baffle_w,Baffle_h,Thresh_w,Thresh_h,ReScan,X,ClipIx",after_frame=true)
Mrk=FullWidSubC.BlankClip(Length=1,Width=W,Height=H,Color=$FFFF0000)
Msk=Hit_Marker(W,H,True,8,8,1,1)
FullWidSubC=FullWidSubC.Overlay(Mrk,Mask=Msk,x=X,y=0,Opacity=0.5)
FullWidSubC=FullWidSubC.AddBorders(0,22,0,0,$F0F080).Subtitle("Full Width Source (Search area in Red, detected subs in purple)",y=2)
Ocr2_T=Ocr2.AddBorders(X,0,Width-X-W,0,$00FF00).AddBorders(0,22,0,0,$F0F080).Subtitle("0] OCR2 : [GreenC.Overlay(Blackness.Mask=Detect3.RemoveGrain(2)).Overlay(Ocr1,Mask=Ocr1)]",y=2)
Ocr1_T=Ocr1.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("1] OCR1 : [Detect2.MT_Hysteresis(Mt_Logic(Text2,Detect3,'and'))]",y=2)
Detect3_T=Detect3.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("2] Detect3 : [Detect2.MT_Hysteresis(Detect1), reduce stray pixels]",y=2)
Detect2_T=Detect2.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle(RT_String("3] Detect2 : [Detect1 Inpand by DHysCNT(%d))]",DHysCNT),y=2)
Detect1_T=Detect1.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("4] Detect1 : Text5 AND Halo5",y=2)
Text5_T=Text5.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("5] Text5 : [Text4.MT_Hysteresis(Text3), reduce stray pixels]",y=2)
Text4_T=Text4.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle(RT_String("6] Text4 : [Text2 Inpand by THysCNT(%d)]",THysCNT),y=2)
Text3_T=Text3.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle(RT_String("7] Text3 : [Swell Text2 by Halo thickness(%d,%d,%d,%d)]",HALO_L,HALO_T,HALO_R,HALO_B),y=2)
Text2_T=Text2.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("8] Text2 : [UnDot, Text1.RemoveGrain(2).RemoveGrain(1)]",y=2)
Text1_T=Text1.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("9] Text1 : [TextInRange via TEXTCOL & TEXTTOL]",y=2)
Halo5_T=Halo5.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("10] Halo5 : [Halo4.MT_Hysteresis(Halo3), reduce stray pixels]",y=2)
Halo4_T=Halo4.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle(RT_string("11] Halo4 : [Halo2 Inpand by HHysCNT(%d)]",HHysCNT),y=2)
Halo3_T=Halo3.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle(RT_String("12] Halo3 : [Close Halo2, Expand.RemoveGrain(1).Inpand by Text thickness(%d,%d))]",TEXT_W,TEXT_H),y=2)
Halo2_T=Halo2.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("13] Halo2 : [UnDot, Halo1.RemoveGrain(2).RemoveGrain(1)]",y=2)
Halo1_T=Halo1.ConvertToRGB32.AddBorders(X,0,Width-X-W,0,$000000).AddBorders(0,22,0,0,$F0F080).Subtitle("14] Halo1 : [HaloInRange via HALOCOL & HALOTOL]",y=2)
StackVertical(FullWidSubC,Ocr2_T,Ocr1_T,Detect3_T,Detect2_T,Detect1_T,Text5_T,Text4_T,Text3_T,Text2_T,Text1_T,Halo5_T,Halo4_T,Halo3_T,Halo2_T,Halo1_T)
} else {
ClipIx==0?Ocr2:ClipIx==1?Ocr1:ClipIx==2?Detect3:ClipIx==3?Detect2:ClipIx==4?Detect1:ClipIx==5?Text5:ClipIx==6?Text4:
\ ClipIx==7?Text3:ClipIx==8?Text2:ClipIx==9?Text1:ClipIx==10?Halo5:ClipIx==11?Halo4:ClipIx==12?Halo3:ClipIx==13?Halo2:Halo1
Last.GScriptClip(SSS1, local=true, args="Detect3,FrameDB,Baffle_w,Baffle_h,Thresh_w,Thresh_h,ReScan,X,ClipIx",after_frame=true)
}
(!UserDB && HasCallCmd)?CallCmd(close=RT_String("""CMD /C chcp 1252 && del "%s" """,FrameDB), hide=true, Synchronous=7):NOP # Auto delete non-User DB file on clip closure.
Return ConvertToRGB32(Matrix="PC601").ResetMask # Alpha = $FF
}

[StainlessS]

Post #4 of 6

Client To view all masks and results


# DetSub_MaskClip_Client.avs

Import(".\DetSub_Sundry.avs")
Import(".\DetSub_MaskClip.avs")
Import(".\DetSub_Args.avs")

############################
ORG=Last

ClipIx = -1 # Full metrics clip
FRAMEDB = Undefined # We dont need it in this script, will auto delete itself [is defined in DetSub_Args.avs]

DetSub_MaskClip(x=X,y=Y,w=W,h=H,
\ Text_w=TEXT_W,Text_h=TEXT_H,TextCol=TEXTCOL,TextTol=TEXTTOL,TextTolT=TextTolT,
\ Halo_L=HALO_L,Halo_T=HALO_T,Halo_R=HALO_R,Halo_B=HALO_B,HaloCol=HALOCOL,HaloTol=HALOTOL,HaloTolT=HaloTolT,
\ clipIx=ClipIx,Matrix=MATRIX,Override=Override,framedb=FRAMEDB)

return last

[StainlessS]

Post 5 of 6

extract subs images to IMAGE directory [Must Exist].


# DetSub_Extract.avs
/*
DetSub_Extract(), extracts subtitle images from video clip.

Function Prototype:

DetSub_Extract(clip c,Int "X"=-10,Int "Y"=Height-120,Int "W"=-X,Int "H"=-8,
\ Int "Text_w"=4,Int "Text_h"=4,Int "TextCol"=$FFFFFF,Int "TextTol"=$0F0F0F,Int "TextTolT"=0,
\ Int "Halo_L"=3,Int "Halo_T"=3,Int "Halo_R"=3,Int "Halo_B"=3,Int "HaloCol"=$000000,Int "HaloTol"=$0F0F0F,Int "HaloTolT"=0,
\ Int "ClipIx"=1,string "Matrix"="rec601",string "Override"="",String "RangeDB"=""
\ )

Args:-

X,Y,W,H, [defaults X=-10, Y=Height-120, W=-X. H=-8]. Coords to search for subtitles, subtitle Detect Area.

Text_W, default 4, Width of text verticals.
Text_H, default 4, Height of text horizontals.

TextCol, default $FFFFFF, Color of Subtitle Text in RGB.
TextTol, default $0F0F0F, RGB per channel tolerance for text.
TextTolT, default 0, Addtional text tolerance tweaker. Added to each of the TextTol R, G, or B channel tolerances. [+/-]

Halo_L, default 3, Width of Left halo verticals.
Halo_T, default 3, Height of Top halo horizontals.
Halo_R, default 3, Width of Right halo verticals.
Halo_B, default 3, Height of Bottom halo horizontals.

HaloCol, default $000000, Color of Subtitle Halo in RGB.
HaloTol, default $0F0F0F, RGB per channel tolerance for Halo.
HaloTolT, default 0, Addtional Halo tolerance tweaker. Added to each of the HaloTol R, G, or B channel tolerances. [+/-]

ClipIx, Default 1, OCR1 [Selects mask for output, 0 -> 2 ONLY. Mask clip, dimensions X,Y,W,H.
ClipIx selects which subs images to extract as images to ImageDir.
0 = OCR2, Green background multi-mask with text in white, halo in black and other guys in green.
1 = OCR1, Text in White.
2 = Detect3, Text + Halo in White.

Matrix, default "rec601", for src conversion to RGB32.
Override, default "" [Not used]. Allows for ignoring ranges where subtitles detected but not existing.
""=Not used, "*..." multiline string of ranges, "OverRide.txt"=Text file of ranges.

OverRide="""* # <<=== NOTE Asterisk (STAR). Example String OverRide (1st char is '*'). Ranges OverRidden to NO SUBTITLE
10 100 # A comment, SPACE or COMMA Separator
200,300 # 200 to 300. NOTE -ve frame count NOT supported.
3000,0 # 3000 till last frame.
"""
RangeDB, default "". If "" then the subtitles ranges DBase is private, and deleted on Function Exit.
If provided with a DBase filename, then DBase will be created and will be available to calling routine on return.
RangeDB DBase Fields:-
0) Start frame of subtitle for current record subtitle range.
1) End frame of subtitle for current record subtitle range.

ImageDir, Default "", ie invalid. MUST be set to existing directory, ".\" is current directory.
Directory whe output images are to be written.

CorrTh, default 0.5, [range 0.45 -> 0.55]. Correlations threshold to detect where subtitles in adjacent frames are the same,
used to detect where to split apart non similar subs. Probably not need change.

Ocr1AveTh, default 254. [0 -> 254]. Applied to binarize ClipIx==1 OCR1 mask, ie text.
Threshold applied after average/blending all SubsArea regions of a subtitle.
0, If any pre-average pixel is above 0, then result is 255(white). [ AvgAll2.mt_binarize(threshold=0) ]
254, If ALL pre-average pixels are above 254, then result is 255(white). [ AvgAll2.mt_binarize(threshold=254) ]
or 1 -> 253, somewhere between. [the pre-average pixels are always either 0 or 255].
Det3AveTh, default 254. [0 -> 254]. Applied to binarize ClipIx==2 Detect3 mask, ie Holo+Text combined.
Threshold applied after average/blending all SubsArea regions of a subtitle.
0, If any pre-average pixel is above 0, then result is 255(white). [ AvgAll2.mt_binarize(threshold=0) ]
254, If ALL pre-average pixels are above 254, then result is 255(white). [ AvgAll2.mt_binarize(threshold=254) ]
or 1 -> 253, somewhere between. [the pre-average pixels are always either 0 or 255].
Where ClipIx==0, ie the OCR2 green Multi-Mask, Det3AveTh is using in binarizing the black Text+Halo combined area
and the Ocr1AveTh used on the foreground white Text only area.

ExpandCnt, Default 0, [0 -> 16, silent limit at 16]. There should be enough clear area around detected subs in SubsArea to Expand.
ClipIx 1, ie OCR1, applies to OCR1 Text. Eg, c.mt_Expand(mode=mt_circle(ExpandCnt)
ClipIx 2, ie Detect3, applies to Detect3 Text+halo combined mask.
ClipIx 0, ie OCR2, applied first to Text+halo combined black mask, and then to OCR1 Text white mask.

ImgType, Default, "bmp". ["bmp", "png", "tif", "tiff"]. Select Image type to write.

Smpte, Default false.
False outputs style "SUB_000000.BMP". where 000000 is first subtitle group range in clip. [record 0 if DBase returned]
True outputs SMPTE style filenames with start and end times, eg 00_01_03_013__00_01_06_010.png

*/


Function DetSub_Extract(clip c,Int "X",Int "Y",Int "W",Int "H",
\ Int "Text_w",Int "Text_h",Int "TextCol",Int "TextTol",Int "TextTolT",
\ Int "Halo_L",Int "Halo_T",Int "Halo_R",Int "Halo_B",Int "HaloCol",Int "HaloTol",Int "HaloTolT",
\ Int "ClipIx",string "Matrix",string "Override",String "RangeDB",String "ImageDir",Float "CorrTh",
\ Int "Ocr1AveTh",Int "Det3AveTh",int "ExpandCnt",
\ String "ImgType",Bool "Smpte"
\ ) {
myName="DetSub_Extract: "
c
IsPlus=FindStr(VersionString,"AviSynth+")!=0
HasCallCmd = RT_FunctionExist("CallCmd")
Assert(RT_FunctionExist("GScriptClip"),myName+"Essential GRunT plugin installed, http://forum.doom9.org/showthread.php?t=139337")
Assert(IsPlus||RT_FunctionExist("GScript"),myName+"Essential GScript plugin installed, http://forum.doom9.org/showthread.php?t=147846")
Assert(RT_FunctionExist("mt_expand"),myName+"Essential MaskTools v2 plugin installed, http://forum.doom9.org/showthread.php?t=98985")
Assert(RT_FunctionExist("RemoveGrain"),myName+"Essential RemoveGrain installed.")
X=Default(X,10) Y=Default(Y,Height-120) W=Default(W,-X) H=Default(H,-8)
W=(W<=0)?Width-X+W:W H=(H<=0)?Height-Y+H:H
ClipIx = Default(ClipIx,1) # Default OCR1
Assert(0 <= ClipIx <=2 ,MyName+"0 <= ClipIx <= 2")
Matrix = Default(Matrix,"Rec601")
Override=Default(OverRide,"")
RangeDB=Default(RangeDB,"") UserDB=(RangeDB!="")
RangeDB=(!UserDB) ? "~DetSub_Extract_RangeDB_"+RT_LocalTimeString+".DB" : RangeDB
RangeDB=RT_GetFullPathName(RangeDB)
ImageDir = Default(ImageDir,"") # ImageDir for Images # Must Exist
Assert(ImageDir!="",myName+"ImageDir Cannot be '', and must Exist."+Chr(10)+"Can use '.\' for current directory")
ImageDir=RT_GetFullPathname(ImageDir)
ImageDir=ImageDir.RevStr
while(ImageDir.FindStr("\")==1 || ImageDir.FindStr("/")==1) { ImageDir=ImageDir.MidStr(2)}
ImageDir=ImageDir.RevStr
RT_DebugF("ImageDir=%s",ImageDir)
Assert(ImageDir.Exist,myName+"ImageDir Does not exist")
CorrTh = Default(CorrTh,0.5) # About 0.45 -> 0.55 [detect same subtile with/without crud]
Ocr1AveTh = Max(Min(Default(Ocr1AveTh,254),254),0)
Det3AveTh = Max(Min(Default(Det3AveTh,254),254),0)
# ExpandCnt = Min(Max(((ClipIx>0) ? Default(ExpandCnt,0) : 0),0),16)
ExpandCnt = Min(Max(Default(ExpandCnt,0),0),16)
ImgType = Default(ImgType,"bmp")
SMPTE = Default(SMPTE,false)
Assert(imgtype=="bmp"||imgtype=="png"||imgtype=="tiff"||imgtype=="tif",myName+"ImgType only 'bmp', 'png', 'tiff' or 'tif'")
Assert(RangeDB !="",myName+"RangeDB Cannot be ''")
FrameDB= ("~DetSub_Extract_FrameDB_"+RT_LocalTimeString+".DB").RT_GetFullPathName
OCR2 = DetSub_MaskClip(x=X,y=Y,w=W,h=H,
\ Text_w=TEXT_W,Text_h=TEXT_H,TextCol=TEXTCOL,TextTol=TEXTTOL,TextTolT=TextTolT,
\ Halo_L=HALO_L,Halo_T=HALO_T,Halo_R=HALO_R,Halo_B=HALO_B,HaloCol=HALOCOL,HaloTol=HALOTOL,HaloTolT=HALOTOLT,
\ clipIx=0,Matrix=MATRIX,Override=Override,framedb=FrameDB)
OCR2.RT_ForceProcess
SubsFrameCnt = RT_DBaseRecords(FrameDB) # Number of possibly nopn separated Subs
RT_DebugF("Detected Subtitle Frames=%d",SubsFrameCnt,name="DetSub_Extract: ")
RT_DBaseAlloc(RangeDB,0,"ii") # fields, 0=StartFrameNo, 1=EndFrameNo
MultiSubCnt=0
SplitCnt=0
MultiSubStart=-1 # Not currently within Subtitle range
FC=FrameCount
OCR1=OCR2.ShowRed(Pixel_Type="Y8") # White Text from red Channel
RT_DebugF("Splitting contiguous subs",name="DetSub_Extract::SplitSub: ")
for(n=0,FC) { # Scan & split Subtitle ranges [might not have clean frames between them]
Status = (n>=FC) ? 0 : RT_DBaseGetField(FrameDB,n,0)
Close = (MultiSubStart>=0 && Status!=1) # If we were scanning subs sequence but this one is invalid, then we will close.
if(Status == 1) { # Valid Subtitle Frame ?
if(MultiSubStart<0) { # n new MultiSub : New start of possibly non-separated sub
MultiSubCnt=MultiSubCnt+1 # NEW possibly non contiguous subs sequence
MultiSubStart=n
S_X1 = RT_DBaseGetField(FrameDB,MultiSubStart,1) # Subs Detected area for start Frame
S_Y1 = RT_DBaseGetField(FrameDB,MultiSubStart,2)
S_X2 = S_X1 + RT_DBaseGetField(FrameDB,MultiSubStart,3) # X + W
S_Y2 = S_Y1 + RT_DBaseGetField(FrameDB,MultiSubStart,4) # Y + H
} else { # already inside MultiSub sequence
R_X1 = RT_DBaseGetField(FrameDB,n,1) # Coords of current Subs frame
R_Y1 = RT_DBaseGetField(FrameDB,n,2)
R_X2 = R_X1 + RT_DBaseGetField(FrameDB,n,3)
R_Y2 = R_Y1 + RT_DBaseGetField(FrameDB,n,4)
U_X1 = min(S_X1,R_X1) # Union of both sets of Coords
U_Y1 = min(S_Y1,R_Y1)
U_X2 = max(S_X2,R_X2)
U_Y2 = max(S_Y2,R_Y2)
Corr=RT_LumaCorrelation(OCR1,OCR1,n=MultiSubStart,n2=n,x=U_X1,y=U_Y1,w=U_X2-U_X1,h=U_Y2-U_Y1)
Close = (Corr<CorrTh)
if(Close) {
SplitCnt=SplitCnt+1
RT_DebugF("%d] SPLIT multi-subtitle, MultiSubCnt=%d SplitSubCnt=%d Corr=%.3f",n,MultiSubCnt,MultiSubCnt+SplitCnt,Corr,name="DetSub_Extract::SplitSub: ")
}
}
}
if(Close) { # We were scanning sequence and either, this frame is not sub OR sub split found.
RT_DBaseAppend(RangeDB,MultiSubStart,n-1) # n is start of next new or split sequence or end of clip
MultiSubStart = (Status==1 && n<FC) ? n : -1
if(MultiSubStart>=0) { # Closure was due to Sub Split, get new split START coords
S_X1 = RT_DBaseGetField(FrameDB,MultiSubStart,1) # Subs Detected area for start Frame
S_Y1 = RT_DBaseGetField(FrameDB,MultiSubStart,2)
S_X2 = S_X1 + RT_DBaseGetField(FrameDB,MultiSubStart,3) # X + W
S_Y2 = S_Y1 + RT_DBaseGetField(FrameDB,MultiSubStart,4) # Y + H
}
}
}
NSubs = RT_DBaseRecords(RangeDB) # Number of SEPARATE INDIVIDUAL Subtitles
RT_DebugF("Contiguous Subs sequences=%d, Total split subtitles=%d",MultiSubCnt,NSubs,name="DetSub_Extract::SplitSub: ")
RangeFile=".\SubRanges.txt".RT_GetFullPathName
RT_FileDelete(RangeFile)
Grn=OCR2.BlankClip(Length=1,Pixel_Type="RGB32",color=$00FF00)
Blk=Grn.BlankClip(color=$000000)
DET3 = mt_logic(OCR2.ShowRed(pixel_type="Y8"),OCR2.ShowGreen(pixel_type="Y8").Invert,"or")
RT_DebugF("Writing %d %s images",NSubs,Select(ClipIx,"OCR2","OCR1","DET3"),name="DetSub_Extract: ")
RT_DebugF("ExpandCnt=%d",ExpandCnt)
fps=c.FrameRate
for(i=0,NSubs-1) {
S=RT_DBaseGetField(RangeDB,i,0)
E=RT_DBaseGetField(RangeDB,i,1)
RT_DebugF("Writing ... Subtitle %d, from range(%d,%d)",i,S,E)
RT_WriteFile(RangeFile,"%d,%d # %d",S,E,i,Append=true)
if(ClipIx==1) { # OCR1
T = OCR1.Trim(S,E).DetSub_AvgAll(Th=Ocr1AveTh,ExpandCnt=ExpandCnt)
} else if(ClipIx==2) { # DET3
T = DET3.Trim(S,E).DetSub_AvgAll(Th=Ocr1AveTh,ExpandCnt=ExpandCnt)
} else { # OCR2
T1 = OCR1.Trim(S,E).DetSub_AvgAll(Th=Ocr1AveTh,ExpandCnt=ExpandCnt)
T2 = DET3.Trim(S,E).DetSub_AvgAll(Th=Det3AveTh,ExpandCnt=ExpandCnt)
T=Grn.Overlay(Blk,Mask=T2).Overlay(T1,Mask=T1)
}
if(SMPTE) {
FNam=Frame2SMPTE(S,fps,3,true) +"__" + Frame2SMPTE(E,fps,3,true)
T.ImageWriter(ImageDir+"\"+FNam+"%.0d.%s",0,-1,ImgType).RT_YankChain(n=0) # Force Write image
} else {
T.Loop.ImageWriter(ImageDir+"\SUBS_%06d"+"."+Imgtype, i, i,ImgType).RT_YankChain(n=i) # Force Write image
}
}
RT_FileDelete(FrameDB)
(!UserDB)?RT_FileDelete(RangeDB):NOP
return Messageclip(RT_String("%s %d Subtitle Images written.",myName,NSubs))
}

[StainlessS]

Post #6 of 6.

And the extractor client


# DetSub_Extract_Client.avs

Import(".\DetSub_Args.avs")
Import(".\DetSub_Sundry.avs")
Import(".\DetSub_MaskClip.avs")
Import(".\DetSub_Extract.avs")

############################
ORG=Last


ClipIx = 1 # Default 1, OCR1 ie text. [0 -> 2 ONLY, 0 OCR2 [green background], 1=Text, 2=Detect3=Text+Halo combined white mask]

#RangeDB = ".\MyRange.DB".RT_GetFullPathName # Returns DBase with 2 fields, start and end of individual [separated] subtitle range.
RangeDB = UnDefined # Dont Make DBase [deleted after use]

ImageDir = ".\IMAGE"# MUST Exist. Output Images written here.
CorrTh = 0.5 # Correlation Threshold, maybe about 0.45 -> 0.55 : used in separating subs without a clear nonsub frame separator

Ocr1AveTh = 254 # Default 254 [0->254]. Applied to white mask pixels of all ClipIx [0,1,2]
# 0=Result pixel white if any single pixel in range is white. 254=result white only if ALL pixels in range are white.
Det3AveTh = 254 # Default 254 [0->254]. Applied to Black mask pixels of only ClipIx 2 [OCR2]
ExpandCnt = 0 # Default 0, Ignored for ClipIx==0. Expands only OCR1(text) or Detect3(text+halo). [Limited @ max 16]

Imgtype = "png" # Write BMP images. [BMP', "PNG", "TIFF", or "TIF" only]
SMPTE = True # SMPTE Time Format

DetSub_Extract(x=X,y=Y,w=W,h=H,
\ Text_w=Text_W,Text_h=Text_h,TextCol=TextCol,TextTol=TextTol,TextTolT=TextTolT,
\ Halo_L=Halo_L,Halo_T=Halo_T,Halo_R=Halo_R,Halo_B=Halo_B,HaloCol=HaloCol,HaloTol=HaloTol,HaloTolT=HaloTolT,
\ ClipIx=ClipIx,Matrix=Matrix,Override=Override,RangeDB=RangeDB,ImageDir=ImageDir,CorrTh=CorrTh,
\ Ocr1AveTh=Ocr1AveTh,Det3AveTh=Det3AveTh,ExpandCnt=ExpandCnt,
\ Imgtype=ImgType, smpte=SMPTE
\)

return last

/*
Ocr1AveTh, default 254. [0 -> 254]. Applied to binarize ClipIx==1 OCR1 mask, ie text.
Threshold applied after average/blending all SubsArea regions of a subtitle.
0, If any pre-average pixel is above 0, then result is 255(white). [ AvgAll2.mt_binarize(threshold=0) ]
254, If ALL pre-average pixels are above 254, then result is 255(white). [ AvgAll2.mt_binarize(threshold=254) ]
or 1 -> 253, somewhere between. [the pre-average pixels are always either 0 or 255].
Det3AveTh, default 254. [0 -> 254]. Applied to binarize ClipIx==2 Detect3 mask, ie Holo+Text combined.
Threshold applied after average/blending all SubsArea regions of a subtitle.
0, If any pre-average pixel is above 0, then result is 255(white). [ AvgAll2.mt_binarize(threshold=0) ]
254, If ALL pre-average pixels are above 254, then result is 255(white). [ AvgAll2.mt_binarize(threshold=254) ]
or 1 -> 253, somewhere between. [the pre-average pixels are always either 0 or 255].
Where ClipIx==0, ie the OCR2 green Multi-Mask, Det3AveTh is used in binarizing the black Text+Halo combined area
and the Ocr1AveTh used on the foreground white Text only area.

ExpandCnt, Default 0, [0 -> 16, silent limit at 16]. There should be enough clear area around detected subs in SubsArea to Expand.
ClipIx 1, ie OCR1, applies to OCR1 Text. Eg, c.mt_Expand(mode=mt_circle(ExpandCnt)
ClipIx 2, ie Detect3, applies to Detect3 Text+halo combined mask.
ClipIx 0, ie OCR2, applied first to Text+halo combined black mask, and then to OCR1 Text white mask.

ImgType, Default, "bmp". ["bmp", "png", "tif", "tiff"]. Select Image type to write.

Smpte, Default false.
False outputs style "SUB_000000.BMP". where 000000 is first subtitle group range in clip. [record 0 if DBase returned]
True outputs SMPTE style filenames with start and end times, eg 00_01_03_013__00_01_06_010.png
*/

[StainlessS]

Thanks, but you didn't say 'thanks' :)

[StainlessS]

Update to DetSub_Client_PASS2.avs
Moved EDITS from post #5 to here

59 separated subs detected [correct with given sample, about 32 without clear nonsub frame separator] but I think about 10 of them have edge crud [stuck for ideas on that].

Good subtitle [well just the first one, other good similar-ish]
https://i.postimg.cc/T3YmNfpP/SUBS-000000.png (https://postimages.org/)

Worst subtitle crud.
https://i.postimg.cc/Twh5Qwsj/SUBS-000048.png (https://postimages.org/)
Edge crud images not too difficult to fix in paint package, flood fill crud with black, a few seconds of effort per sub.

EDIT: @VX
Your AvgAll2() thingy reduced the below image crud to above.
https://i.postimg.cc/FsRMVJNC/Det-Sub-Client-PASS1-00.png (https://postimages.org/)

And this
https://i.postimg.cc/h49LZCPf/Det-Sub-Client-PASS1-01.png (https://postimages.org/)

to
https://i.postimg.cc/JzdZHvKd/SUBS-000026.png (https://postimages.org/)

Bottom two images, the 'f' of "facilitator" is slimmed down in fixed version,
do you have a similar routine to produce the greatest amount of white [rather than least],
using that Hysterisis whosit, could fix the slimming effect of AvgAll2().
EDIT: I guess that change to mt_binarize(threshold=0) would probably do it.

Using AvgAllMinMaxFix() instead of AvgAll()


Function AvgAll2(clip c) { c.FrameCount() <= 1 ? c : Merge(AvgAll2(c.SelectEven()), AvgAll2(c.SelectOdd())) }
#Function AvgAll(clip c) { c.AvgAll2().mt_binarize(threshold=254) }
Function AvgAllMinMaxFix(clip c) {
Ave = c.AvgAll2()
thin = Ave.mt_binarize(threshold=254)
Fat = Ave.mt_binarize(threshold=0)
Return thin.MT_Hysteresis(fat)
}

Result, with better 'f' of "facilitator", [all generally better]
https://i.postimg.cc/kMTTyJJT/SUBS-000026.png (https://postimages.org/)image sharing (https://postimages.org/)

Might change to using Clipblend instead of AvgAll2().

EDIT:
@VX
Btw, I'm interested in detection method you done in "DetectSub_MI", would it be possible to detect subs in a mask and extract them to images? One image per sub with time range in the filename, for example like "0_00_56_958__0_00_57_957.tiff
You got routine to convert <StartFrameNo, EndFrameNo, FPS> to your time format string ?

EDIT: I presume that results with live video rather than anime will be somewhat better, less chance of edge crud. [due to less static nature of live video]

[VoodooFX]

I'm stuck trying to run your last script from my thread:


LWLibavVideoSource("D:\subs_anime.mkv")

DetSub(Last, x="0",y="0",w="-0",h="-0",
\ Text_w="3",Text_h="3",TextCol="$FCFCFC",TextTol="$030303",
\ Halo_L="3",Halo_T="3",Halo_R="3",Halo_B="3",HaloCol="$102A37",HaloTol="$121212",
\ backColor="$00FF00",ClipIx="0",Matrix="rec601",db="MyDBBB.DB",Override="")


Error:
Script error: the named argument "x" to DetSub had the wrong type

Before re-re-write I was able to run it with these cords. My sample is cut.



You got routine to convert <StartFrameNo, EndFrameNo, FPS> to your time format string ?
I've no idea how to do it.

[StainlessS]

That looks like a function prototype [EDIT: ie args in double quotes signifies optional args with defaults], you seem to be calling function with eg X as string, it requires an int.
[not the coords themselves that are problem, you are using InPaintDeLogo style string coords].
(In fact you call with ALL string args)

EDIT:
I've no idea how to do it.
That Ok, I guess I know how.

EDIT:
Oh dear, I totally messed up the function prototype, by putting default in quotes rather than arg names,
and did not notice the mistake as my brain stuck like it was when i did it [ *** SORRY *** :( ].

anyways, fixed the prototype in post#3, as here. [BackColor arg was dropped, now always Green $00FF00]

function Prototype:
DetSub(clip c,Int "X"=-10,Int "Y"=Height-120,Int "W"=-10,Int "H"=-8,
\ Int "Text_w"=3,Int "Text_h"=3,Int "TextCol"=$FCFCFC,Int "TextTol"=$030303,
\ Int "Halo_L"=3,Int "Halo_T"=3,Int "Halo_R"=3,Int "Halo_B"=3,Int "HaloCol"=$030303,Int "HaloTol"=$030303,
\ Int "ClipIx"=0,string "Matrix"="rec601",string "Override"="",String "DB"="",Bool "DB_reUse"=False)

[VoodooFX]

My bad, after my post I noticed that you have updated the script there with the user part. I successfully run it and compared to the new InpaintDelogo version.

If there are persistent artifacts then "AvgAll2" wouldn't help much.

PS
Didn't had time to look at you new scripts, do you do range detection part employing stuff from previous subs detection or fresh detection from the mask?

[StainlessS]

Original DetSub() thing just detects if sub on individual frames, and writes DBase of detect coords [if detected].

DetSub_MakeRangesDB() thing, detects continuous frames where [not necessarily same] subs detected, and writes private DB of those ranges.
[I think sample has 27 such ranges, and the DB 27 records, with only Start, End fields].

Then scans above mentioned ranges comparing adjacent frames, looking for subtitle change without intervening clear nosubs frame,
and writes separated subs ranges to yet another DB. [in sample case DB has 59 records, ie 59 separated subs ranges]

Then scans that last DB and picks out [trims] separated range and does the avgAll2() thing [actually as post #9 AvgAllMinMaxFix()] to average subs, and remove a bit of crud.

Finally for each trimmed range average frame, writes a text file with ranges for each separated sub, and also writes BMP image for each, and then
returns a clip with each frame the final detect cleaned up-ish sub, in sample case, 59 frames, ie 59 separated subs in sample.

I'm changing the way it works to make the DetSub_MakeRangesDB() thing [under some other name, I usually use first name enters my head, and change later],
be the 'master routine' which will call the current DetSub [soon named DetSub_Mask], and do the whole thing in single [internally multipass] script run.

EDIT:
A final detect stage might do some kind of detection of pixel fluxtuations, stationary-ish = GOOD subs+halos and other where coincide with detected subs/halos
would be considered edge crud.
Maybe AvsInPaint has such a pixel fluxtuation type function, which you use for InPaintDeLogo.
Advantage to us would being able to apply that to already detected separated subs ranges.

[VoodooFX]

...whole thing in single script run.
Nice!

Maybe it will help, from some old tools I remember that they were doing this kind of detection:
https://i.imgur.com/RbXNs2R.png

Simplified logic: scan lines for white pixels, 2 most white lines should detect like the image above(discard line if it's too close), then ~half distance between 'red' lines extend to top and bottom.

I'll look at my backup hdds' for those tools, maybe there are source codes.

EDIT:
Probably it helped to separate subs from artifacts, so it wont help for range detection as we supply a refined mask already.

[StainlessS]

Bottom and top of lower case [small] chars already considered, but complications of edge crud above same chars, and we dont know how many
lines of subs there will be, nor size of charcters, nor if all lower case, so I kind of forgot it altogether. [too many 'ifs' and 'buts'].

[VoodooFX]

Maybe AvsInPaint has such a pixel fluxtuation type function, which you use for InPaintDeLogo.
Advantage to us would being able to apply that to already detected separated subs ranges.

Nothing there is be better than AvgAll2.mt_binarize in this case, I think.

Bottom and top of lower case [small] chars already considered, but complications of edge crud above same chars, and we dont know how many
lines of subs there will be, nor size of charcters, nor if all lower case, so I kind of forgot it altogether. [too many 'ifs' and 'buts'].

Yes, some additional logic would be needed for multi line detection (subs standard is max 2 lines).
"Edge crud" is a bad case and this sample is kinda synthetic... I'll check for some subs masks to do tests.

EDIT:
I've sent you PM with few masks.

[StainlessS]

Got Masks thanks.

(subs standard is max 2 lines)
I saw some time in last few months [maybe 6 months] a DVD with subs on about 5 or 6 lines [yeah, really, bout 1/4 or even 1/3 of frame taken up, and kept showing some on top of frame too,
also, I think some subs in our sample have subs on frame top too].

The DVD may have been season 1 "My Name Is Earl", where seems to have been translated into Chinese, and then from Chinese back to English,
is quite hilarious in some places reading what it eventually returned as.
The massively multi-line subs might have been Chinese language or other subs on that DVD [dont think was English subs].
[Seemed to be fair quality VHS cap [or TV station source], with HBO semi-transparent logo -Boxset does not look dodgy, but is a bit strange - not bought new]

The DVD in question was seemingly of Chinese origin, Both English & Chinese on box cover [also think was NTSC].
I think I had heard long time ago that My Name Is Earl is/was hugely popular in China, one is curious as to why.

[VoodooFX]

...a DVD with subs on about 5 or 6 lines...
That I would call intertitles. :D

Here are some subs tools with sources:

AviSubDetector
https://forum.doom9.org/showthread.php?t=89802
http://web.archive.org/web/20071031111130/http://animeburg.omake.ru/avisubdetector.htm
I member it being good in comparisons to other tools at that time, I didn't managed to make it useful, maybe because I tried non anime stuff on it, when tool is aimed for anime.
By "useful" I meant getting subs faster than just simply writing them down manually.

SubLog Extractor (Vdub)
https://www.softpedia.com/get/Multimedia/Video/Other-VIDEO-Tools/SubLog-Extractor.shtml
Don't remember about it, for some reason it's saved on my hdd.

VideoSubFinder
https://sourceforge.net/p/videosubfinder/src/ci/master/tree/
Still active dev and tool does decent job.
From my tests: it can produce dozens of errors/artifacts if you let it do masks, lots of manual work to deal with them, could be fine-tuning problem on my side, so I simply supply my mask for it as video.
Problem is that it tries to extract mask from my mask and sometimes damages some mask/subs lines, but its method to extract ranges is pretty good.

EDIT:

If "0_00_56_958__0_00_57_957.tiff" format is not possible/hard to do, there is option to nag SubtitleEdit's (https://forum.doom9.org/showthread.php?t=162721) dev to support format like "000022__000374__23_976.tiff".

[StainlessS]

Thanks VX.
"0_00_56_958__0_00_57_957.tiff",
Not really a problem.

Why is tiff required, BMP not OK ? [compressed size ?]
I think ImageWriter supports tiff, so could make optional.

Found some dfferences between using AvgAll2() and the Hysterisis version and mod with ClipBlend instead of SelecteEven/Odd merge thingy.
ClipBlend version can be worse results, I'm guessin' due to ClipBlend averaging and rounding only after all frames sampled [single rounding for entire range],
whereas the merge thingy rounds with loss of precision at every recurse.
Anyway, I'll have to extract the edge crud ranges and investigate exactly what is happening there.

Maybe that average All whotsit dropped altogether if we instead scan trim/crop range of original source and somehow get flutuations for RGB pixels,
[as we used tolerances in detecting halo/text], maybe fluctuating RGB is edge crud.
EDIT: tolerances being somewhat spatial requirement, but fluctuations temporal.

EDIT: 1st 2 links zips dead.

EDITl: Oops, changed multiple "ClipClop"s to "ClipBlend"s.

[VoodooFX]

Why is tiff required, BMP not OK ? [compressed size ?]
I think ImageWriter supports tiff, so could make optional.
Don't remember now, could be for some OCR thing or just randomness of the example.

1st 2 links zips dead.
Zips in the archived site works, need to wait some seconds when you press on download links there.

Maybe that average All whotsit dropped altogether if we instead scan trim/crop range of original source and somehow get flutuations for RGB pixels,
Now tested it on sub lines with most artifacts on InpaintDelogo's mask, most of them disappeared, even AvgAll2.mt_binarize(200) did good job.

About temporal fluctuations: then why not take the mask from InpaintDelogo? ;)
I tried to employ RGB and YV12 colors in creating dynamic masks there, but it had small effect on reducing artifacts, could be that it didn't worked well with temporal stuff, or that clips were not anime, or I was doing it wrong, so I ditched that idea - and actually everything is done in greyscale there. Colored hardcoded subtitles are mostly a thing of the fansubs, I don't see point in dealing with such videos.
I see that in your script color stuff works pretty well.

ClipBlend instead
I'm a bit worried about that "53" frames thingy slowing the script down. Edit: Nevermind, it won't have effect here. Edit2: I'm not sure...

[VoodooFX]

Here I chained those worse ranges for example (slowed to 15fps):

https://i.imgur.com/rJTTFUU.gif
Video: https://imgur.com/aUErj2r

[StainlessS]

Well I found my mistake producing edge crud and will post fix later.
We expand text by halo 'thickness', and also halo by text 'thickness', then AND the two together
to get text + halo area.
Mistake is that after expand of halo [to close up the hole where text lives], we should have then
shrunk expanded halo again [to get rid of oversized halo that extends outsize of the original halo, and is cause of edge crud where coincides with not text white stuff].

[StainlessS]

DetSub v0.0,
see 7z first post. [EDIT: 1st 6 posts updated]

Added subs image extractor, works in single pass. [although need setup coords etc using Masks script].

Extracts BMP, TIFF, PNG and as eg "SUB_000006.BMP".
In either "Name_012345.bmp" style or STPME style "00_02_17_023__00_02_21_000.png".
Also can expand masks.
some examples.

00_02_17_023__00_02_21_000.png
https://i.postimg.cc/GpgFkj9f/00-02-17-023-00-02-21-000.png (https://postimages.org/)


The tiff file type I did was not recognised by Postimage, and would not give me a link, but VD2 loads it ok. [mind you Windows 10 didnt like AVS tiffs either.

So here same file as png
00_02_17_023__00_02_21_000.png
https://i.postimg.cc/hvf3LjSL/00-02-17-023-00-02-21-000.png (https://postimages.org/)


7z again here
DetSub as 7z:- https://www.mediafire.com/file/wsqsho09oiu8fxw/DetSub_WIP_v0.0.7z/file
All scripts in folder and also IMAGE folder included inside [writes output images there].
All you need is the sample in above script, resaved as avi via ffmpeg or Vdub2, and away you go.

EDIT: Damn, D9 dont like some of the files that Postimage was ok with. [ImageWriter prob ???]
I deleted them from post. [EDIT: OK, maybe I gotta set alpha mask or something for D9, they were all black images]

EDIT: Maybe I should also allow jpg output just for D9 and PostImage, I'll do it next time.

EDIT: Maybe have DebugView open during extract, shows what its doing.
DebugView v4.9, SysInternals [ie M$]:- https://docs.microsoft.com/en-us/sysinternals/downloads/debugview
Nice to have open on a 2nd monitor during development, years since I've had 2 monitors, forgot how great it is.

[EDIT: I kinda miss my little old metal 9inch green-screen industrial monitor that I used with Sinclair QL [EDIT: and before it, Sinclair ZX Spectrum], very dinky.
2nd monitor on this machine is HP 19inch 4:3 which cost me £1.50 @ CEX, great condition, good little monitor,
they just wanted rid because nobody wants them today. I got two spare Dell 17inch (garbage color) , maybe I hook
them up to my other machines too for debug].

EDIT: to add JPG extract, edit line 126 of DetSub_Extract.avs to this

Assert(imgtype=="bmp"||imgtype=="png"||imgtype=="tiff"||imgtype=="tif"||imgtype=="jpg"||imgtype=="jpeg",myName+"ImgType only 'bmp', 'png', 'tiff', 'tif', 'jpg' or 'jpeg'")


EDIT: as jpg, SUBS_000000.jpg
https://i.postimg.cc/bY523JyZ/SUBS-000000.jpg (https://postimages.org/)

and expanded a bit [no particular use, but it looks nice]
https://i.postimg.cc/pLHQJTCD/SUBS-000000.jpg (https://postimages.org/)

[VoodooFX]

Thanks, I'll test it when I'll have time.

I remember that OCR soft likes them big, so would be useful options to output images x2 and x4 in size. PointResize should be good.

[StainlessS]

You seemed a bit perturbed when I changed the tolerance arg, so I made mod,
Now got both methods [same for halo]

TEXTCOL = $F3F3F3 # Approx RGB color of text
TEXTTOL = $0E0E0E # Per channel text tolerance
TEXTTOLT = 0 # Additional text Tweak Tolerance, added to each individual channel of TEXTTOL [+/-].


PointResize, ok.

[VoodooFX]

Tested 'DetSub_MaskClip' script, almost no edge crud left, good improvement.
Noticed few new artifacts - parts of letters disappear:

Frame 2314:
https://i.imgur.com/nAr8ael.png

Frame 2772:
https://i.imgur.com/GpXNKNw.png

That would be a problem if a mask is used for inpainting or overlay with lower quality video. Solution would be to take that averaged/extracted frame and loop it back into the final mask, that would make a mask less flickery too.

Not tested the extract script yet, because I got carried away with new experiments to reduce artifacts further for InpaintDelogo. :D

PS:
Thinking taking into account that halo, but somehow making it simple. Anyway, few new parameters, here they come... :D

[StainlessS]

Not tested the extract script yet
Well if you already tried the mask script, then it would have only taken about 90 seconds to try the extract script.

Yep, gettin' rid of edge crud produced a few hiccoughs like that.

Here with the the extract binarizing Ocr1AveTh @ default 254 [maybe too severe].

Sub 33
https://i.postimg.cc/TYYHxYs8/SUBS-000033-254.jpg (https://postimg.cc/fkpvjZZ5)

Sub 40
https://i.postimg.cc/3xkSyf7n/SUBS-000040-254.jpg (https://postimg.cc/YvwfZ3vW)

Here with the the extract binarizing Ocr1AveTh @ 200 [just a 1st wild guess, maybe a good default]
Sub 33
https://i.postimg.cc/XvwgFTPc/SUBS-000033.jpg (https://postimg.cc/rdwW2YMK)

Sub 40
https://i.postimg.cc/YSw6pdS4/SUBS-000040.jpg (https://postimg.cc/0zcMndDx)

Extract script gonna change a bit, better I hope.

EDIT: Oops, had it set to UpSize=2, so double sized. [not implemented in posted scripts]

EDIT: Anyways, surely the inpaint thing would want to use text + halo combo mask [DET3], and also expanded a bit, those missing fragements might not make a lot of difference to inpaint.
[but point seems moot with binarizing thing at 200].

EDIT: back @ binarizing 254, text+halo and expand by 3
Not so good really. [but binarize @ 200 would have no problems]
Sub 33
https://i.postimg.cc/5NsPPdnx/SUBS-000033.jpg (https://postimages.org/)

Sub 40
https://i.postimg.cc/GhmKTgr6/SUBS-000040.jpg (https://postimages.org/)

EDIT: Currently modding [extractor] to change from DET3/OCR1/OCR2 model,
to being able to specifiy to ouput/handle Text Only, Halo Only, Text+Halo handled together, text+halo handled separately.
Eg, output text+halo as white mask, but apply binarize separately to text and halo, then re-combine for output.

EDIT:
Well if you already tried the mask script, then it would have only taken about 90 seconds to try the extract script.
What a good guesser I am, I added total time taken to debug output, and on my machine took 89.33 secs [So I'm about 0.75% in error :) ]

[VoodooFX]

Finally got time to test the extractor.

For me it took ~6mins to run (on my sample). Btw, on AVSMeter it didn't work.
Here are images OCRed to srt: https://pastebin.com/RRQT8Cgs

Timing is off on extracted images, that pink box in DetSub_MaskClip is accurate though.

[StainlessS]

AVSMeter it didn't work.
Did not for one second expect it to.
Returns only a MessageClip("done") style message after task is completed.

Your Pastebin thing flashes up some text (no idea what it was) then is blank [may have been subs or pastebin instructions or something].

Timing is off on extracted images
That is probably down to using only single precision float in script. [EDIT: and float FPS]

How do you ascertain that it is wrong ?
[EDIT: Presumably you find subs start/end incorrect, BUT, how do you convert SMPTE to frameNo, is that also in script?]

Test script, 1 frame out on last frame @ 59:59:29 < -- > 59:59:28

Test script

Function Frame2SMPTE(int Frameno,float Fps,int "digits",bool "File") { # ssS Mod of fn by tsp:- https://forum.doom9.org/showthread.php?p=551000#post551000
ch = Default(File,false) ?"_":":"
digits = Default(digits, int(log(max(Fps,1))/log(10))+1) # Log10 v2.60+, [Log10(n) = Log(n) / Log(10)]
hh=int(Frameno/3600/Fps) mm=int((Frameno-hh*3600*Fps)/60/Fps)
ss=int((Frameno-hh*3600*Fps-mm*60*Fps)/Fps) ff=int((Frameno-hh*3600*Fps-mm*60*Fps-ss*Fps))
return String(hh,"%02.0f")+ch+string(mm,"%02.0f")+ch+string(ss,"%02.0f")+ch+string(ff,"%0"+string(digits)+".0f")
# return RT_String("%02d%s%02d%s%02d%s%0*d",hh,ch,mm,ch,ss,ch,digits,ff)
}

Colorbars().Killaudio
DIGITS=3
ShowSMPTE
SSS=""" return Subtitle( Frame2SMPTE(current_frame,FrameRate,DIGITS,false),align=8) """
Scriptclip(SSS)


EDIT: I am not aware of any plugin which returns SMPTE calc'ed in 64 bit.
Think that ExactDedup() has stuff for MKV timescodes, which I assume are the same.

[VoodooFX]

Your Pastebin thing flashes up some text (no idea what it was) then is blank...
Maybe some blocking addon, here is link to raw: https://pastebin.com/raw/RRQT8Cgs

How do you ascertain that it is wrong ?
I played video with srt subtitles, or you can look at AvsPmod:

First subtitle frames 0-189 (00:00:00.000-00:00:07.883), to be precise - 190th frame should be the end of subtitle (00:00:07.925).

Range in the extracted image: 00:00:00.000-00:00:07.021. It's off almost by 1 second.

EDIT:

[EDIT: Presumably you find subs start/end incorrect, BUT, how do you convert SMPTE to frameNo, is that also in script?]

Why convert back to frameNo? SRT subtitles operates in timecode, if you want other subs(where it operates with frameNo) format then the subtitle programs easily can convert to that.

[StainlessS]

00:00:07.925
That aint SMPTE, is it.
The .925 thing is not specified as frames, but as fractional part of a second, [ie as VDub or SRT uses],
so you want as fractional part not mins + seconds + frames.

EDIT:
00:00:00.000-00:00:07.021. It's off almost by 1 second.
The Filename ending "_021" is seconds + frames, not fractional seconds, ie 21 frames @ 23.976 FPS.

I'll see if I can do it right, this time. [but still best requires double precision]
Guess I just got it in my head that you wanted SMPTE.

SMPTE:- https://en.wikipedia.org/wiki/SMPTE_timecode

[VoodooFX]

Oh, that's why it looked so weird to me. :)
It's not SMPTE...

[StainlessS]

Just posting this here so I can find it again [from long time past, from Mugfunky]:- https://forum.doom9.org/showthread.php?p=685919#post685919

here you go:

#
# Mug's Timecode stuff.
#
# tc:
# enter a timecode string in quotes, and out comes an integer frame number.
#
# itc:
# reverse of tc - enter a frame number, and out comes a SMPTE timecode.
#
# - for both of these, you can enter a framerate as well (last.framerate is useful)
#
#
# super:
# outputs a SMPTE timecode in a shaded box. useful for subtitling.
#
# - you can also enter a "start timecode" in quotes (like "10:00:00:00")
# which helps if you're syncing with a tape's timecode.
#



function tc (string "timecode", float "rate")
{
rate=default(rate,25)

frames=value(rightstr(timecode,2))
secs=value(rightstr(timecode,5).leftstr(2))*rate
mins=value(rightstr(timecode,8).leftstr(5))*60*rate
hours=value(rightstr(timecode,11).leftstr(8))*60*60*rate
int(hours+mins+secs+frames)
}

function itc (int "framecount", float "rate", bool "ms")
{
rate=default(rate,25)
ms = default(ms, false)

drop = (rate==29.97)? true : false
rate2 = (drop==true)? 30 : rate

hours=floor((framecount/rate)/3600)%60
mins=floor((framecount/rate)/60.0)%60
secs=floor(framecount/rate)%60
milli=floor(1000*framecount/rate)%6000%1000
fmilli=framecount/rate - floor(framecount/rate)
#frames=floor(fmilli*rate2)
frames=framecount%int(rate)

dframes = (drop==false)? frames : (secs==0)&&(mins%10!=0)? floor(fmilli*rate2) + 2 : frames

return (ms==false)? (string(hours,"%02.0f")+":"+string(mins,"%02.0f")+":"+string(secs,"%02.0f")+":"+string(frames,"%02.0f")) :
\ (string(hours,"%02.0f")+":"+string(mins,"%02.0f")+":"+string(secs,"%02.0f")+":"+string(milli,"%03.0f"))
}

function super (clip c, string "offset", bool "ms")
{
global rate = c.framerate
global bheight=int(c.height*0.15/4)*4
bwidth=int(c.width*0.4/4)*4
off=int(c.height*0.15/4)*4
ms = default(ms, false)

offset = default(offset,"00:00:00:00")
global offset = tc(offset)
global ms = ms

box=c.crop((c.width-bwidth)/2, c.height-(bheight+off), bwidth, bheight).levels(0,1,255,0,160)
left=c.crop(0, 0, (c.width-bwidth)/2, 0)
right=c.crop((c.width-bwidth)/2 + bwidth, 0, (c.width-bwidth)/2, 0)
top=c.crop((c.width-bwidth)/2, 0, bwidth, c.height-(bheight+off))
bottom=c.crop((c.width-bwidth)/2, c.height-off, bwidth, off)

box = ScriptClip(box, "Subtitle(String(itc(current_frame+offset, rate, ms=ms)),align=2,y=int(.225*bheight) + bheight/2, size=round(.45*bheight), spc=int(.3*bheight), text_color=$ffffff)")

middle=stackvertical(top,box,bottom)
stackhorizontal(left,middle,right)

}



EDIT:
And some [PDF from Adobe] stuff bout that there crazy drop frame timecodes thingy[direct download of pdf, not web page]:- https://www.connect.ecuad.ca/~mrose/pdf_documents/timecode.pdf

EDIT: More stuff on drop frame timecode [from wiki]:- http://avisynth.nl/index.php/ShowFrameNumber
ShowFrameNumber(),
ShowSMPTE(),
Showtime() # which I was not aware of (or forgot).


ShowSMPTE() note
Note:
With certain exceptions, SMPTE timecode has no concept of fractional frame rates (like 24.5 fps for example).

ShowSMPTE source clips must have an integer framerate (18, 24, 25, 30, 31,...) or a drop-frame rate ('29.97' being the most common).
Supported drop-frame rates are listed in the table below. If that's not the case an error will be thrown.

If the framerate is not integral or drop-frame (let's call it "nonstandard" for short), use ShowFrameNumber or ShowTime instead.

You may encounter media sources that are almost at a standard framerate, but not quite – perhaps due to an error in processing at some point,
or perhaps the source was something like a security camera or a video game console. In this case you should force the clip to the nearest standard
framerate with AssumeFPS.



Drop-Frame versus Non-Drop-Frame Time Code

When television began, it was black-and-white only. At that time NTSC ("American" standard) television ran at
30 frames per second (60 fields per second). When the television engineers added color, they slowed the frame
rate by the precise ratio 1000/1001, due to technical reasons. NTSC televisions now run at 30×1000/1001 or
approximately 29.97002997 frames per second. This is commonly called "29.97 fps." 29.97 is the nominal framerate,
a convenient shortcut term for 30×1000/1001.

This slight slowing of the framerate complicates the display of timecode. A second of time no longer consists of a
whole number of frames. If the timecode readout simply advanced the seconds counter every 30 frames, the timecode
reading would be slower than clock time by about 3.6 seconds per hour. Timecode displays cannot show "fractional"
frames (their whole purpose is to uniquely identify every frame) so they drop the display of just enough frame numbers
to make the displayed timecode correspond to real or clock time. This is done in a prescribed and repeatable fashion:
the first two frame numbers of every minute, except for the tenth minute, are dropped, ie

You may encounter the term "NDF" - this means "non-drop-frame." As you would expect, this is used for all
the integer framerates. Sometimes though, video running at drop-frame rates will have NDF timecode.
This is most common for short-form videos of a few minutes' duration at most: some video professionals prefer not
to skip frame numbers at all, even though the time display will be off slightly. To get ShowSMPTE to show NDF timecode
at drop-frame rates, see the example below.


Showing non-drop-frame timecode at drop-frame rates:

ColorBars ## (framerate = 29.97)
ShowSMPTE(size=24, y=24) ## timecode (top of screen) is DF (drop-frame)
C=Last
AssumeFPS(30) ## force integer framerate
ShowSMPTE ## timecode (bottom of screen) is NDF
AssumeFPS(C) ## fps returned to original
return Last


## DF (top) skips frame numbers at frames 1800, 3598, 106094...
## NDF (bottom) does not skip numbers but runs slower than real time
## (frame 106094 = DF "00:59:00:02" == NDF "00:58:56:14")

Using offset, x, y, font, size, and text_color arguments:

ShowSMPTE(offset="00:00:59:29", x=360, y=576, font="georgia", size=24, text_color=$ff0000)

[VoodooFX]

There is Timer script from djcj: https://forum.doom9.org/showthread.php?t=168241

What about those frame properties added to newer AvS+, couldn't they be employed for something for extraction?

Yeap, tiff's are borked for some reason.

[StainlessS]

For me, AVS written tiffs were not acceptable to PostIlmage.org,
and windows 10 did not like them either, but VDub2, no probs.

Why do our American cousins always have to be different ?, I mean but why ?,
I hate stuff like that [drop frame], and UTF8 and Unicode and all other nonsense [OK, USA aint always to blame - but I'm gonna blame them anyways.]
The sooner that English (the real one not USA Pidgin-English) is declared to be the only acceptable language spoken / written any/every-where the better.

In a lousy mood, a bit earlier went head-over-tit slipping on wet road at busy intersection, luckily no articulated lorries nor double decker buses were in
close proximity, so I did not get squished, grazed elbows and I also missed my bus, ... life's a bitch and then you die [and it dont get no better after that].

Tomorrow is another day, much like this one probably :)

EDIT: Anyways, just watched the last 20 mins of The Martian [which I curtailed watching yesterday], so maybe got me some feel-good-factor there,
great movie, Ridley Scott's best, maybe. [EDIT: I only bought The Martian bout 4 weeks ago, watched it about 5 times already, I must like it].

[VoodooFX]

I'm fascinated by how the Italians produce English, reading it always feels like I'm cracking the Enigma rig, so when I produce something [not nearly]similar I proudly leave it as is for the comic value.

Disclaimer:
English is my 7th language and I never tried to learn it and I've not seen an English grammar book in my life.

PS:
I've noticed that "slipping" thing too, some say that it's something to do with the age, but clearly Earth is getting more slippery, and they worry about some warmings, silly humans.

[StainlessS]

I proudly leave it as is for the comic value
ArHa, so you do it on purpose.

It fascinates me how anybody could speak so many languages, I have probs with just the one :(
Peter Ustinov:- https://en.wikipedia.org/wiki/Peter_Ustinov
could speak 6 langs fluently, and a few more not so fluent [although I have seen him credited with about 26 langs somewhere],
beats the hell out of me how he/you do that.

Yip, earth is gettin' more slippery, and it dont help that the shoes I was wearing had no tread left, need to bin them, nearly as slippery
as Doctor Martens on wet grass.

Something from a little while back:



So basically 'Heat Haze'. [I'de never heard the term Fata Morgana, looked Heat Haze up and it got me to Wikipedia Mirage:- https://en.wikipedia.org/wiki/Mirage].
I would not have any idea how to simulate heat haze.

Regarding the wikipedia Mirage link, Fata Morgana paragraph, "atmospheric duct" hilited popup, I think I once experienced that myself.
Back in 1981, I was stayng at a guest house in Bedfordshire countryside, about 5 of us were watching an Italian movie on analogue TV.
Due to weird atmosperic conditions, we could not pick up UK TV, only a single Italian TV station (maybe due to 'atmospheric ducting' which
probably acts similar to microwave WaveGuide used in Radar and microwave communications). the Italian movie of course had no subtitles
(as it was broadcast for Italians, not Brits), so we asked the Swiss girl with us to translate (she spoke Swiss German, Italian, French and English),
and we all sat there transfixed as she translated for a good 10 minutes, at which point she realised that nobody was watching the movie,
we were all fixed upon her. I said something like, "Enough of the French, how about giving it to us in English".
Some additional quite strange linguistic ducting had occurred.

Woz well weird. [EDIT: We normally (never) could not pick up any foreign TV stations at all]


EDIT: Doc Martens:- https://www.drmartens.com/uk/en_gb/

EDIT: On the slipping thing, this guy had a hellova bad slip 2 days ago,
Man crushed to death between Tube and platform when nobody was there to help him:- https://metro.co.uk/2021/09/07/man-crushed-to-death-between-tube-and-platform-at-waterloo-station-15220654/
I guess I was lucky.

[VoodooFX]

I've slimmed down your extractor to the core bits, now need to figure out what all those RT_DBASE thingies do and adapt it to InpaintDelogo's type of artifacts, probably the coords stuff there won't be helpful for me.
So far, I've replaced SMPTE bits with this:

FNam=Frame2Time(S,fps) +"__" + Frame2Time(E+1,fps)


function Frame2Time(int frame, float fps) {
msTime = round(frame * 1000 / fps)

_s = msTime / 1000
_ms = msTime % 1000
_m = _s / 60
_s = _s % 60
_h = _m / 60
_m = _m % 60

hh = string(_h, "%02.0f")
mm = string(_m, "%02.0f")
ss = string(_s, "%02.0f")
ms = string(_ms, "%03.0f")

timeStr = hh+"_"+mm+"_"+ss+"_"+ms
return timeStr
}

[StainlessS]

DetSub_WIP_v0.1.7z
https://www.mediafire.com/file/r3jzixuv3xv6abo/DetSub_WIP_v0.1.7z/file

Have not updated 1st post zip, nor thread scripts.
Sub_000001.tiff now start at 1 [instead of 0, same as sub index in a subtitle app].


# DetSub_Extract.avs

DetSub_Extract(), extracts subtitle images from video clip.

Function Prototype:

DetSub_Extract(clip c,Int "X"=-10,Int "Y"=Height-120,Int "W"=-X,Int "H"=-8,
\ Int "Text_w"=4,Int "Text_h"=4,Int "TextCol"=$FFFFFF,Int "TextTol"=$0F0F0F,Int "TextTolT"=0,
\ Int "Halo_L"=3,Int "Halo_T"=3,Int "Halo_R"=3,Int "Halo_B"=3,Int "HaloCol"=$000000,Int "HaloTol"=$0F0F0F,Int "HaloTolT"=0,
\ Int "ClipIx"=1,string "Matrix"="rec601",string "Override"="",String "RangeDB"="",String "ImageDir"="",Float "CorrTh"=0.65,
\ Int "AveBinTh"=230, int "ExpCnt"=0, int "ExpCnt2"=ExpCnt,
\ int "UpSize"=1, String "ImgType"="bmp",Bool "FrameTime"=false,String "LogFile"="DetSub_Extract_LOG.txt")

Args:-

X,Y,W,H, [defaults X=-10, Y=Height-120, W=-X. H=-8]. Coords to search for subtitles, subtitle Detect Area.

Text_W, default 4, Width of text verticals.
Text_H, default 4, Height of text horizontals.

TextCol, default $FFFFFF, Color of Subtitle Text in RGB.
TextTol, default $0F0F0F, RGB per channel tolerance for text.
TextTolT, default 0, Addtional text tolerance tweaker. Added to each of the TextTol R, G, or B channel tolerances. [+/-]

Halo_L, default 3, Width of Left halo verticals.
Halo_T, default 3, Height of Top halo horizontals.
Halo_R, default 3, Width of Right halo verticals.
Halo_B, default 3, Height of Bottom halo horizontals.

HaloCol, default $000000, Color of Subtitle Halo in RGB.
HaloTol, default $0F0F0F, RGB per channel tolerance for Halo.
HaloTolT, default 0, Addtional Halo tolerance tweaker. Added to each of the HaloTol R, G, or B channel tolerances. [+/-]

ClipIx, default 1, OCR1 [Selects mask for output, 0 -> 2 ONLY. Mask clip, dimensions X,Y,W,H.
ClipIx selects which subs images to extract as images to ImageDir.
0 = OCR2, Green background multi-mask with text in white, halo in black and other guys in green.
To extract OCR1 (Text) mask from ClipIx=0 (OCR2)
OCR1 = OCR2.ShowRed(Pixel_Type="Y8") # OCR1 Text from OCR2 red Channel
To extract DET3 (combined Text+Halo) mask from ClipIx=0 (OCR2)
DET3 = mt_logic(OCR2.ShowRed(Pixel_Type="Y8"),OCR2.ShowGreen(pixel_type="Y8").Invert,"or") # DET3 from OCR2
1 = OCR1, Text in White.
2 = Detect3, Text + Halo in White.

Matrix, default "rec601", for src conversion to RGB32.
Override, default "" [Not used]. Allows for ignoring ranges where subtitles detected but not existing.
""=Not used, "*..." multiline string of ranges, "OverRide.txt"=Text file of ranges.

OverRide="""* # <<=== NOTE Asterisk (STAR). Example String OverRide (1st char is '*'). Ranges OverRidden to NO SUBTITLE
10 100 # A comment, SPACE or COMMA Separator
200,300 # 200 to 300. NOTE -ve frame count NOT supported.
3000,0 # 3000 till last frame.
"""
RangeDB, default "". If "" then the subtitles ranges DBase is private, and deleted on Function Exit. [so long as CallCmd plugin is installed].
If provided with a DBase filename, then the DBase will be be available to calling routine on return.
RangeDB DBase Fields:- [DBase record number is the zero relative ordinal number of the subtitle range]
0) Start frame of subtitle for current record subtitle range.
1) End frame of subtitle for current record subtitle range.

ImageDir, Default "", ie invalid. MUST be set to existing directory, ".\" is current directory.
Directory where output images are to be written.

CorrTh, default 0.65, [range about 0.6 -> 0.7]. Correlation threshold to detect where subtitles in adjacent frames are the same,
used to detect where to split apart non similar subs. Probably not need change.

AveBinTh, default 230. [0 -> 254]. Applied to re-binarize pixels [after subtitle range has been temporally averaged/blended to produce a single frame].
The result pixel is compared with the AveBinTh threshold. If blended result pixel is above TextAveTh, then is made 255, else 0.
0, If result blended pixel is above 0, at least 1 of the pre-blend pixels was 255 and result is 255(white), else 0, black.
254, If average pixel is above 254, then result is 255(white). (This can make any fluctuating pixels black, and only non fluctuating
or 1 -> 253, somewhere between. [if result blended pixel is eg 128, might indicate that about half the pixels are fluctuating].

ExpCnt, default 0, [0 -> 16, silent limit at 16]. There should be enough clear area around detected subs in SubsArea to Expand.
Applied to white pixels of OCR1, DET3, and OCR2 masks.
ExpCnt2, default ExpCnt, [0 -> 16, silent limit at 16].
Applied only to halo (black) pixels of OCR2 mask. Should usually be at least same as ExpCnt (otherwise expanded text can hide halo).

UpSize, Default 1, [1,2,3,4]. 1=No upsize. 2, 3 and 4, PointResize upsize output by that multiplier.

ImgType, Default "bmp", ["bmp", "png", "tif", "tiff", "jpg", "jpeg"]. Select Image type to write.

FrameTime, Default false.
False outputs style "SUB_000001.BMP". where 000001 is first subtitle group range in clip. [record 0 if DBase returned]
True outputs Frame time style filenames with start and end times, eg 00_01_03_123__00_01_06_456.png



Partial Debug output from DeSub_Extract

00000468 0.24139710 RT_ForceProcess: Commencing Forced process
00000469 1.96403956 RT_ForceProcess: 199] 4.99% nFrms=200 T=1.713sec : 116.75FpS 0.008565SpF
00000470 3.42596316 RT_ForceProcess: 399] 9.99% nFrms=200 T=1.469sec : 136.15FpS 0.007345SpF
00000471 4.83787680 RT_ForceProcess: 599] 14.98% nFrms=200 T=1.406sec : 142.25FpS 0.007030SpF
00000472 6.28145981 RT_ForceProcess: 800] 20.00% nFrms=201 T=1.453sec : 138.33FpS 0.007229SpF
00000473 7.69002962 RT_ForceProcess: 1000] 24.99% nFrms=200 T=1.405sec : 142.35FpS 0.007025SpF

# ...

00000485 24.84640121 RT_ForceProcess: 3404] 85.00% nFrms=201 T=1.453sec : 138.33FpS 0.007229SpF
00000486 26.31667900 RT_ForceProcess: 3604] 89.99% nFrms=200 T=1.468sec : 136.24FpS 0.007340SpF
00000487 27.69695091 RT_ForceProcess: 3804] 94.98% nFrms=200 T=1.391sec : 143.78FpS 0.006955SpF
00000488 29.01732254 RT_ForceProcess: 4005] 100.00% nFrms=201 T=1.312sec : 153.20FpS 0.006527SpF
00000489 29.01734161 RT_ForceProcess: Time=28.769secs (0.479mins) : Avg 139.247FpS 0.007181SpF
00000490 29.01754189 DetSub_Extract: Detected Subtitle Frames=4006
00000491 29.01794815 DetSub_Extract::SplitSub: Splitting contiguous subs
00000492 30.63635254 DetSub_Extract::SplitSub: 1, [0,189]
00000493 31.33093834 DetSub_Extract::SplitSub: 2, [227,315] Corr=0.093 *SPLIT*
00000494 31.62722588 DetSub_Extract::SplitSub: 3, [316,358] Corr=0.121 *SPLIT*
00000495 31.98716545 DetSub_Extract::SplitSub: 4, [359,410]
00000496 32.34925842 DetSub_Extract::SplitSub: 5, [448,498] Corr=0.097 *SPLIT*
00000497 32.70031738 DetSub_Extract::SplitSub: 6, [499,543] Corr=0.057 *SPLIT*

# ...

00000546 49.71937943 DetSub_Extract::SplitSub: 55, [3535,3659]
00000547 49.93394852 DetSub_Extract::SplitSub: 56, [3676,3705] Corr=0.103 *SPLIT*
00000548 50.15644073 DetSub_Extract::SplitSub: 57, [3706,3741]
00000549 50.48969650 DetSub_Extract::SplitSub: 58, [3901,3947] Corr=0.027 *SPLIT*
00000550 50.62948990 DetSub_Extract::SplitSub: 59, [3948,3969]
00000551 50.63078690 DetSub_Extract::SplitSub: Contiguous Subs sequences=28, Total split subtitles=59
00000552 50.63088226 DetSub_Extract: Writing 59 OCR1 images
00000553 51.45705795 DetSub_Extract: Writing ... Subtitle 1, [0,189] Times=00:00:00.000__00:00:07.883
00000554 51.92165756 DetSub_Extract: Writing ... Subtitle 2, [227,315] Times=00:00:09.468__00:00:13.138
00000555 52.17190552 DetSub_Extract: Writing ... Subtitle 3, [316,358] Times=00:00:13.180__00:00:14.932
00000556 52.46656036 DetSub_Extract: Writing ... Subtitle 4, [359,410] Times=00:00:14.973__00:00:17.100
00000557 52.75485611 DetSub_Extract: Writing ... Subtitle 5, [448,498] Times=00:00:18.685__00:00:20.771
00000558 53.02936935 DetSub_Extract: Writing ... Subtitle 6, [499,543] Times=00:00:20.812__00:00:22.648

# ...

00000607 66.76883698 DetSub_Extract: Writing ... Subtitle 55, [3535,3659] Times=00:02:27.439__00:02:32.611
00000608 66.97946167 DetSub_Extract: Writing ... Subtitle 56, [3676,3705] Times=00:02:33.320__00:02:34.529
00000609 67.20022583 DetSub_Extract: Writing ... Subtitle 57, [3706,3741] Times=00:02:34.571__00:02:36.031
00000610 67.48464966 DetSub_Extract: Writing ... Subtitle 58, [3901,3947] Times=00:02:42.704__00:02:44.623
00000611 67.64614868 DetSub_Extract: Writing ... Subtitle 59, [3948,3969] Times=00:02:44.665__00:02:45.540
00000612 67.64663696 DetSub_Extract: Corr_BelowTh_Max=0.257724 Corr_AboveTh_Min=0.939325
00000613 67.69595337 DetSub_Extract: Time = 67.44Secs (1.12Mins)

[VoodooFX]

Is that timer shows elapsed time? I remember that your scripts in my tests for some reason took ~06:00 minutes (v0.0), when your extractor modded in InpaintDelogo took ~01:40. Both scripts without extractor had almost same speed ~00:30.
Maybe because of my sample? ( Sample is there - https://drive.google.com/file/d/1joXmOQtSnKMuYHWie5zbVeZn3jma3132 )
Could you share your avi sample?

EDIT:
Modded one shouldn't be faster than few percents. [It's not posted anywhere yet.]

I used my timing func:

This line goes at the start of a script:
ScriptStartTime = TimerFX()
This line goes at the end of a script:
ScriptRunTime = TimerFX(ScriptStartTime)


function TimerFX(int "startTime") {
startTime = default(startTime, 0)

a = Value(Time("%#S"))
b = Value(Time("%#M")) *60
c = Value(Time("%#H")) *3600
x = int(a+b+c)

s = (startTime == 0) ? x : x - startTime
s = (s < 0) ? 86400 - startTime + x : s

hours = floor(s / 3600)
s1 = s - hours * 3600
minutes = floor(s1/60)
seconds = s1 - minutes * 60

hh = String(hours, "%02.0f")
mm = String(minutes, "%02.0f")
ss = String(seconds, "%02.0f")

timeMs = x
timeStr = hh+":"+mm+":"+ss
out = (startTime == 0) ? timeMs : timeStr
return out
}

[StainlessS]

Sample as posted here:- https://forum.doom9.org/showthread.php?p=1949702#post1949702
However, I converted To Ut_Video avi via FFMPEG.


setlocal

REM Where to Find ffmpeg
set FFMPEG="C:\BIN\ffmpeg.exe"

REM Where to get input file, No terminating Backslash, "." = current directory
set INDIR="."


REM Where to place output file, No terminating Backslash.
set OUTDIR="D:"


FOR %%A IN (*.wmv *.mpg *.avi *.flv *.mov *.mp4 *.m4v *.RAM *.RM *.mkv *.TS *.ogv *264 *.webm *.m2v *.VOB) DO (
%FFMPEG% -i "%INDIR%\%%A" -vcodec utvideo -acodec pcm_s16le "%OUTDIR%\%%~nxA.AVI"

)

Pause

Much easier [at times] than struggling with LSMASH or ffms2.

EDIT:
Is that timer shows elapsed time?
Yes, total elapsed time. [EDIT: i7-8700 3.2GHz 6C 12T]
DetSub_Extract: Time = 67.44Secs (1.12Mins)
That is to detect sub frames, split contiguous [no clear non sub frame between subtitles] into individual subtitle sequences, average/blend individual sequence,
then binarize, optional expand, optional upsize, and then write each result search area single frame to subtitle RGB24 image.
EDIT: RGB24 result image seems to make windows 10 not hate tiff or png files. [prev probs with RGB32]
EDIT: If Expanding, may add a few seconds [I'm thinking wth maybe ExpCnt=6 about 7 seconds, maybe, not sure. also, Blend via ClipBlend (or fallback AvgAll2 thingy)]

EDIT:
My mod of your function,

Function Frame2Time(int frame, float fps,int "digits",bool "File") { # ssS, mod of VooDooFX @ https://forum.doom9.org/showthread.php?p=1951839#post1951839
ch = Default(File,false) ?"_":":"
digits = Default(digits, 3)
msTime = round(frame * 1000 / fps)
_s = msTime / 1000
_ms = msTime % 1000
_m = _s / 60
_s = _s % 60
_h = _m / 60
_m = _m % 60
hh = string(_h, "%02.0f")
mm = string(_m, "%02.0f")
ss = string(_s, "%02.0f")
ms = string(_ms,"%0"+string(digits)+".0f")
timeStr = hh+ch+mm+ch+ss+(File?"_":".")+ms
return timeStr
}

[StainlessS]

That is to detect sub frames, split contiguous [no clear non sub frame between subtitles] into individual subtitle sequences, average/blend individual sequence,
then binarize, optional expand, optional upsize, and then write each result search area single frame to subtitle RGB24 image.

Not quite true, thats how it used to work, excluding your upsize suggestion.
I totally forgot that we dont binarize it now, so that arg is likely disappearing, it dont do nuttin [AvgBinTh].
Instead, once clipblend averaged, we feed it [the blended single frame subs cropped search area, with X=Y=W=H=0] back into the DetSub_Mask() whotsit,
and get back new subs mask from that, so, the text/halo colors are averaged for the DetSub_Mask thing, and it does whatever binarizing is needed
when checking what text/halo is within tolerance.
[Took me a while to figure out how it was working without the binarize step, was a little while since I changed it, and totally forgot].

[VoodooFX]

Text1=Area.ColorKeyMask(TextCol,Text_Tol_R,Text_Tol_G,Text_Tol_B).ShowAlpha(Pixel_Type="Y8")
Halo1=Area.ColorKeyMask(HaloCol,Halo_Tol_R,Halo_Tol_G,Halo_Tol_B).ShowAlpha(Pixel_Type="Y8")


I accidentally noticed that those marked in red should be swapped.

From wiki:
ColorKeyMask(clip clip, int color [, int tolB, int tolG, int tolR])

[StainlessS]

Oops, thanks VX.

Also wiki, [ http://avisynth.nl/index.php/Layer ]

ColorKeyMask

Clears pixels in the alpha channel by comparing to a transparent color (default black).

Each pixel with a color differing less than (tolB,tolR,tolG) (default 10d) is set to transparent (black); otherwise it is left unchanged – note, it is NOT set to opaque (white). That's why you might need ResetMask before applying this filter. This behaviour allows an aggregate mask to be constructed with multiple calls to ColorKeyMask.

When tolR or tolG are not set, they get the value of tolB by default. Normally you start with a ResetMask, then chain a few calls to ColorKeyMask to cause transparent holes where each color of interest occurs. See Overlay for examples.

ColorKeyMask(clip clip, int color [, int tolB, int tolG, int tolR])

clip clip =

Source clip. Color format must be RGB32.
AVS+ also supports RGB64 and PlanarRGBA.

int color = black

Transparent color. Default black.

int tolB = 10d
int tolG = (tolB)
int tolR = (tolB)

Color tolerance. See description above.


Above function prototype, ColorKeyMask(clip clip, int color [, int tolB, int tolG, int tolR])
color is NOT optional, and so has no default value, and so cannot default to black.
[also, color not optional by testing function and is as described in wiki function prototype (or same as two lines above, in bold)]

[Reel.Deel]

Oops, thanks VX.

Also wiki, [ http://avisynth.nl/index.php/Layer ]

Above function prototype, ColorKeyMask(clip clip, int color [, int tolB, int tolG, int tolR])
color is NOT optional, and so has no default value, and so cannot default to black.
[also, color not optional by testing function and is as described in wiki function prototype (two lines above, in bold)]

There was a discussion about that here: https://forum.doom9.org/showthread.php?p=1210939#post1210939

I updated that in the yet-to-be-published updated docs for AviSynth+: https://github.com/Reel-Deal/AviSynthPlus/blob/docs-update-1/distrib/docs/english/source/avisynthdoc/corefilters/mask.rst#colorkeymask

Edit: Indeed no default, just tried ColorKeyMask() and it gives this error: "Invalid arguments to function ColorKeyMask". The source seems to suggest that when called as ColorKeyMask() color would default to 0 but that is not the case. https://github.com/AviSynth/AviSynthPlus/blob/master/avs_core/filters/layer.cpp#L420

[StainlessS]

I think maybe I'de have to go with Gavino in linked thread. [Otherwise, cannot be described in our avs style function prototype, without a whole lot more explanation].
currently,

ColorKeyMask "ci[]i[]i[]i"

Color is unnamed, and remaining int args are un-named optionals.
But, I still say,
Probably best if all args given as named optionals in code. I'm defo with Gavino .
ie,
ColorKeyMask "c[color]i[tolB]i[tolG]i[tolR]i"
Making all optional, and named, or
ColorKeyMask "ci[tolB]i[tolG]i[tolR]i"
Color un-named compulsory, others named optionals.
Then can be described in avs style prototype without extra explanation or ambiguity.

EDIT:
So avs style prototype could be either
[CODE]
ColorKeyMask(clip, int "color"=$000000, int "tolB"=10,int "tolG"=tolB,int "tolR"=tolB) # Color default=black, "c[color]i[tolB]i[tolG]i[tolR]i"

or

ColorKeyMask(clip, int color, int "tolB"=10,int "tolG"=tolB,int "tolR"=tolB) # No default for un-named color, "ci[tolB]i[tolG]i[tolR]i"