StainlessS
30th January 2022, 23:03
Single blended frame at scene change is hard to detect [EDIT: Easier if known 50:50 blend]
and usually prevents scene change detection (looks nasty too).
This is not perfect, but is good enough for me to use.
Perhaps others may have ideas to make a little more perfect. [Please say if you have better/improved method]
Scan complete clip using eg VirtualDub2, "Video Analysis Pass" to create frames file.
Clip prompting creation was PAL DVD source, originally from [I think] Canada TV [NTSC ???], perhaps originally on VHS,
seemed to be partly progressive, and partly interlaced. [dont think 'clean' enough for anything else but VHS master tape].
Blended frames were originally blended fields before doublerate deinterlacing.
Not a totally good conversion from Canadian to PAL DVD.
I am applying this after doublerate deinterlacing using QTGMC,
partly implemented in DBSC script for blended frame @ Scene change detect, and dynamic border cropping,
could not get it working as well as required so stripped out for dedicated script function, later embed back into DBSC.
Detects SINGLE frame blend ONLY, not dissolves.
No reason why it would not work with HBD converted to 8 bit. [Real result is a frame numbers txt file].
Will Even work on 8 bit RGB [internally converted to Luma-Y], but would be slow compared with external conversion to YUV.
The test Doublerate deinterlaced clip I'm using is about 72Mins long [EDIT: after trim of about 50,000 frames), and I'm guessing no more than about
15-20 false +ve blend detects, as only replaces with before or after frame, so very little damage is likely when false detect.
Its probably easiest to add/remove any frame number from txt file than try tweak args, have tried to make it autonomous
without need of user interaction. [I hope I have suceeded moderately well].
Most false +ve detects that I've noticed, were in a sequence where camera seemed to be in center of a "roundabout"
(childs playground whirligig thingy), spinning around very rapidly and pointing at surrounding walls whizzing by in a blur,
dont think I could have purposely picked a more difficult clip to test with.
EDIT: The Blend frames do not have to be 50/50 blend, can be any ratio.
Fluctuating border can badly affect detection, eg top and bottom black line after douberate deinterlace,
however, x,y,w,h defaults should avoid those.
# DetBlendFrameSC.avs
/*
*** EXPERIMENTAL ***
Req RT_Stats.
DetBlendFrameSC(clip c,String "FramesFile"="BlendFrameSC.txt",Float "dfactor"=0.90,Float "dMin"=40.0,
\ int "x"=16,int "y"=16, int "w"=-x, int "h"=-y,Bool "ClipClopIx"=False, Bool "Show"=false,
\ Bool "Force_ZOK"=False,Bool "Force_AltOK"=False)
Purpose:- To write a frames file of frame numbers that are single frame blend at scene change. (blend of last frame of scene and first frame of following scene).
These Target frames are very difficult to detect as scene changes.
Can fix blended frame using eg FrameSel or ClipClop, replace with frame before or after the blended frame.
To Replace blended frame with eg ClipClop clip 1, use a clip src.DuplicateFrame(0) [replace with previous frame], OR src.DeleteFrame(0) [replace with following frame].
We use a modified result from RT_LumaCorrelation for distance measure between frames.
See Pearson’s Distance here: http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
where Pearson's Distance is equivalent to:
PD(x,y) = 1.0 - Correlation(x,y) # PD Range 0.0 -> 2.0
However we mod to
PD(x,y) = (1.0 - Correlation(x,y)) * 255.0 # Range 0.0 -> 510.0
If detecting too many blends Increase dFactor or dMin, else too few then decrease.
See script function for further info
*/
Function DetBlendFrameSC(clip c,String "FramesFile",Float "dfactor",Float "dMin",int "x",int "y", int "w", int "h",Bool "ClipClopIx",Bool "Show",Bool "Force_ZOK",Bool "Force_AltOK") {
c
myName = "DetBlendFrameSC: "
FramesFile = default(FramesFile,"BlendFrameSC.txt")
dFactor = default(dFactor,0.9)
dMin = default(dMin,40.0)
x = default(x,16) # Test area, allow for ignoring small border/crud, as for crop.
y = default(y,16)
w = default(w,-x) # default -ve x
h = default(h,-y) # default -ve y
ClipClopIx = default(ClipClopIx,False) # True precede frame numbers in FramesFile with a ClipClop() index 1, else NO clip index written ie just frame number for eg FrameSel().
show = default(show,False)
Force_ZOK = Default(Force_ZOK,False) # Temporary debugging toggle, ZOK always TRUE (Only use AltOK in decision {tuning script - maybe removed later})
Force_AltOK = Default(Force_AltOK,False) # Temporary debugging toggle, AltOK always TRUE (Only use ZOK in decision {tuning script - maybe removed later})
Assert(FramesFile!="",myName+"FramesFile cannot be ''")
Assert(0.0 < dFactor < 10.0,myName+"0.0 < dFactor < 10.0")
Assert(0.0 <= dMin <= 50.0, myName+"0.0 <= dMin < 50.0")
FramesFile = FramesFile.RT_GetFullPathName()
FramesFile.RT_FileDelete()
FmtS = """
n = current_frame
x=%d y=%d w=%d h=%d dFactor=%f dMin=%f ClipClopIx=%s FramesFile="%s" Force_ZOK=%s Force_AltOK=%s
Prv = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n ,x=x,y=y,w=w,h=h)) * 255.0 # Distance, prev <-> current
Nxt = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+1,x=x,y=y,w=w,h=h)) * 255.0 # Distance, current <-> Next
Sum = Prv+Nxt
Z = Sum * dFactor + dMin
Alt = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n+1,x=x,y=y,w=w,h=h)) * 255.0 # Distance, Prev <-> Next, either side of current
pAl = (1 - RT_LumaCorrelation(Last,Last,n=n-2,n2=n ,x=x,y=y,w=w,h=h)) * 255.0 # Distance, either side of previous
nAl = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+2,x=x,y=y,w=w,h=h)) * 255.0 # Distance, either side of next
ZOK = (Z < Alt)
AltHICOR = 0.5
AltLOCOR = 0.65
AltHI = (1.0 - AltHICOR) * 255 # Distance 127.5.00 equivalent to correlation 0.50
AltLO = (1.0 - AltLoCOR) * 255 # Distance 89.25 equivalent to correlation 0.65
AltOK = (Alt > AltHi && (pAl < AltLo || nAl < AltLo)) # ALT must look like scenechange and at least one of prv/nxt ALTs like NOT Scene change
# AltOK helps avoid false +ve's, and nearly detects single frame blends OK all on its own.
Blend = (ZOK||Force_ZOK) && (AltOK || Force_AltOK)
(!Blend) ? NOP : ClipClopIx ? RT_WriteFile(FramesFile,"1 %%d",n,Append=True) : RT_WriteFile(FramesFile,"%%d",n,Append=True)
%s
Return Last
"""
SHOW_S = (show) ? """RT_Subtitle("%d] dfact=%.3f : dmin=%.3f\nPrv=%7.3f : Nxt=%7.3f : Sum=%7.3f\n" +
\ "pAl=%7.3f : nAl=%7.3f : Alt=%7.3f\nZ=Sum*dFact+dMin=%7.3f : ZOK=(Z<Alt)=%s\nAltOK=(Alt>%.2f&&(pAl<%.2f||nAl<%.2f))=%s\nBlend=(\a%cZOK\a- && \a%cAltOK\a-)=\a%c%s",
\ n,dFactor,dMin,Prv,Nxt,Sum,pAl,nAl,Alt,Z,ZOK,AltHi,AltLo,AltLo,AltOK,Force_ZOK?66:ZOK?33:45,Force_AltOK?66:AltOK?33:45,Blend?33:45,Blend) Blend?Subtitle("BLEND",align=5,size=48):NOP"""
\ : ""
SSS = RT_String(FmtS,x,y,w,h,dFactor,dMin,ClipClopIx,FramesFile,Force_ZOK,Force_AltOK,SHOW_S)
# RT_DebugF("%s",SSS,name=myName) RT_WriteFile(".\DetBlendFrameSC_SSS.Log","%s",SSS) # Peruse SSS, if UnComment
Return Scriptclip(SSS)
}
Client
# Import(".\DetBlendFrameSC.avs")
AviSource(".\S01E03.avi")
### CONFIG ####
trim(53550,0)
#KillAudio
SHOW = True
ClipClopIx = FALSE # False = FrameSel
FORCEWRITE = FALSE # True, force write frames file, and just show "DONE" when finished (will seem to hang for some time).
##### END CONFIG ####
SHOW = (FORCEWRITE) ? FALSE : SHOW # Switch OFF SHOW if FORCEWRITE
FramesFile=(ClipClopIx) ? "BlendFrameSC_ClipClop.txt" : "BlendFrameSC_FrameSel.txt" # Choose filename based on ClipClopIx
DetBlendFrameSC(FramesFile=FramesFile,ClipClopIx=ClipClopIx,show=SHOW)
(FORCEWRITE) ? RT_ForceProcess() : NOP
(FORCEWRITE) ? MessageClip("DONE") : NOP
return Last
Detect with Metrics
https://i.postimg.cc/gJsw1FSp/Det-Blend-Frame-SC-01.jpg (https://postimages.org/)
and usually prevents scene change detection (looks nasty too).
This is not perfect, but is good enough for me to use.
Perhaps others may have ideas to make a little more perfect. [Please say if you have better/improved method]
Scan complete clip using eg VirtualDub2, "Video Analysis Pass" to create frames file.
Clip prompting creation was PAL DVD source, originally from [I think] Canada TV [NTSC ???], perhaps originally on VHS,
seemed to be partly progressive, and partly interlaced. [dont think 'clean' enough for anything else but VHS master tape].
Blended frames were originally blended fields before doublerate deinterlacing.
Not a totally good conversion from Canadian to PAL DVD.
I am applying this after doublerate deinterlacing using QTGMC,
partly implemented in DBSC script for blended frame @ Scene change detect, and dynamic border cropping,
could not get it working as well as required so stripped out for dedicated script function, later embed back into DBSC.
Detects SINGLE frame blend ONLY, not dissolves.
No reason why it would not work with HBD converted to 8 bit. [Real result is a frame numbers txt file].
Will Even work on 8 bit RGB [internally converted to Luma-Y], but would be slow compared with external conversion to YUV.
The test Doublerate deinterlaced clip I'm using is about 72Mins long [EDIT: after trim of about 50,000 frames), and I'm guessing no more than about
15-20 false +ve blend detects, as only replaces with before or after frame, so very little damage is likely when false detect.
Its probably easiest to add/remove any frame number from txt file than try tweak args, have tried to make it autonomous
without need of user interaction. [I hope I have suceeded moderately well].
Most false +ve detects that I've noticed, were in a sequence where camera seemed to be in center of a "roundabout"
(childs playground whirligig thingy), spinning around very rapidly and pointing at surrounding walls whizzing by in a blur,
dont think I could have purposely picked a more difficult clip to test with.
EDIT: The Blend frames do not have to be 50/50 blend, can be any ratio.
Fluctuating border can badly affect detection, eg top and bottom black line after douberate deinterlace,
however, x,y,w,h defaults should avoid those.
# DetBlendFrameSC.avs
/*
*** EXPERIMENTAL ***
Req RT_Stats.
DetBlendFrameSC(clip c,String "FramesFile"="BlendFrameSC.txt",Float "dfactor"=0.90,Float "dMin"=40.0,
\ int "x"=16,int "y"=16, int "w"=-x, int "h"=-y,Bool "ClipClopIx"=False, Bool "Show"=false,
\ Bool "Force_ZOK"=False,Bool "Force_AltOK"=False)
Purpose:- To write a frames file of frame numbers that are single frame blend at scene change. (blend of last frame of scene and first frame of following scene).
These Target frames are very difficult to detect as scene changes.
Can fix blended frame using eg FrameSel or ClipClop, replace with frame before or after the blended frame.
To Replace blended frame with eg ClipClop clip 1, use a clip src.DuplicateFrame(0) [replace with previous frame], OR src.DeleteFrame(0) [replace with following frame].
We use a modified result from RT_LumaCorrelation for distance measure between frames.
See Pearson’s Distance here: http://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient
where Pearson's Distance is equivalent to:
PD(x,y) = 1.0 - Correlation(x,y) # PD Range 0.0 -> 2.0
However we mod to
PD(x,y) = (1.0 - Correlation(x,y)) * 255.0 # Range 0.0 -> 510.0
If detecting too many blends Increase dFactor or dMin, else too few then decrease.
See script function for further info
*/
Function DetBlendFrameSC(clip c,String "FramesFile",Float "dfactor",Float "dMin",int "x",int "y", int "w", int "h",Bool "ClipClopIx",Bool "Show",Bool "Force_ZOK",Bool "Force_AltOK") {
c
myName = "DetBlendFrameSC: "
FramesFile = default(FramesFile,"BlendFrameSC.txt")
dFactor = default(dFactor,0.9)
dMin = default(dMin,40.0)
x = default(x,16) # Test area, allow for ignoring small border/crud, as for crop.
y = default(y,16)
w = default(w,-x) # default -ve x
h = default(h,-y) # default -ve y
ClipClopIx = default(ClipClopIx,False) # True precede frame numbers in FramesFile with a ClipClop() index 1, else NO clip index written ie just frame number for eg FrameSel().
show = default(show,False)
Force_ZOK = Default(Force_ZOK,False) # Temporary debugging toggle, ZOK always TRUE (Only use AltOK in decision {tuning script - maybe removed later})
Force_AltOK = Default(Force_AltOK,False) # Temporary debugging toggle, AltOK always TRUE (Only use ZOK in decision {tuning script - maybe removed later})
Assert(FramesFile!="",myName+"FramesFile cannot be ''")
Assert(0.0 < dFactor < 10.0,myName+"0.0 < dFactor < 10.0")
Assert(0.0 <= dMin <= 50.0, myName+"0.0 <= dMin < 50.0")
FramesFile = FramesFile.RT_GetFullPathName()
FramesFile.RT_FileDelete()
FmtS = """
n = current_frame
x=%d y=%d w=%d h=%d dFactor=%f dMin=%f ClipClopIx=%s FramesFile="%s" Force_ZOK=%s Force_AltOK=%s
Prv = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n ,x=x,y=y,w=w,h=h)) * 255.0 # Distance, prev <-> current
Nxt = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+1,x=x,y=y,w=w,h=h)) * 255.0 # Distance, current <-> Next
Sum = Prv+Nxt
Z = Sum * dFactor + dMin
Alt = (1 - RT_LumaCorrelation(Last,Last,n=n-1,n2=n+1,x=x,y=y,w=w,h=h)) * 255.0 # Distance, Prev <-> Next, either side of current
pAl = (1 - RT_LumaCorrelation(Last,Last,n=n-2,n2=n ,x=x,y=y,w=w,h=h)) * 255.0 # Distance, either side of previous
nAl = (1 - RT_LumaCorrelation(Last,Last,n=n ,n2=n+2,x=x,y=y,w=w,h=h)) * 255.0 # Distance, either side of next
ZOK = (Z < Alt)
AltHICOR = 0.5
AltLOCOR = 0.65
AltHI = (1.0 - AltHICOR) * 255 # Distance 127.5.00 equivalent to correlation 0.50
AltLO = (1.0 - AltLoCOR) * 255 # Distance 89.25 equivalent to correlation 0.65
AltOK = (Alt > AltHi && (pAl < AltLo || nAl < AltLo)) # ALT must look like scenechange and at least one of prv/nxt ALTs like NOT Scene change
# AltOK helps avoid false +ve's, and nearly detects single frame blends OK all on its own.
Blend = (ZOK||Force_ZOK) && (AltOK || Force_AltOK)
(!Blend) ? NOP : ClipClopIx ? RT_WriteFile(FramesFile,"1 %%d",n,Append=True) : RT_WriteFile(FramesFile,"%%d",n,Append=True)
%s
Return Last
"""
SHOW_S = (show) ? """RT_Subtitle("%d] dfact=%.3f : dmin=%.3f\nPrv=%7.3f : Nxt=%7.3f : Sum=%7.3f\n" +
\ "pAl=%7.3f : nAl=%7.3f : Alt=%7.3f\nZ=Sum*dFact+dMin=%7.3f : ZOK=(Z<Alt)=%s\nAltOK=(Alt>%.2f&&(pAl<%.2f||nAl<%.2f))=%s\nBlend=(\a%cZOK\a- && \a%cAltOK\a-)=\a%c%s",
\ n,dFactor,dMin,Prv,Nxt,Sum,pAl,nAl,Alt,Z,ZOK,AltHi,AltLo,AltLo,AltOK,Force_ZOK?66:ZOK?33:45,Force_AltOK?66:AltOK?33:45,Blend?33:45,Blend) Blend?Subtitle("BLEND",align=5,size=48):NOP"""
\ : ""
SSS = RT_String(FmtS,x,y,w,h,dFactor,dMin,ClipClopIx,FramesFile,Force_ZOK,Force_AltOK,SHOW_S)
# RT_DebugF("%s",SSS,name=myName) RT_WriteFile(".\DetBlendFrameSC_SSS.Log","%s",SSS) # Peruse SSS, if UnComment
Return Scriptclip(SSS)
}
Client
# Import(".\DetBlendFrameSC.avs")
AviSource(".\S01E03.avi")
### CONFIG ####
trim(53550,0)
#KillAudio
SHOW = True
ClipClopIx = FALSE # False = FrameSel
FORCEWRITE = FALSE # True, force write frames file, and just show "DONE" when finished (will seem to hang for some time).
##### END CONFIG ####
SHOW = (FORCEWRITE) ? FALSE : SHOW # Switch OFF SHOW if FORCEWRITE
FramesFile=(ClipClopIx) ? "BlendFrameSC_ClipClop.txt" : "BlendFrameSC_FrameSel.txt" # Choose filename based on ClipClopIx
DetBlendFrameSC(FramesFile=FramesFile,ClipClopIx=ClipClopIx,show=SHOW)
(FORCEWRITE) ? RT_ForceProcess() : NOP
(FORCEWRITE) ? MessageClip("DONE") : NOP
return Last
Detect with Metrics
https://i.postimg.cc/gJsw1FSp/Det-Blend-Frame-SC-01.jpg (https://postimages.org/)