Detecting blinking Timestamp

[scharfis_brain]

Heya,

I've got a captured VHS video containing a blinking timestamp.
The timestamp doesn't change since the clock-battery of the camcorder was empty. The timestamp is present for approx 1 second and absent for another second.

I want to reliably detect the timestamp when it occurs.
(When it is being detected, I'll cover it with a mask followed by interpolation stuff.)

Here you'll find a cropped example (It is field-separated VHS followed by selectevery(25,1) ):
https://easyupload.io/w0wlqw

I hope that someone is able to help me out.

[johnmeyer]

I've created a lot of scripts to detect big changes from one frame to the next, including detecting a photographer flash; detecting black frames; detecting dropped frames (tough, but doable); and detecting scene changes.

There are some functions that StainlessS has developed, but most of the time I find that the built-in AVISynth YDifference functions do the best job. They simply sum the difference in pixel luma value, for each pixel location to the same pixel location in an adjacent frame, either looking forward to the next frame or backwards to the previous frame.

If I were doing your job, I'd first mask the entire video, leaving only the portion where the timestamp appears. I'd then apply a YDifference function to just that small area and compare it to an adjacent frame. When the timestamp either appears or disappears, you should get a massive jump in the YDifference value.

One other thing I've done when just that simple test isn't reliable enough is to create a moving average of YDifference values and then compare the YDifference (to next or previous) to that moving average. I do this because some portions of the video have a lot of movement, and therefore the YDifference values will go up when that happens. All you are trying to do is detect when the YDifference gets large compared to what is "normal" for the nearby video.

You know what you're doing, but if you need code, you can either search on my name or I can post some code snippets.

[ChaosKing]

My simple attempt in Vapoursynth. Works surprisingly well except for some scenes where the clock is blending with the rest of the video like the car around frame 420
Sample https://filehorst.de/d/dgfEJadh


import vapoursynth as vs
import mvsfunc as mvf
import functools
core = vs.core

clip = core.lsmas.LWLibavSource(source=r"D:\Invalid-Timestamp.avi")

clock = mvf.ToRGB(clip)
clock = core.std.CropRel(clock, left=202, top=38, right=91, bottom=36).flux.SmoothT(temporal_threshold=9)


def NaiveDetect(n, f, clip, clock ):

clip = core.text.Text(clip, text="Min: "+str(f.props.PlaneStatsMin)
+"\nMax: " + str(f.props.PlaneStatsMax)
+"\nAvg: " + str(f.props.PlaneStatsAverage)[0:7] )

if (f.props.PlaneStatsMin + f.props.PlaneStatsAverage) < 2 and f.props.PlaneStatsMax > 210:
return core.text.Text(clip, text="CLOCK", alignment=2)
else:
return clip

clip = clip.std.FrameEval(functools.partial(NaiveDetect, clip=clip, clock=clock), prop_src=clock.std.PlaneStats())

[wonkey_monkey]

Personally I would use a fixed mask without worrying about when the timestamp disappears. It's fairly narrow text, so should be hidden quite well by most delogo filters, and switching it on and off will only draw more attention to the area.

[scharfis_brain]

@johnmeyer: I'll give that a shot.
I tried using edgemasks before. But this wasn't satisfying.

@ChaosKing: Hmm, now I have to figure out VapourSynth ;-) Newer used it before.

@wonkey_monkey: I will use Deshaker to temporally fill in the gaps. Since the footage is very shaky it will benefit from temporally unmasking the area of the timestamp when the timestamp is not present.

[StainlessS]

This nearly works in avs [as per CK post, but detector does not fire], needs some added chaos from the king. [I dont know VS]

What it looks like, Chaos King test area bounded by red. [Re-Done after fix, removed AverageLuma]

# Fixed with AveLuma removed
clip = AviSource(".\Invalid-Timestamp.avi").ConvertToYV24
clock = clip.Crop(202,38,-91,-36).FluxsmoothT(temporal_threshold=9)

SSS = """
n = current_frame # Frame number
ymin = clock.YPlaneMin # Testing CLOCK not clip
ymax = clock.YPlaneMax # Ditto
IsClock = (Ymin < 2 && Ymax > 210)
Subtitle(String(n,"%.0f] ")+((IsClock) ? " *** CLOCK ***\n" : "\n") + String(Ymin," Mn=%3.0f") + String(Ymax," : Mx=%3.0f"),lsp=0,Font="Courier New")
return Last
"""
clockShow=clock.Addborders(2,2,2,2,$FF0000) # Surround test area in red border for display
clockShow=clockShow.Addborders(0,2*20,clip.width-clockShow.width,2) # Add space for 2 lines of text and extend to same width as clip
clip=StackVertical(clockShow,clip) # Stack display area above clip
Clip.Scriptclip(SSS)


No clock
https://i.postimg.cc/xTr14z3h/Choas-King-00.jpg (https://postimages.org/)

Clock
https://i.postimg.cc/PxtRx2H9/Choas-King-01.jpg (https://postimages.org/)

[ChaosKing]

Remove the Yave and it should work

IsClock = (Ymin < 2 && Ymax > 210)
Subtitle(String(n,"%.0f ")+((IsClock) ? " *** CLOCK ***\n" : "\n") + String(Yave,"Av=%.3f") + String(Ymin," Mn=%.3f") + String(Ymax," Mx=%.3f"),lsp=0)

[StainlessS]

Yep works fine, thanks.

script fixed.

[ChaosKing]

clock = mvf.ToRGB(adjust.Tweak(clip, bright=20))

[...]
if (f.props.PlaneStatsMin < 10 and f.props.PlaneStatsMax > 250)

With these changes it is almost perfect... but I get different values in the avisynth version. Happy testing.

[StainlessS]

Without the tweak, seems that the artificial clock halo is always 1 [darkest pixels of it], and is the main detect item (in avs, presume other dark pixels in pic are 16->235).

EDIT: Also presume that Clock halo deliberately 1 [ie lowest without being 0], where 0 has special meaning [vertical blanking flyback or something].

EDIT: There is the occasional NON clock pixel that is also YMIn=1, eg Frame 585 (but YMAX=135 so NON detect OK).

[StainlessS]

EDIT: There is the occasional NON clock pixel that is also YMIn=1, eg Frame 585 (but YMAX=135 so NON detect OK).

Dont know if my below logic is messed up or not, but think can possibly apply some additional check on CLOCK detect as per prev post using ChaosKing logic.
Seems to me that YPlaneMedian should be within a certain range given that where CLOCK present then there are a minimum number of low Luma pixels(HALO), and
also a minumum number of high luma value pixels(TEXT). So, if YPlaneMedian is below some value, or above some value, then CLOCK is not present.

Is my logic correct here, or do I need a little more sleep ??? [EDIT: If wrong then AverageLuma should be within certain range instead, or maybe both]

Here script to additionally show YPlaneMedian for current frame clock detect area, and also min and max YPlaneMedian encountered so far, and where CLOCK was detected.
On Full scan on ONLY frames where clock detected, shows minimum YPlaneMedian as 41, and max YPlaneMedian as179,
so maybe additional check to clock detect should Override to NOT CLOCK if yplanemedian is below 41 or above 179 [or massage it a bit to below (41-2)=39 and above (179+2)=181].


# Show YMedian min and max values, and also where clock detected
clip = AviSource(".\Invalid-Timestamp.avi").ConvertToYV24
clock = clip.Crop(202,38,-91,-36).FluxsmoothT(temporal_threshold=9)

Global MEDLO= 256
Global MEDHI= -1
Global CMEDLO= 256
Global CMEDHI= -1

SSS = """
n = current_frame # Frame number
ymin = clock.YPlaneMin # Testing CLOCK not clip
ymax = clock.YPlaneMax # Ditto
ymed = clock.YPlaneMedian # Ditto
IsClock = (Ymin < 2 && Ymax > 210)

if(ymed<MEDLO) {
Global MEDLO=yMed
}
if(ymed>MEDHI) {
Global MEDHI=yMed
}
if(IsClock) {
if(ymed<CMEDLO) {
Global CMEDLO=yMed
}
if(ymed>CMEDHI) {
Global CMEDHI=yMed
}
}
Subtitle(String(n,"%.0f] ") + ((IsClock) ? " *** CLOCK ***\n" : "\n") + String(Ymin,"Mn=%3.0f") + String(Ymax," : Mx=%3.0f") + String(Ymed," : Md=%3.0f") +
\ String(MEDLO, "\nLO=%3.0f") + String(MEDHI, " : HI=%3.0f") + "(YMedian ALL)" +
\ String(CMEDLO,"\nLO=%3.0f") + String(CMEDHI," : HI=%3.0f") + "(YMedian CLOCK )",lsp=0,Font="Courier New")

return Last
"""
clockShow=clock.Addborders(2,2,2,2,$FF0000) # Surround test area in red border for display
clockShow=clockShow.Addborders(0,4*20,clip.width-clockShow.width,2) # Add space for 4 lines of text and extend to same width as clip
clip=StackVertical(clockShow,clip) # Stack display area above clip.
Clip.Scriptclip(SSS)


Image on full scan, last frame.
https://i.postimg.cc/4yWDLF5r/Choas-King-00.jpg (https://postimages.org/)

EDIT: So IsClock maybe could be detected something like this

TOL = 2
IsClock = (Ymin < 2 && Ymax > 210) && (YMed >= (41-TOL) && YMed <= (179+TOL))


EDIT:
Here Frame 585 where YMin=1 YMax=135, so Ymax 135<= 210 Does not detect clock, but additionally YMed=62 also would override to not detect.
EDIT: "but additionally YMed=62 also would override to not detect"
RUBBISH, would not override here, I do need some sleep.
https://i.postimg.cc/v8NWj3Y2/Choas-King-01.jpg (https://postimages.org/)

[StainlessS]

Experimental version with optional Overrides for YPlaneMedian and AverageLuma, also shows Flags for 'C' Clock detected[EDIT: Pre-Override], 'M' Median override enabled, 'A' AvergeLuma override enabled,
and 'O' Overriden by either of the enabled overrides. Also shows Override count so far.

EDIT: Changed Flagging for 'M' and 'A', 'M' and 'A' show if in legal Clock range and relevant Override enabled, even if 'C' clock detect flag is not shown.

# Show YMedian min and max values, and also where clock detected
clip = AviSource(".\Invalid-Timestamp.avi").ConvertToYV24
clock = clip.Crop(202,38,-91,-36).FluxsmoothT(temporal_threshold=9)

CLK_MIN = 1 # YPlaneMin Below or equal possible CLOCK
CLK_MAX = 211 # YPlaneMax Above or equal possible CLOCK

DET_WITH_MED_TOL=true # Detect override using YPlaneMedian with Tolerance
MEDLOLIM = 41 # Min val for Clk YPlaneMedian
MEDHILIM = 179 # Max val for Clk YPlaneMedian
MEDTOL = 2 # Additional tolerance for YPlaneMedian check

DET_WITH_AVE_TOL=true # Detect override using AverageLuma with Tolerance
AVELOLIM = 47.0 # Min val for Clk AverageLuma
AVEHILIM = 156.36 # Max val for Clk AverageLuma
AVETOL = 2.0 # Additional tolerance for AverageLuma check

# Globals, extremes encountered so far in clip
Global MEDLO= 256
Global MEDHI= -1
Global CMEDLO= 256
Global CMEDHI= -1
Global AVELO= 256
Global AVEHI= -1
Global CAVELO= 256
Global CAVEHI= -1

Global TotalClks =0 # Count of CLOCK frames detected
Global OverrideCnt = 0 # Count of clock detects overriden by either Median or Average

SSS = """
n = current_frame # Frame number
ymin = clock.YPlaneMin # Testing CLOCK not clip
ymax = clock.YPlaneMax # Ditto
ymed = clock.YPlaneMedian # Ditto
yave = clock.AverageLuma # Ditto

IsClk = (Ymin <= CLK_MIN && Ymax >= CLK_MAX)
MedIsCLK = (YMed >= MEDLOLIM-MEDTOL) && (YMed <= MEDHILIM+MEDTOL)
AveIsCLK = (YAve >= AVELOLIM-AVETOL) && (YAve <= AVEHILIM+AVETOL)

IsClock = IsClk && (!DET_WITH_MED_TOL || MedIsCLK) && (!DET_WITH_AVE_TOL || AveIsCLK)
Global OverrideCnt = (IsClock!=IsClk) ? OverrideCnt + 1 : OverrideCnt # Overriden by either Med or Ave Flagged as 'O'(some overriden)

if(yave<AVELO) {
Global AVELO=yave
}
if(yave>AVEHI) {
Global AVEHI=yave
}
if(IsClock) {
if(yave<CAVELO) {
Global CAVELO=yMed
}
if(yave>CAVEHI) {
Global CAVEHI=yave
}
}
if(ymed<MEDLO) {
Global MEDLO=yMed
}
if(ymed>MEDHI) {
Global MEDHI=yMed
}
if(IsClock) {
if(ymed<CMEDLO) {
Global CMEDLO=yMed
}
if(ymed>CMEDHI) {
Global CMEDHI=yMed
}
}

Global TotalClks = (IsClock) ? TotalClks + 1 : TotalClks # Incr global count if detected

FLGS=((IsClk)?"C":"-")+((MedIsClk&&DET_WITH_MED_TOL)?"M":"-")+((AveIsClk&&DET_WITH_AVE_TOL)?"A":"-") + ((IsClock!=IsClk)?"O":"-")
Subtitle(String(n,"%.0f] ") + FLGS + String(TotalClks," : Clks=%.0f") + ((IsClock) ? " * CLOCK *\n" : "\n") +
\ String(Ymin,"Mn=%3.0f") + String(Ymax,":Mx=%3.0f") + String(Ymed,":Md=%3.0f") + String(Yave,":Av=%.3f") +
\ String(MEDLO, "\nMLO=%3.0f") + String(MEDHI, ":MHI=%3.0f") + "(YMedian ALL)" +
\ String(CMEDLO,"\nMLO=%3.0f") + String(CMEDHI,":MHI=%3.0f") + "(YMedian CLK )" +
\ String(AVELO, "\nALO=%7.3f") + String(AVEHI, ":AHI=%7.3f") + "(YAve ALL)" +
\ String(CAVELO,"\nALO=%7.3f") + String(CAVEHI,":AHI=%7.3f") + "(YAve CLK)" +
\ String(OverrideCnt,"\nOverrideCnt=%.0f")
\ ,lsp=0,Font="Courier New"
\)

return Last
"""
clockShow=clock.Addborders(2,2,2,2,$FF0000) # Surround test area in red border for display
clockShow=clockShow.Addborders(0,7*20,clip.width-clockShow.width,2) # Add space for 7 lines of text and extend to same width as clip
clip=StackVertical(clockShow,clip) # Stack display area above clip.
Clip.Scriptclip(SSS)

On last frame, both overrides enabled [EDIT: and also both in OK non-override range], no overriden detects found. (but this is only small sample of frames).
https://i.postimg.cc/LXwjq3s8/Choas-King-00.jpg (https://postimages.org/)

EDIT: 2nd last frame, with clock [EDIT: and also both in OK non-override range]
https://i.postimg.cc/0NFfkpVL/Choas-King-02.jpg (https://postimages.org/)

EDIT: Above, cropping on detect area may have chopped off some of clock halo, if cropping changed then will need modify override numbers limits for Ymedian and Averageluma.
EDIT: Overrides so that a couple of single stray noise pixels will not upset the apple cart. [clip exhibits some VHS noise]

[ChaosKing]

I had another idea. What if we remove the clock with a good delogo plugin. After that a "diff-value is calculated" between the original and the delogo clip. A clock frame should then in theory show a higher diff value. Could this work?

[StainlessS]

Maybe but I think your clock crop coords chop off some halo [would also need a bit of extra size for noise around clock]

EDIT: Does the YPlaneMedia override sound OK to you [methinks avergeLuma override is a bit flawed (but still seems to work)].

EDIT: YPlaneMin=1 alone, pretty much nails it most of the time [but there is VHS noise and this only small sample of frames].

EDIT: Milk is sour (EDIT: well will be tomorrow), me gotta risk goin' out to the shops, wish me luck.

[scharfis_brain]

but there is VHS noise and this only small sample of frames]

I could upload an undecimated file with all 50 fields per second, if you like.


@all: Thanks for your effort! I hope I've got some time during the easter holidays to try your suggestions.

[StainlessS]

I could upload an undecimated file with all 50 fields per second, if you like.
Please dont, me is quite happy for you to do further tests. :)
You can easily switch off either of the overrides.

If you change crop coords to include chopped off halo, then, switch off both overrides, and use the new limits shown in metrics,
ie above metrics CLK lines

MEDLOLIM = 41 # Min val for Clk YPlaneMedian
MEDHILIM = 179 # Max val for Clk YPlaneMedian

AVELOLIM = 47.0 # Min val for Clk AverageLuma
AVEHILIM = 156.36 # Max val for Clk AverageLuma


Maybe just do scan on this small sample again, then enable Overrides and scan the full clip (see if any were overridden).

Got some milk, beer and cider, so I'm sorted for a while.

EDIT: Let us know if you had complete success, was ChaosKing detector that does the job, Nice one CK :)

EDIT:
EDIT:
Here Frame 585 where YMin=1 YMax=135, so Ymax 135<= 210 Does not detect clock, but additionally YMed=62 also would override to not detect.

RUBBISH, would not override here, I do need some sleep.
Would only Override if YMin[YMin<=1] <= YMed < (MEDLOLIM-MEDTOL ie 41-2) OR (MEDHILI+MEDTOL ie 179+2) < YMed <= YMax[YMax>=211] [allowing for YMed==YMin or YMed==YMax, I think ???]

[ChaosKing]

I think I got it to 100% detection rate (have not found a single miss detection yet) :D

clip = adjust.Tweak(clip, bright=18) # propably not nececcery
clock = mvf.ToRGB(clip)

clock = core.std.CropRel(clock, left=202, top=38, right=91, bottom=36).flux.SmoothT(temporal_threshold=9)

clock_frame = clock.std.FreezeFrames(first=0, last=clip.num_frames-1, replacement=300) # our reference frame for the clock mask
clock_mask = clock_frame.std.Binarize()###.std.Inflate() make frame black & white

blank = core.std.BlankClip(clock ) # blank clip
c3 = clock.std.Prewitt() # detect edges !!
clock3 = core.std.MaskedMerge( blank, c3 ,clock_mask) # cut out everything not clock related = content around the clock


# now we use only the Avg to detect the clock. (# Min is always 0 Max most of the time 255)
if f.props.PlaneStatsAverage > 0.22:
# rest of the code is the same

clip = haf.Overlay(clip,mvf.ToYUV(clock3), x=200) # show the detector


Gif https://imgur.com/a/uuYhA4n
complete sample: https://filehorst.de/d/dtqEoqbJ

[StainlessS]

Looks good to me, although I have to admit to blinking more than once.

clip = AviSource(".\Invalid-Timestamp.avi").ConvertToYV24
clock = clip.Crop(202,38,-91,-36).FluxsmoothT(temporal_threshold=9)
CK=FFVideoSource(".\detect.mkv")

CLK_MIN = 1 # YPlaneMin Below or equal possible CLOCK
CLK_MAX = 211 # YPlaneMax Above or equal possible CLOCK

DET_WITH_MED_TOL=true # Detect override using YPlaneMedian with Tolerance
MEDLOLIM = 41 # Min val for Clk YPlaneMedian
MEDHILIM = 179 # Max val for Clk YPlaneMedian
MEDTOL = 2 # Additional tolerance for YPlaneMedian check

DET_WITH_AVE_TOL=true # Detect override using AverageLuma with Tolerance
AVELOLIM = 47.0 # Min val for Clk AverageLuma
AVEHILIM = 156.36 # Max val for Clk AverageLuma
AVETOL = 2.0 # Additional tolerance for AverageLuma check

# Globals, extremes encountered so far in clip
Global MEDLO= 256
Global MEDHI= -1
Global CMEDLO= 256
Global CMEDHI= -1
Global AVELO= 256
Global AVEHI= -1
Global CAVELO= 256
Global CAVEHI= -1

Global TotalClks =0 # Count of CLOCK frames detected
Global OverrideCnt = 0 # Count of clock detects overriden by either Median or Average

SSS = """
n = current_frame # Frame number
ymin = clock.YPlaneMin # Testing CLOCK not clip
ymax = clock.YPlaneMax # Ditto
ymed = clock.YPlaneMedian # Ditto
yave = clock.AverageLuma # Ditto

IsClk = (Ymin <= CLK_MIN && Ymax >= CLK_MAX)
MedIsCLK = (YMed >= MEDLOLIM-MEDTOL) && (YMed <= MEDHILIM+MEDTOL)
AveIsCLK = (YAve >= AVELOLIM-AVETOL) && (YAve <= AVEHILIM+AVETOL)

IsClock = IsClk && (!DET_WITH_MED_TOL || MedIsCLK) && (!DET_WITH_AVE_TOL || AveIsCLK)
Global OverrideCnt = (IsClock!=IsClk) ? OverrideCnt + 1 : OverrideCnt # Overriden by either Med or Ave Flagged as 'O'(some overriden)

if(yave<AVELO) {
Global AVELO=yave
}
if(yave>AVEHI) {
Global AVEHI=yave
}
if(IsClock) {
if(yave<CAVELO) {
Global CAVELO=yMed
}
if(yave>CAVEHI) {
Global CAVEHI=yave
}
}
if(ymed<MEDLO) {
Global MEDLO=yMed
}
if(ymed>MEDHI) {
Global MEDHI=yMed
}
if(IsClock) {
if(ymed<CMEDLO) {
Global CMEDLO=yMed
}
if(ymed>CMEDHI) {
Global CMEDHI=yMed
}
}

Global TotalClks = (IsClock) ? TotalClks + 1 : TotalClks # Incr global count if detected

FLGS=((IsClk)?"C":"-")+((MedIsClk&&DET_WITH_MED_TOL)?"M":"-")+((AveIsClk&&DET_WITH_AVE_TOL)?"A":"-") + ((IsClock!=IsClk)?"O":"-")
Subtitle(String(n,"%.0f] ") + FLGS + String(TotalClks," : Clks=%.0f") + ((IsClock) ? " * CLOCK *\n" : "\n") +
\ String(Ymin,"Mn=%3.0f") + String(Ymax,":Mx=%3.0f") + String(Ymed,":Md=%3.0f") + String(Yave,":Av=%.3f") +
\ String(MEDLO, "\nMLO=%3.0f") + String(MEDHI, ":MHI=%3.0f") + "(YMedian ALL)" +
\ String(CMEDLO,"\nMLO=%3.0f") + String(CMEDHI,":MHI=%3.0f") + "(YMedian CLK )" +
\ String(AVELO, "\nALO=%7.3f") + String(AVEHI, ":AHI=%7.3f") + "(YAve ALL)" +
\ String(CAVELO,"\nALO=%7.3f") + String(CAVEHI,":AHI=%7.3f") + "(YAve CLK)" +
\ String(OverrideCnt,"\nOverrideCnt=%.0f")
\ ,lsp=0,Font="Courier New"
\)

return Last
"""
clockShow=clock.Addborders(2,2,2,2,$FF0000) # Surround test area in red border for display
clockShow=clockShow.Addborders(0,7*20,clip.width-clockShow.width,2) # Add space for 7 lines of text and extend to same width as clip
clip=StackVertical(clockShow,clip) # Stack display area above clip.

Clip.Scriptclip(SSS)

StackVertical(CK.convertToYV24.AddBorders(0,0,clip.width-clockShow.width,0),Last)


No clock [CK 100% detector on top, Taken from CK linked MKV clip]
https://i.postimg.cc/Vkjhw4R1/Choas-King-02.jpg (https://postimages.org/)

Clock
https://i.postimg.cc/hjsQR9vq/Choas-King-03.jpg (https://postimages.org/)

Nice Job CK.

EDIT: I dont fancy converting that into AVS, perhaps you could do it for Scharfis :)

[ChaosKing]

AVS version incomplete.
clip = AviSource(".\Invalid-Timestamp.avi").ConvertToYV24
clip = clip.Tweak(bright = 20)
clock = clip.Crop(202,38,-91,-36).FluxsmoothT(temporal_threshold=9)

clock_frame = clock.FreezeFrame(0, clock.framecount(),300)
clock_mask = clock_frame.mt_Binarize()

blank = BlankClip(clock)
c3 = clock.mt_edge(mode="prewitt") <-- output in avs is different from vapoursynth std.Prewitt()
clock3 = mt_merge(blank, c3, clock_mask)


SSS = """
n = current_frame # Frame number
ymin = clock3.YPlaneMin # Testing CLOCK not clip
ymax = clock3.YPlaneMax # Ditto
yave = clock3.AverageLuma # Ditto
IsClock = (Ymin < 30 && Ymax > 250) # TODO
Subtitle(String(n,"%.0f ")+((IsClock) ? " *** CLOCK ***\n" : "\n") + String(Yave,"Av=%.3f") + String(Ymin," Mn=%.3f") + String(Ymax," Mx=%.3f"),lsp=0)
return Last
"""
clockShow=clock3.Addborders(2,2,2,2,$FF0000) # Surround test area in red border for display
clockShow=clockShow.Addborders(0,2*20,clip.width-clockShow.width,2) # Add space for 2 lines of text and extend to same width as clip
clip=StackVertical(clockShow,clip) # Stack display area above clip
Clip.Scriptclip(SSS)

Does someone know what the equivalent of std.Prewitt() in Avisynth would look like?

[StainlessS]

Is it not the mt_edge thing, http://avisynth.nl/index.php/MaskTools2/Mt_edge


mt_edge (clip, string "mode", int "thY1", int "thY2", int "thC1", int "thC2", int "Y", int "U", int "V", string "chroma", int "offX", int "offY" int "w", int "h")

# ...

string mode = "sobel"

mode chooses the 3x3 convolution kernel used for the mask computing.
There are 7 predefined kernel, "sobel", "roberts", "laplace", "cartoon", "min/max", "hprewitt", and "prewitt".
Additionally, you can also enter also a custom 3x3 kernel. The normalization factor of the kernel is automatically
computed and ceiled to the closest power of 2, to allow faster processing. You can specify your own normalization
factor by adding it to the list of coefficients ( "1 1 1 1 -8 1 1 1 1 8" for example )

# ...

"hprewitt" is equivalent to:

mt_logic(mt_edge("1 2 1 0 0 0 -1 -2 -1 1"), mt_edge("1 0 -1 2 0 -2 1 0 -1 1"), mode="max")

"prewitt" is a more robust kernel and is equivalent to:

mt_logic(mt_logic(mt_edge("1 1 0 1 0 -1 0 -1 -1 1"),mt_edge("1 1 1 0 0 0 -1 -1 -1 1"),mode="max"),mt_logic(mt_edge("1 0 -1 1 0 -1 1 0 -1 1"),mt_edge("0 -1 -1 1 0 -1 1 1 0 1"),mode="max"),mode="max")



EDIT: So something like

# c3 = clock.std.Prewitt() # detect edges !!
c3=clock.mt_edge("prewitt")

[ChaosKing]

# c3 = clock.std.Prewitt() # detect edges !!
c3=clock.mt_edge("prewitt")

That is what I use in my script (the bold font :cool:)
But the output is very different from std.Prewitt(). It does not detect a single edge.

[StainlessS]

# c3 = clock.std.Prewitt() # detect edges !!
c3=clock.mt_edge("prewitt")

That is what I use in my script (the bold font :cool:)
But the output is very different from std.Prewitt(). It does not detect a single edge.

OK (kicks ones self --- I was x-lating from original VS script, not the Avs script where you already tried it).

EDIT: Did you try the "hprewitt" whotsit ? [EDIT: "hprewitt" not much different]

[StainlessS]

ChaosKing,

try below, ThY1, 0=always solid clock. Starts to sometimes work about ThY=85 but way higher than default 10. 200 sort of works OK but not always [eg frame 29].

clip = AviSource(".\Invalid-Timestamp.avi").ConvertToYV24
clip = clip.Tweak(bright = 20)
clock = clip.Crop(202,38,-91,-36).FluxsmoothT(temporal_threshold=9)

clock_frame = clock.FreezeFrame(0, clock.framecount(),300)
clock_mask = clock_frame.mt_Binarize()

blank = BlankClip(clock)

#c3 = clock.mt_edge(mode="prewitt") # <-- output in avs is different from vapoursynth std.Prewitt()

ThY1 = 200 # try ThY1 > 10(default)
ThY2 = ThY1

c3 = clock.mt_edge(mode="prewitt",thY1=ThY1,thY2=ThY2) # <-- output in avs is different from vapoursynth std.Prewitt()

clock3 = mt_merge(blank, c3, clock_mask)

SSS = """
n = current_frame # Frame number
ymin = clock3.YPlaneMin # Testing CLOCK not clip
ymax = clock3.YPlaneMax # Ditto
yave = clock3.AverageLuma # Ditto
IsClock = (Ymin < 30 && Ymax > 250) # TODO
Subtitle(String(n,"%.0f ")+((IsClock) ? " *** CLOCK ***\n" : "\n") + String(Yave,"Av=%.3f") + String(Ymin," Mn=%.3f") + String(Ymax," Mx=%.3f"),lsp=0)
return Last
"""
clockShow=clock3.Addborders(2,2,2,2,$FF0000) # Surround test area in red border for display
clockShow=clockShow.Addborders(0,2*20,clip.width-clockShow.width,2) # Add space for 2 lines of text and extend to same width as clip
clip=StackVertical(clockShow,clip) # Stack display area above clip
Clip.Scriptclip(SSS)

[ChaosKing]

Looks good with: IsClock = (yave > 58)

EDIT
Fails on frame 2693, while VS is ok

EDIT2
ThY1 = 222
(yave > 42)

[StainlessS]

Dont know if recent mod to mt_edge, maybe try with old masktools2 but maybe need YV12 and mod for even coords.

Gonna get into the feathers now, good night.

EDIT: Docs say this


mt_edge : string mode("sobel"), int thY1(10), int thY2(10), int thC1(10), int thC2(10)

# ...

thX1 is the low threshold and thX2 the high threshold. Under thX1, the pixel is set to zero, over thX2, to 255, and inbetween, left untouched.


where presumably X of eg thX1 means either "ThY1" or ThC1".

[ChaosKing]

The edges look better if I use ConvertToPlanarRGB() before prewitt instead of yv24() => that means I get also a higher Avg value now.
c3 = clock.ConvertToPlanarRGb().mt_edge(mode="prewitt",thY1=ThY1,thY2=ThY2)

With
ThY1 = 30
IsClock = (yave > 94)
Avg is now always around 95-100. I think we got it now.

p.s. I'm too lazy to test YV12 and mod with even coords :D

[StainlessS]

I've changed to this (was wanting to use black halo too) clock = clip.Crop(195,34,56,38)
is centered with small gap around,
BUT, viewing only cropped area, jump to eg 2419, clock is shifted right !

Looking for better mask frames,
122 Light
372 Dark
2004 Light
2517 Dark

EDIT: 2761, shift back to center again. 2824 shift right again.

Prewitt sometimes used like this:- https://forum.doom9.org/showthread.php?p=1879026#post1879026

### Gibbs Noise Block ###
Edge=MT_Edge("prewitt",ThY1=20,ThY2=40).RemoveGrain(17)


Prewitt exposé by Fluffy[it is different in AVS compared to VS]:- https://forum.doom9.org/showthread.php?p=1839205#post1839205

[ChaosKing]

Aha, good to now that they are diffrent implementation of prewitt. All makes sense now.

[ChaosKing]

Instead or removegrain I tried a blur+binarize combo

clock.ConvertToPlanarRGb().mt_edge(mode="prewitt",thY1=ThY1,thY2=ThY2).blur(1.58).blur(1.58).mt_Binarize(220)

the high thresh of binarize keeps only the almost white stuff.

StainlessS, I think you had some kind of frame surgeon function which can work with a list of frames did you?

[StainlessS]

StainlessS, I think you had some kind of frame surgeon function which can work with a list of frames did you?

Sorry did not see the edit/last line [or I meant to come back to it].

I got lots of stuff that can work with a list of frames, what kind of "work with" are we talking about ?

Clipclop does frame replacement, that it ? [can eg process entire clip using some filter, and only replace the filtered frames for list of frame numbers].
max 255 filtered clips possible.

Clipclop:- https://forum.doom9.org/showthread.php?t=162266

FrameSurgeon can do frame replacements with same frame from a filtered version of source [or other] clip, or with eg frame before or after target frame, or interpolate a frame. [EDIT: Also delete frame]
[255 replacement clips using clipclop]

FrameSel, can pull frames list out of a clip (together with optional number frames from either side of target frames) and put them back where it got them from,
using same frames list. [Or select all frames NOT in frames list, ie Reject the frames rather than select them]

EDIT: ClipCLop example, can use clip indexes rather than NickNames, and CMD file of frame numbers instead of SCMD multi-line string of numbers.
However, below requires clip index and frame number(or range).
If only single filter clip required, then could be hacked to work using FrameSel functions [EDIT: ie no clip index needed, just frame number or range].

Example usage script using NickNames:
###
Avisource("D:\avs\test.avi")
ORG=Last

V1 = FFT3DFilter(Plane=0,Sigma=1.6) # Light Luma
V2 = FFT3DFilter(Plane=0,Sigma=2.0) # Med Luma
V3 = FFT3DFilter(Plane=0,Sigma=4.0) # High Luma
V4 = FFT3DFilter(Plane=3,Sigma=1.6) # Light Chroma
V5 = FFT3DFilter(Plane=3,Sigma=2.0) # Med Chroma
V6 = FFT3DFilter(Plane=3,Sigma=4.0) # High Chroma
V7 = FFT3DFilter(Plane=4,Sigma=1.6) # Light Luma+Chroma
V8 = FFT3DFilter(Plane=4,Sigma=2.0) # Med Luma+Chroma
V9 = FFT3DFilter(Plane=4,Sigma=4.0) # High Luma+Chroma
V10= FlipHorizontal() # Flip-H
V11= FlipVertical() # Flip-V
V12= Invert() # Invert

NickNames =""" # Psuedonyms for clips (clip index number)
L0 = 1 # Light Luma
L1 = 2 # Med Luma
L2 = 3 # High Luma
C0 = 4 # Light Chroma
C1 = 5 # Med Chroma
C2 = 6 # High Chroma
LC0 = 7 # Light Luma + Chroma
LC1 = 8 # Med Luma + Chroma
LC2 = 9 # High Luma + Chroma
FH = 10 # Flip-H
FV = 11 # Flip-V
INV = 12 # Invert
"""

SCMD=""" # Clip editing commands in string, can also use commands in file
C0 0,99 # Light Chroma frames @ 0 -> 99
L0 100,-200 # Light Luma frames @ 100, 200 frames ie frames 100->299
INV 300,399 # Invert 300->399
L0 400,499 # Light Luma frames 400->499
FH 500,599 # Flip-H 500->599
LC2 600,699 # High Luma + Chroma
C1 800 # Med Chroma, Single frame
1 900,999 # Light Luma, We used the clip number instead of a NickName
FV 1000,1099 # Flip-V
LC1 2000,0 # Med Luma + Chroma, 2000 -> lastframe
"""

SHOW=True

ClipClop(ORG,V1,V2,V3,V4,V5,V6,V7,V8,V9,V10,V11,V12,scmd=SCMD,nickname=NickNames,show=SHOW)


EDIT: Using FrameSel (can use CMD filename rather than SCMD multi-line string command of frames)

Here, Example to do some simple editing on selected frames, try changing REJECT to True.

Colorbars.Trim(0,-10).ShowFrameNumber() # 10 frames
sCmd="2;5;6" # Frames in SCmd string # EDIT: frames 2, 5 and 6. SEMI-COLON is soft end of line
REJECT=False # False=modify SCmd frames : True=modify frames NOT in SCmd
SELECTED=FrameSel(scmd=sCMD,ordered=TRUE,reject=REJECT,debug=true) # We are going to use FrameRep, must use ORDERED=True (OR REJECT=True)
SELECTED=Invert(SELECTED) # Do some editing
FrameRep(Last,SELECTED,scmd=sCMD,reject=REJECT,debug=true) # Put edited frames back into original Last clip (audio from Last)
return Last

EDIT: You can also use FrameSel just to view a list of frames (or all those not in list using Reject arg), so could eg extract all clock frames
as detected by your initial naive script, and process in some way to refine your masking stuff.
EDIT: You can write frame number in scriptclip using eg RT_WriteFile(".\Frames.txt", "%d", FrameNo,Append=True). # EDIT: Append=True aded
or write clip index and frame number using eg RT_WriteFile(".\Frames.txt", "%d %d", ClipIndex, FrameNo,Append=True). # EDIT: Append=True aded
or write clip index and frame range using eg RT_WriteFile(".\Frames.txt", "%d %d %d", ClipIndex, StartNo, EndNo,Append=True). # EDIT: Append=True aded
EDIT: A clip range can be COMMA ',' or SPACE ' ' separated in CMD file or SCMD multi-line string.
FrameSel can use semi colon "soft end of line" in SCMD string, ClipClop and Prune Cannot [I think] (multiple frames/ranges on single line).
EDIT: Ranges can be specified as separate single frames, makes no difference.

[ChaosKing]

I found your DoctorFrames() function and played a bit it. Results are not bad but sometimes a interpolated frame can look a bit weird (interpolation artifacts). Have not tried a clock mask yet.


I tried to write the frame number to a file with (under the IsClock = (yave > 94) line) IsClock ? RT_WriteFile("list.txt","%i", n , Append=True) : nop
but no list.txt is created (only a empty record.txt)

[StainlessS]

try
IsClock ? RT_WriteFile(".\list.txt","%d", n , Append=True) : nop
EDIT: Above "%i" fixed to "%d".

I think either Vista or W7 screwed up [EDIT: Implied] directory relative filenames, need to use ".\" # DAMN, D9 Non-code block swallows single BACKSLASH, gotta use two. [EDIT it, and it swallows it again].
("list.txt" would write to virtualstore or C: directory or user directoy, not sure which)
And I forgot to use Append=true, nice catch.

EDIT: Pre-scriptclip, I would use
RT_FileDelete(".\list.txt") # clean old file away.

EDIT: FrameSurgeon is later version of DoctorFrames.

EDIT:
(only a empty record.txt)
What is record.txt ?

[StainlessS]

Mistake,
NOT
IsClock ? RT_WriteFile(".\list.txt","%i", n , Append=True) : nop
BUT
IsClock ? RT_WriteFile(".\list.txt","%d", n , Append=True) : nop
%d, not %i

[ChaosKing]

try
IsClock ? RT_WriteFile(".\\list.txt","%d", n , Append=True) : nop
I think either Vista or W7 screwed up [EDIT: Implied] directory relative filenames, need to use "." # DAMN, D9 Non-code block swallows single BACKSLASH, gotta use two. [EDIT it, and it swallows it again].
("list.txt" would write to virtualstore or C: directory or user directoy, not sure which)
And I forgot to use Append=true, nice catch.

EDIT: Pre-scriptclip, I would use
RT_FileDelete(".\list.txt") # clean old file away.

EDIT: FrameSurgeon is later version of DoctorFrames.

EDIT:

What is record.txt ?


.\list.txt did not work either but a full path D:\list.txt works. (I'm using win10 x64)

Ok will try FrameSurgeon

[StainlessS]

ChaosKing,
Two things, note the Mistake above post, I did not notice that you'de used '%i' instead of '%d',
and second, post #34
IsClock ? RT_WriteFile(".\\list.txt","%d", n , Append=True) : nop
You seem to have copied above from my post whilst I was fighting with D9 forum and its fancy editing out of them above double backslashes in "list.txt",
probably the reason did not work proper.

[ChaosKing]

@scharfis_brain did you get rid of that annoying time stamp clocky thing? I'm curious how the result looks like.

[scharfis_brain]

Yeah.

Thanks for all your suggestions.

I finally did the detection of the timestamp using two masks:
- One mask for passing through only the bright parts of the timestamp.
- The 2nd mask for passing through only the dark parts of the timestamp.
with some dead simple averageluma-trickery I was able to reliably detect the timestamp.

After detection I was using three instances of deshaker
1) extrapolate colors into border, use future and past frames for edge compensation
2) additional masking-color and same settings of 1)
3) settings of 1) without edge compensation/extrapolation but with mask

some overlay instances later I got my deshaked and timestamp removed video.

I might later look for a sample that will not violate anybodys privacy.

[ChaosKing]

The same clip would be enough for me :D

[scharfis_brain]

Here it is: https://easyupload.io/chsfi1

It is not perfect. But its artifacts are less distracting than the blinking timestamp.
Also note that the cutout represents only a part of the bottom right corner of the image frame.

[StainlessS]

It is not perfect.

Damn Scharfy, thats one bleedin' good job, well impressive, perfect enough for me. :)

[ChaosKing]

It's the beating heart of the video :D
Nice video stabilisation btw.

[Reel.Deel]

Prewitt exposé by Fluffy[it is different in AVS compared to VS]:- https://forum.doom9.org/showthread.php?p=1839205#post1839205

This may be a long shot but does anyone happen to have a copy of that post? The thread was deleted unfortunately. :mad:


Edit:
This is like the third thread that I see that has been deleted in the last few months. Kinda sucks that people spend time to reply just to have their messages deleted when the OP deleted the thread.

[VoodooFX]

This may be a long shot but does anyone happen to have a copy of that post? The thread was deleted unfortunately. :mad:


Edit:
This is like the third thread that I see that has been deleted in the last few months. Kinda sucks that people spend time to reply just to have their messages deleted when the OP deleted the thread.

Looks like it was HAvsFunc thread by HolyWu. Here you go, that post you are looking for:


The Masktools Prewitt is super weird, yes. To refresh the maths: the Prewitt operator as commonly implemented in image processing calculates the magnitude of a gradient in a 3x3 pixel grid, horizontally and vertically, by applying two convolutions, one for the horizontal and one for the vertical direction, commonly referred to as Gx and Gy respectively:
Gx (horizontal):
-1 0 1
-1 0 1
-1 0 1

Gy (vertical):
1 1 1
0 0 0
-1 -1 -1

The total gradient magnitude is then calculated as sqrt(Gx² + Gy²). A common optimization to avoid the square root is to approximate the magnitude by simply adding the absolute values of Gx and Gy, i.e. abs(Gx) + abs(Gy). This is a decent approximation if one of them is small (that is, the edge is straight, either vertically or horizontally) but pretty bad otherwise.

The Vapoursynth implementation (https://github.com/vapoursynth/vapoursynth/blob/master/src/core/genericfilters.cpp#L1239) is just straight up the canonical Prewitt - the pixel values on the 3x3 grid are called a<column><row>, so the upper left corner of the grid is a11 and the bottom right corner is a33:
float scale = params.scale;
float gx, gy;

gx = a31 + a32 + a33 - a11 - a12 - a13;
gy = a13 + a23 + a33 - a11 - a21 - a31;

return std::sqrt(static_cast<float>(gx * gx + gy * gy)) * scale;
(Some surrounding code omitted for clarity.)

Masktools does something completely different (https://github.com/pinterf/masktools/blob/16bit/masktools/filters/mask/edge/edgemask.cpp#L69) - I guess it works in practice, but it's not what is typically called a Prewitt operator:
const int p90 = a11 + a21 + a31 - a13 - a23 - a33;
const int p180 = a11 + a12 + a13 - a31 - a32 - a33;
const int p45 = a12 + a11 + a21 - a33 - a32 - a23;
const int p135 = a13 + a12 + a23 - a31 - a32 - a21;

const int max1 = max<int>( abs<int>( p90 ), abs<int>( p180 ) );
const int max2 = max<int>( abs<int>( p45 ), abs<int>( p135 ) );
const int maxv = max<int>( max1, max2 );

return threshold<Byte, int>( maxv, nLowThreshold, nHighThreshold );

p90 and p180 are the same as Gx and Gy, but then there's also a "diagonal Prewitt" calculation, using these two kernels:
"G45"
1 1 0
1 0 -1
0 -1 -1

"G135"
0 1 1
-1 0 1
-1 -1 0
Instead of the usual Prewitt which kinda sums the horizontal and vertical magnitude, this returns the maximum magnitude - max(max(Gx, Gy), max(G45, G135)).


EDIT:

Ha, I noticed this post there: :sly:


Dear HolyWu,

If you have some spare time, would you be so kind to port InpaintDelogo to VS? I only found DelogoHD available for VS, but making an lgd file is a pain-in-the-a** compared to VoodooFX's automated process...

https://forum.doom9.org/showthread.php?t=176860

Thanks in advance!

[StainlessS]

Nice to know that VX has a copy of Doom9 forum on his hard drive :)

[kedautinh12]

About prewitt, asd-g was ported it to avisynth
https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/Prewitt.avsi

[VoodooFX]

Nice to know that VX has a copy of Doom9 forum on his hard drive :)
I was thinking about it, but no I don't have a copy, I hope someone else is preserving Doom9, just in case.

It's just from Google's cache: http://webcache.googleusercontent.com/search?q=cache:forum.doom9.net/showthread.php?t=166582

[johnmeyer]

Here's the Wayback Machine cache of how doom9.org looked in previous years:

https://web.archive.org/web/*/http://forum.doom9.org

It has been saved over 900 times in the past twenty years. You should be able to find pretty much anything you want, although sometimes each "save" doesn't capture an entire site, so you have to merge posts from two or three snapshots. As one example of how to use this, I was able to find a now-deleted blog that included a description of an event in which I participated, but which is no longer found by Google, probably because it no longer exists.

[StainlessS]

Thanks guys,

And the google search command for VX results was presumably this [ where was thread No 166582 ]

cache:forum.doom9.net/showthread.php?t=166582


EDIT: Where browser address for this post page is page 3

https://forum.doom9.org/showthread.php?t=181240&page=3

Just remove the "https://" and "&page=3" and precede with "cache:", to find the entire thread in cache.

Can also use eg

"prewitt" NEAR "VoodooFX" site:forum.doom9.org

to find posts containing both "prewitt" and "VoodooFX", quite near to each other.

Was not aware of the cache: thingy.

EDIT: Search for

"prewitt" NEAR "TheFluff" site:forum.doom9.org

Fails to find the thread, Google seems to be now caching the failed attempts ["thread not found"] type pages instead of the lost thread itself.
But, the VoodooFX Google Cache: search thingy still works. [at the moment]

[Reel.Deel]

Looks like it was HAvsFunc thread by HolyWu. Here you go, that post you are looking for:


:thanks: VoodooFX! How did you find the thread that the post belonged to? I searched on google for the link in StainlessS' post but came up empty handed. Mind you, it was 5am for me (just like it is now) so I did not search any further and then posted here. Very sad that a thread with 33 pages was deleted, it should just have been closed :( .

Here's the Wayback Machine cache of how doom9.org looked in previous years:


Yeah, I know about archive.org ... Unfortunately only a small percentage of off all doom9 is saved on there. For example, the thread in question from the previous post, only the first 2 pages of the 33 are actually saved on archive.org. That is true for many of the threads with multiple pages, even some threads with just one page are not saved. And then there's the problem that a thread can have many different links, for example


This "Detecting blinking Timestamp" thread: https://forum.doom9.org/showthread.php?t=181240
But if I reference the very first post it has different link: https://forum.doom9.org/showthread.php?p=1906692#post1906692
And just the post by itself, another link: https://forum.doom9.org/showpost.php?p=1906692&postcount=1

So sometimes people post links to a specific post and if you check the archive that link is not saved but sometimes the main thread link is archived but the page that the post that the user referenced is not.

I've gotten into the habit that anytime I see an informational thread or webpage, I archived it, the Internet Archive extension for Firefox makes it really easy: https://addons.mozilla.org/en-US/firefox/addon/wayback-machine_new/


@StainlessS

Thanks for the additional info about Google cache. Seems the havsfuc thread was deleted not too long ago, otherwise Google would not have it cached.

[StainlessS]

This thread cache search dont work now, comes up with 404 [not found]. (where t=181240 is this thread)

cache:forum.doom9.org/showthread.php?t=181240

Maybe too many people doing search on it perturbed Google a bit,
and VX cache Search only works for first page of the missing thread.

EDIT:
But if I reference the very first post it has different link: https://forum.doom9.org/showthread.p...92#post1906692
I suspect that D9/vBulletin post number is unique ID, where was the 1,906,692th post on Doom9.
(post 1906693 is likely in some other thread).
A thread is probably a list of non contiguous post numbers. [deletion of a post is just a removal of post number from thread list - and perhaps the post text too(although I suspect not)]

EDIT: from main D9 page,
Threads: 152,746, Posts: 1,835,279, Members: 91,529
Welcome to our newest member, yo3ff

posts = 1,835,279, and as 1906692 is higher number, suggests that a lot of posts have been deleted.

Also, thread No 181240 is higher than current thread count (152,746), hinting at a number of threads (about 30,000) deleted.

EDIT: Latest thread number seems to be 183416 (most recent in New Posts with 0 replies) :- https://forum.doom9.org/showthread.php?t=183416