Detect Position of Watermark in movie and remove with logo inpaint

[JohnDoe]

I have a couple of videos with white text watermarks. The watermarks are not always at the same position. In the case of one film, the watermark jumps to a different position every 300 frames. I would like to remove these watermarks with the avs inpaint delogo script. For this one film it is needed to create 130 different masks for avs inpaint. Can someone help me how I can analyze the frames and find the positions automatically? And maybe to automatically create the masks. I also have the watermark itself in a clean form on a black background.

Thank you!

[ChaosKing]

A sample would be great...

[JohnDoe]

Here is a small sample:
Sample.rar (https://www.file-upload.net/download-14076722/Sample.rar.html)
Password: sample0815

[johnmeyer]

Delogo can blur it out. However, there is nothing that can remove it without a trace. Since they used pure white, you can almost certainly devise a moving mask using some of StainlessS' tools like I did for tracking this moving noise bar and removing it:

https://www.youtube.com/watch?v=qx26T6WOZ_4

This will require that you are reasonably comfortable with AVISynth.

[JohnDoe]

I already understood that i cannot remove it completely. But I like the result from https://forum.doom9.org/showthread.php?t=176860. My only concern is to automatically determine the position of the watermarks and, if possible, to automatically generate the masks for the Delogo Script (usually BMP files). Otherwise it is a huge job to do it by hand for every position.

[ChaosKing]

My automation idea would be a brute force method.
1: Extract the font/logo
2: overlay and shift logo by 1px for every x and y value
3: check via makediff when the diff value for x,y is the smallest => optimal overlay
4: apply delogo function
5: repeat after 300 frames

[StainlessS]

Chaos, frame 0 of sample Crop(558,648,384,24) has black background, white text.

[StainlessS]

Any chance of a bit bigger sample, maybe about 60MB ?

[ChaosKing]

Yes saw that, but I'm currently stuck on the compare with every x,y coord part
clip = mvf.ToRGB(clip)
font_crop = core.std.CropRel(clip, left=559, top=649, right=981, bottom=409).std.FreezeFrames(first=0, last=clip.num_frames-1, replacement=0)
font_mask = font_crop.std.Binarize()

[StainlessS]

I'm taking a bit of an alternate route.

Making an RT_DBase, which will create records for each and every frame.
(all I've got so far) [EDIT: Changed as in BLUE at bottom of post]

# MkDBase.avs
# source file with first BLACK frame with text removed/trimmed out.
AviSource(".\SampleStripped.avi")
FRMSTART=122 # 1st Frame at which NEW position sequence starts
FRAMES_VISIBLE=300 # How many frames the logo occupies normally (maybe not first instance sequence)
DB=RT_GetFullPathName(".\SampleStripped.DB")
TypeString="biiiiiii" # bool processed, int GroupNo, int start frame, int end frame, int left, int top, int width, int height.
FC = Framecount
RT_DBaseAlloc(DB,0,TypeString) # NOW Only Single Record for each group (start with 0 records)
GroupNo=0 S=0 E=FRMSTART-1
RT_DebugF("%d] S=%d E=%d (Size=%d)",GroupNo,S,E,E-S+1)
RT_DBaseAppend(DB,false,GroupNo,S,E,0,0,0,0)
for(S=E+1,FC-1) {
E=Min(S+FRAMES_VISIBLE-1,FC-1)
GroupNo=GroupNo+1
RT_DebugF("%d] S=%d E=%d (Size=%d)",GroupNo,S,E,E-S+1)
RT_DBaseAppend(DB,false,GroupNo,S,E,0,0,0,0)
S = E # Next Loop S = E + 1
}

Return Last

Create a group of records for each position, 300 frame groups except for first [and last] group which may be smaller.
Maybe scan on each group, creating group average frame using eg ClipBlend, maybe something like compare
Ave frame for current group with average frame for next group, something like that. [might need scene change stuff too]

EDIT: SampleStripped has first frame removed.

EDITED:
EDIT: CHANGED to single DB record for each group.

[StainlessS]

Create Group Blend clip.

# Scan.avs

Global PROC_FLD = 0
Global GROUP_FLD = 1
Global START_FLD = 2
Global END_FLD = 3
Global LFT_FLD = 4
Global TOP_FLD = 5
Global WID_FLD = 6
Global HIT_FLD = 7

# source file with first BLACK frame with text removed/trimmed out.
AviSource(".\SampleStripped.avi")
ConvertToYV24
DB=RT_GetFullPathName(".\SampleStripped.DB")
NGroups=RT_DBaserecords(DB) # Group Count

BlendC = Last.BlankClip(Length=0)
for(GroupNo=0,NGroups-1) {
S=RT_DBaseGetField(DB,GroupNo,START_FLD)
E=RT_DBaseGetField(DB,GroupNo,END_FLD)
FrmCnt=E-S+1
RT_DebugF("Group=%d S=%d E=%d Size=%d",GroupNo,S,E,FrmCnt)
Tmp=Last.Trim(S,-FrmCnt).Clipblend.Trim(FrmCnt-1,-1)
T = Tmp.RT_YInRangeLocate(n=0,Lo=230,Hi=255,Debug=True,Thresh_W=0,Thresh_h=0,Baffle_W=2,Baffle_H=2,Rescan=True)
if(T) {
Tmp=Tmp.Overlay(Tmp.BlankClip(Width=YIRL_W,Height=YIRL_H,color=$FF0000),x=YIRL_X,y=YIRL_Y,Opacity=0.5)
}
BlendC = BlendC + Tmp
}

Return BlendC

Failry typical group blend frame [3rd of 3 in sample]
https://i.postimg.cc/0K1Mkf17/Scan-00.jpg (https://postimg.cc/0K1Mkf17)

EDIT: Based on only this small sample looks like is easily cracked. [we want bigger sample].
EDIT: Provided there is some movement in each group, we should be able to isolate the white text reasonably easily without scanning
every x,y coord, using RT_YInRangeLocate, or RT_RGBInRangeLocate.
EDIT: And a scene change within Group could actually be beneficial to isolating logo.

EDITED:
EDIT: CHANGED to single DB record for each group.
Change args to RT_YInRangeLocate.

[ChaosKing]

hmm I'm still not sure how you would get the "logo positon" in the end?

But maybe your approach can be combined with http://avisynth.nl/index.php/AutoOverlay

[StainlessS]

hmm I'm still not sure how you would get the "logo positon" in the end?[/url]

RT_YInRangeLocate, or RT_RGBInRangeLocate.

EDIT:

Function RT_YInRangeLocate(clip c,int "n"=current_frame,int "Delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0,int "Baffle"=8,float "Thresh"=0.0, \
int "Lo"=128,int "Hi"=255,Int "Matrix"=(Width>1100||Height>600?3:2),String "Prefix"="YIRL_",bool "Debug"=false, \
int "Baffle_W"=Baffle,int "Baffle_H"=Baffle,Float "Thresh_w"=Thresh,Float "Thresh_h"=Thresh,Bool "Rescan")

Function to scan frame n+delta for an area where percentage of pixels in luma range Lo to Hi (inclusive) breaks a threshold over at
least Baffle consecutive number of scanlines (all four sides).
Returns True on Successful location else false. On success sets Local var Coords.

RT_YInRangeLocate tests scanlines from outer most scanlines (h and v), towards inner most, stops where it finds Baffle consecutive scanlines
that contain more than Thresh percent of pixels in range Lo to Hi. Object interiors are not tested and so could be hollow.
Would also give +ve result for a disjoint shape like this
--
| |
--
Args:
n: default = current_frame in runtime environment.
Delta: Default = 0
X:Y:W:H: All default=0 as in crop. Area to search, default is full frame.
Baffle: Default=8, minimum number of scanlines that must have a percentage of pixels in range Lo to Hi, greater than Thresh. Avoids noise.
Thresh: Default=0.0 (0.0->100.0). Percentage pixels above which breaks Threshold. Default is any pixel in the range Lo to Hi will break threshold.
Lo: Default 128, lower bound of pixel values to search for.
Hi: Default 255, upper bound of pixel values to search for.
Matrix: Default for RGB is 3(PC709) if width > 1100 or height > 600 Else 2(PC601) : YUV not used
Prefix: Default "YIRL_", prefix for Local Var coords set on successful location of object, eg YIRL_X, YIRL_Y, YIRL_W, YIRL_H.
Debug: Default false. If true outputs info to debugView.
Baffle_W, Default Baffle. Thickness of Left and Right side vertical edges, silently limited to width W of search area.
Baffle_H, Default Baffle. Thickness of Top and Bottom horizontal edges, silently limited to height H of search area.

RT_Stats v2.00 Beta 02, Added args Thresh_w and Thresh_h, and Bool ReScan.
Say we are scanning to find top horizontal edge. We will scan and count number of white pixels in a horizontal scanline.
If this percenage count EXCEEDS Thresh_w, and the total number of similarly exceeding consecutive scanlines exceeds Baffle_h
(NOTE Baffle_h NOT Baffle_w, although scanlines are horizontal, we are counting the thickness of consecutive lines vertically)
then we have found the top edge. Same thing for bottom edge using same variables Thesh_w and Baffle_h.
Scanning vertical edges uses Thresh_h and Baffle_w.
NOTE, for RT_Stats v2.00 Beta 02, you can also supply eg Thresh_w = -12.0 (minus) which is taken as a pixel count of +12, rather
than as a percentage. This Thresh_x -ve pixel count will have a teeny weeny bit subtracted before use, as the pixel count must be
greater (something like Thresh_w=Abs(Thresh_w)- 1.0/scanline_width). NOTE, the RT_YInRangeLocator() default of 0.0 detects more
than 0.0 pixels in scanline percentage count, ie any pixel in range (lo->hi) breaks the threshold and counted against
baffle.
The new Bool ReScan arg, forces a horizontal edge (top and bottom) rescan if a new vertical edge is found, and a vertical edge (left and right)
rescan if a new horizontal edge is found. If top and bottom scan broke threshold and passed baffle, followed by left and right scans doing
the same, the top and bottom pixel objects that caused the detections could well have 'moved' outside of the detected area, ie left and right
edges moving inwards may leave those objects outside because in the left/right scan they did not satisfy thresh and baffle requirements,
and detection will likely be bad. Rescan forces a further scan until all four sides scanned sides are a detection (or not). (Will not be
any great overhead if the initial detect was good as it will just recan the same scanlines accessed immediately prior). The internal routine
that RT_YInRangeLocate() uses, returns percentage of pixels that are in the required range, each time an edge is detected, the number of
required pixels is recalculated as a percentage of the reduced scanline lengths (whether given as percentage, or as a -ve pixel count).


As we did for JohnMeyer football game with scrolling bars, [telecine stuff].

[StainlessS]

Not quite there yet, got to twiddle a little bit more but for 2nd group got this [red is detection].
https://i.postimg.cc/Z93wKJWg/Scan-01.jpg (https://postimg.cc/Z93wKJWg)

EDIT: 1st and last group are short groups, the one detected is full 300 frame group.

[poisondeathray]

Nice work SS!

[StainlessS]

Changed both previous scripts to use only single record for each group, instead of one for each frame.

Detector is in-situ but still need some figging around/tweaking [got to try remember how it works, would ideally be much thicker target not flimsy text.]

2 of the 3 detects are just too big (detects white from elsewhere), maybe have to try scan subsection of the detected blocks, dont know of good algo
to do that, gonna have to make one up.
Detector best has RT_Stats v2.0 beta 12 or so.

EDIT:
First fail detect (group 0)
https://i.postimg.cc/0KgfTF5v/Scan-02.jpg (https://postimg.cc/0KgfTF5v)

Group 1 GOOD detect (already posted previously)
https://i.postimg.cc/Z93wKJWg/Scan-01.jpg (https://postimg.cc/Z93wKJWg)

2nd fail detect (group 2, last one)
https://i.postimg.cc/RqfG7SGZ/Scan-03.jpg (https://postimg.cc/RqfG7SGZ)

[StainlessS]

Changed to RgbInRangeLocate()


# Scan.avs

Global PROC_FLD = 0
Global GROUP_FLD = 1
Global START_FLD = 2
Global END_FLD = 3
Global LFT_FLD = 4
Global TOP_FLD = 5
Global WID_FLD = 6
Global HIT_FLD = 7

DETECT=true
SCANRGB=True

# source file with first BLACK frame with text removed/trimmed out.
AviSource(".\SampleStripped.avi")
(SCANRGB) ? ConvertToRGB24 : ConvertToYV24
DB=RT_GetFullPathName(".\SampleStripped.DB")
NGroups=RT_DBaserecords(DB) # Group Count

BlendC = Last.BlankClip(Length=0)
for(GroupNo=0,NGroups-1) {
S=RT_DBaseGetField(DB,GroupNo,START_FLD)
E=RT_DBaseGetField(DB,GroupNo,END_FLD)
FrmCnt=E-S+1
RT_DebugF("Group=%d S=%d E=%d Size=%d",GroupNo,S,E,FrmCnt)
Tmp=Last.Trim(S,-FrmCnt).Clipblend.Trim(FrmCnt-1,-1)
if(DETECT) {
if(!SCANRGB) {
T = Tmp.RT_YInRangeLocate(n=0,Lo=230,Hi=255,Debug=True,Thresh_W=0,Thresh_h=0,Baffle_W=2,Baffle_H=2,Rescan=True)
if(T) {
Tmp=Tmp.Overlay(Tmp.BlankClip(Width=YIRL_W,Height=YIRL_H,color=$FF0000),x=YIRL_X,y=YIRL_Y,Opacity=0.5)
}
} else {
T = Tmp.RT_RGBInRangeLocate(n=0,rLo=220,rHi=255,gLo=220,gHi=255,bLo=220,bHi=255,Debug=True,Thresh_W=0.0,Thresh_h=0.0,Baffle_W=2,Baffle_H=2,Rescan=false)
if(T) {
Tmp=Tmp.Overlay(Tmp.BlankClip(Width=RGBIRL_W,Height=RGBIRL_H,color=$FF0000),x=RGBIRL_X,y=RGBIRL_Y,Opacity=0.5)
}
}
}
BlendC = BlendC + Tmp
}

Return BlendC


Group 0
https://i.postimg.cc/hJZzQdN3/Scan-00.jpg (https://postimg.cc/hJZzQdN3)

Group 1
https://i.postimg.cc/PCtP6yjR/Scan-01.jpg (https://postimg.cc/PCtP6yjR)

Group 2
https://i.postimg.cc/8jG554jz/Scan-02.jpg (https://postimg.cc/8jG554jz)

Dont seem too bad. :)

EDIT: Changed args to RT_YInRangeLocate and full detect there too for Luma.

[johnmeyer]

StainlessS,

I can see a lot of situations where your new invention would be useful. When I get back to my office I need to add this newcode to my AVISynth function folder. Thanks for the work you've done!

Since I'm on the road, I can't test this, but I do have a question: what keeps the rectangle from moving from frame-to-frame? Since the delogo process will be fed by this function, and since that logo removal process isn't perfect, I would think that each time the text moves to a new location you'd want to create a mask and use that mask until the location moves again. By contrast, with the football film you referenced, the mask had to move to a new location for each frame.

[StainlessS]

The source clip is 500 frames [+ 1 frame 0, that has black frame with mask], which we actually dont need for detect].

There are 3 groups [positions where logo appears, 1st and 3rd group short because at ends of clip, middle or 2nd group and all other groups
where longer clip, are 300 frames long.
We create a DBase just so we know where we are later in fix sequence.

[I'm changing how the thing works a little, breaking into smaller simpler operations, so description may not be exactly how appears in prev scripts].

Using DBase, we create a Detect Clip which is just 3 frames for this sample. One frame for each position group and is result of Blend of all frames
within the group using Clipblend() [Blends all 300 frames with equal weight].
It takes some time to create the blended Detect clip, so we do this only once [or once for RGB24 and once for YV24 if you like].

We can then use RT_YInRangeLocate or RT_RgbInRangeLocate to try find position.
Perhaps we were lucky in the 3 good detects, and it may be necessary to do some kind of additional search (
where lots of white on frame in maybe static blended scene, might find block of whit-ish that is way too big).

Detected logos so far have had detect areas
378 x 22,
377 x 21
374 x 21

If we detect a big block [detecting nothing at all would probably be a little unexpected, unless sequence with no logo]
then might be able to step over the detected block, x and y stepping [by maybe 40 [EDIT: Overlapping] pixels at a time, and doing Locate and
look for detect where about 374 x 21 to about 378 x 22 in size.

Anyways, if cannot find location at all for a block, maybe can enable some kind of Override where could set postion manually
in some file.

Maybe one later script calls InPaintDelogo with one group of frames at a time.
Something like that.

Still need work.

EDIT: Dont take too much notice of posted scripts, they have already changed.
[But posted ones will work as is [EDIT: provided 1st black frame with logo trimmed out] , need RT_Stats2.0 Beta latest]

EDIT: I've already posted that I changed from DBase record for each frame, to single one for each group[scripts were changed], I had anticipated doing this anyway, but
started out with one for each frame just incase was necessary.

EDIT: JohnMeyer football Telecine fix using RT_YInRangeLocate, I think I posted a pic of detector detecting telecine bands, but seems that
the pic got swallowed up when PhotoBucket image host turned nasty, here the thread anyway:- https://forum.doom9.org/showthread.php?p=1623732

[johnmeyer]

EDIT: JohnMeyer football Telecine fix using RT_YInRangeLocate, I think I posted a pic of detector detecting telecine bands, but seems that
the pic got swallowed up when PhotoBucket image host turned nasty, here the thread anyway:- https://forum.doom9.org/showthread.php?p=1623732Your help in that thread made me a hero with a certain sports collector.

I too got hit by the Photobucket fiasco, but I was able to download all the photos, and for certain threads that I thought were important (two of which you helped me with), I re-inserted the photos, using another hosting service. So, in that thread you just linked to, all of the original photos are still there.

[StainlessS]

Yep, I managed to recover about 85%+ of my pics, however some musta had a 'Do not ResussScitate' type order applied to them.
I probably removed the bad link. [EDIT: Think I spent about 2 days recovering pics from Photobucket, was well tedious]

A bit more good news.

When using detector on Blended Group Detect clip, a simple Sharpen(1.0) whacks up the flimsey text to Y=255 [or maybe it was R,G,B all up to 255, probably both as used PC.601].
So, in most cases the text should be easily found unless something else somehow survives blending [ie static full white chunk of something].
EDIT: Usually set Lo threshold to about 222, so 255 text big help for detector. [Of course the blurred/blended parts of frame aint got much in the way of edges to sharpen, so almost free of potential probs]
EDIT: I wonder if Delogo type plugs do a Sharpen() when looking for opaque white logo on blended frame average, it would defo be beneficial.

So, looking up for this problem.


EDIT:
Your help in that thread made me a hero with a certain sports collector.
Neither of us would have got very far without IanB pointer in the right direction, that was a Master Class.

[JohnDoe]

Here is a little bit longer sample!

Sample2.rar (https://www.file-upload.net/download-14084562/Sample2.rar.html)

Pass: secret0815

Thank you!

[StainlessS]

JohnDoe,
Can you please try out these 3 scripts, do they work ?, just to see if I need to do a "set manual override script".
Also, is this a one-off, or do you have many similar clips ?

The 2nd script, can make copy and rename copy to 2B_MakeDetectClipYV24.avs and set "MAKERGB=FALSE".

1_MakeDBase.avs

# 1_MakeDBase.avs

# Source file with first BLACK frame with text removed/trimmed out.
AviSource(".\SampleStripped.avi")

################### SAMPLE 1
#FRMSTART=122 # 1st Frame at which NEW position sequence starts
################### SAMPLE 2
FRMSTART=72 # 1st Frame at which NEW position sequence starts
###################

FRAMES_VISIBLE=300 # How many frames the logo occupies normally (maybe not first or last instance sequence)


DB=RT_GetFullPathName(".\SampleStripped.DB")
TypeString="biiiiiiiiiii" # bool processed, int GroupNo, int start frame, int end frame, int left, int top, int width, int height. Int OX, int OY, int OW, int OH (OVERRIDE start scan COORDS)
FC = Framecount
RT_DBaseAlloc(DB,0,TypeString) # NOW Only Single Record for each group (start with 0 records pre-allocated)
GroupNo=0 S=0 E=FRMSTART-1
RT_DebugF("%d] S=%d E=%d (Size=%d)",GroupNo,S,E,E-S+1)
RT_DBaseAppend(DB,false,GroupNo,S,E,0,0,0,0,0,0,0,0)
for(S=E+1,FC-1) {
E=Min(S+FRAMES_VISIBLE-1,FC-1)
GroupNo=GroupNo+1
RT_DebugF("%d] S=%d E=%d (Size=%d)",GroupNo,S,E,E-S+1)
RT_DBaseAppend(DB,false,GroupNo,S,E,0,0,0,0,0,0,0,0)
S = E # Next Loop, S = E + 1
}


2A_MakeDetectClipRGB24.avs

# 2A_MakeDetectClipRGB24.avs

# Might need 64 bit avisynth and VirtualDub2.

MAKERGB=True # True make RGB24 clip(UtVideo 'ULRG') , else YV24 clip("DirectStreamCopy" ie Un-Compressed.) ::: Save Clip in VDub2
# Save as DetectClipRGB24.avi or DetectClipYV24.avi

Global START_FLD = 2
Global END_FLD = 3

Matrix = "PC.709"

AviSource(".\SampleStripped.avi")
(MAKERGB) ? ConvertToRGB24(Matrix=Matrix) : ConvertToYV24

ORG = Last

DB=RT_GetFullPathName(".\SampleStripped.DB")
NGroups=RT_DBaserecords(DB) # Group Count, Same number of frames to output
BlendC = Last.BlankClip(Length=NGroups) # Number of blended detect frames to be generated.

SSS = """
GroupNo = current_frame
S=RT_DBaseGetField(DB,GroupNo,START_FLD) # Start frame for this group.
E=RT_DBaseGetField(DB,GroupNo,END_FLD) # End frame for this group.
FrmCnt=E-S+1 # Number of frames to be blended together for detect clip frame.
Return ORG.Trim(S,-FrmCnt).ClipBlend.Trim(FrmCnt-1,-1) # Create single frame blend of all frames from group
"""

BlendC.ScriptClip(SSS)


3_ScanAndDetect.avs

# 3_ScanAndDetect.avs

###### CONFIG ###########################################
DORGB = True # True use "DetectClipRGB24.avi" clip, else use "DetectClipYV24.avi" clip.
DOSHARP = True # If true, do Sharpen(1.0) on blended detect clip before detect, boosts text brightness without much effect on rest of blended frame
FORCE = False # If True, then force set successful detect coords even if already set previously, Otherwise only set if not already detected previously.
SUBS = True # If True, then show detect coords on TLHS of frame.

LOGO_W = 380 # Expected Logo width
LOGO_W_TOL = 6 # Logo width accetpable tolerance +-
LOGO_H = 22 # Expected Logo Height
LOGO_H_TOL = 4 # Logo height accetpable tolerance +-

LO = 252 # Try 252 to 255 IF SHARPEN(1.0), otherwise about maybe 222 to 230
HI = 255 # Leave at 255

if (DORGB) {
RLO = LO
GLO = LO
BLO = LO
RHI = HI
GHI = HI
BHI = HI
Thresh_W = 0
Thresh_H = 0
Baffle_W = 6 # Suggest 6 if sharpen, or 3 if no Sharpen
Baffle_H = 16
Rescan = true
} Else {
Thresh_W = 0
Thresh_H = 0
Baffle_W = 6 # Suggest 6 if sharpen, or 3 if no Sharpen
Baffle_H = 16
Rescan = true
}

FN = (DORGB) ? ".\DetectClipRGB24.avi" : ".\DetectClipYV24.avi"

INRNGLOCATE_DEBUG = False # RT_YinRngLocate & RT_RgbinRngLocate DEBUGGING

##### END CONFIG ########################################


Global PROC_FLD = 0
Global GROUP_FLD = 1
Global START_FLD = 2
Global END_FLD = 3
Global LFT_FLD = 4
Global TOP_FLD = 5
Global WID_FLD = 6
Global HIT_FLD = 7
Global OX_FLD = 8 # Override start search coords (all zero = full frame)
Global OY_FLD = 9 #
Global OW_FLD = 10 #
Global OH_FLD = 11 #

FN = RT_GetFullPathName(FN)
DB = RT_GetFullPathName(".\SampleStripped.DB")
LOG = RT_GetFullPathName(".\SampleStripped_ScanLog.LOG")

AviSource(FN)

(DOSHARP) ? Sharpen(1.0) : NOP

RT_FileDelete(LOG)

SSS="""
GroupNo = current_frame # Detect clip frame number or GroupNo

ISSET = RT_DBaseGetfield(DB,GroupNo,PROC_FLD) # Were coords already set by previous scan ?
OLD_X = (ISSET) ? RT_DBaseGetField(DB,GroupNo,LFT_FLD) : 0
OLD_Y = (ISSET) ? RT_DBaseGetField(DB,GroupNo,TOP_FLD) : 0
OLD_W = (ISSET) ? RT_DBaseGetField(DB,GroupNo,WID_FLD) : 0
OLD_H = (ISSET) ? RT_DBaseGetField(DB,GroupNo,HIT_FLD) : 0

X=RT_DBaseGetfield(DB,GroupNo,OX_FLD) # Start search coords, scan Full Frame(0,0,0,0) or @ User Set Override Start search coords.
Y=RT_DBaseGetfield(DB,GroupNo,OY_FLD)
W=RT_DBaseGetfield(DB,GroupNo,OW_FLD)
H=RT_DBaseGetfield(DB,GroupNo,OH_FLD)
ORIDE=X!=0||Y!=0||W!=0||H!=0 # True if User Override to Search start Coords.
Tmp = Last.Trim(GroupNo,-1) # Group BLEND detect frame

# Search for logo in blended Detect clip frame for current group(ie text position)
if(DORGB) {
GOT = Tmp.RT_RGBInRangeLocate(n=0,x=X,y=Y,w=W,h=H,
\ rLo=RLO,rHi=RHI,gLo=GLO,gHi=GHI,bLo=BLO,bHi=BHI,Debug=INRNGLOCATE_DEBUG,Thresh_W=Thresh_W,Thresh_H=Thresh_H,Baffle_W=Baffle_W,Baffle_H=Baffle_H,Rescan=RESCAN,Prefix="GOT_")
} Else {
GOT= Tmp.RT_YInRangeLocate(n=0,x=X,y=Y,w=W,h=H,
\ Lo=LO,Hi=HI,Debug=INRNGLOCATE_DEBUG,Thresh_W=Thresh_W,Thresh_h=Thresh_h,Baffle_W=Baffle_W,Baffle_H=Baffle_H,Rescan=Rescan,Prefix="GOT_")
}

ISSAME = ISSET && GOT_X==OLD_X && GOT_Y==OLD_Y && GOT_W==OLD_W && GOT_H==OLD_H # Changed detect coords ?
Tmp=(ISSET) ? Tmp.Overlay(Tmp.BlankClip(Width=OLD_W,Height=OLD_H,color=$FFFF00),x=OLD_X,y=OLD_Y,Opacity=0.5) : Tmp # If Previously set then Show YELLOW.

if(!GOT) {
s = (ISSET) ? RT_String("%d] %4d,%4d,%4d,%4d # FAILED DETECT : PRE-EXISTING in YELLOW",GroupNo,OLD_X,OLD_Y,OLD_W,OLD_H) : RT_string("%d # FAILED DETECT",GroupNo)
} Else {
GOOD_DET = Abs(GOT_W-LOGO_W)<=LOGO_W_TOL && Abs(GOT_H-LOGO_H)<=LOGO_H_TOL # Size within expected bounds
if(ISSAME) {
s = RT_String("%d] %4d,%4d,%4d,%4d # NO CHANGE",GroupNo,OLD_X,OLD_Y,OLD_W,OLD_H)
} Else if(GOOD_DET) {
if(!ISSET||FORCE) {
RT_DBaseSetField(DB,GroupNo,PROC_FLD,True)
RT_DBaseSetField(DB,GroupNo,LFT_FLD,GOT_X)
RT_DBaseSetField(DB,GroupNo,TOP_FLD,GOT_Y)
RT_DBaseSetField(DB,GroupNo,WID_FLD,GOT_W)
RT_DBaseSetField(DB,GroupNo,HIT_FLD,GOT_H)
s = RT_String("%d] %4d,%4d,%4d,%4d # %s",
\GroupNo,GOT_X,GOT_Y,GOT_W,GOT_H,
\ (!ISSET)?((!ORIDE)?"NEW DETECT(Green)":"OVERRIDE NEW DETECT(Green)"):((!ORIDE)?"REPLACED DETECT(Blue)":"OVERRIDE REPLACED DETECT(Blue)"))
Tmp=Tmp.Overlay(Tmp.BlankClip(Width=GOT_W,Height=GOT_H,color=(!ISSET)?$00FF00:$0000FF),x=GOT_X,y=GOT_Y,Opacity=0.5) # SET,
} Else {
Tmp=Tmp.Overlay(Tmp.BlankClip(Width=GOT_W,Height=GOT_H,color=$FF00FF),x=GOT_X,y=GOT_Y,Opacity=0.5) # IGNORED, Is Already set, MAGENTA
s= RT_string("%d] %4d,%4d,%4d,%4d # IGNORED : ALREADY SET",GroupNo,GOT_X,GOT_Y,GOT_W,GOT_H)
}
} Else {
Tmp=Tmp.Overlay(Tmp.BlankClip(Width=GOT_W,Height=GOT_H,color=$FF0000),x=GOT_X,y=GOT_Y,Opacity=0.5) # Bad Detect RED, not set
s= RT_string("%d] %4d,%4d,%4d,%4d # INVALID SIZE",GroupNo,GOT_X,GOT_Y,GOT_W,GOT_H)
}
}
RT_DebugF("%s",s)
RT_WriteFile(LOG,"%s",s,Append=true)
Tmp = (SUBS) ? Tmp.RT_Subtitle("%s",s) : Tmp
Return Tmp
"""
Last.ScriptClip(SSS)
Return Last


You need RT_Stats 2.0 Latest Beta from MediaFire in my sig below this post.

This is no fix just yet, just want to know if it detects 100% OK.

EDIT: What I'm gettin' on Sample 2. [with scripts as supplied]

0] 16, 97, 380, 22 # NEW DETECT(Green)
1] 501, 52, 380, 22 # NEW DETECT(Green)
2] 1325, 541, 380, 22 # NEW DETECT(Green)
3] 71, 878, 379, 22 # NEW DETECT(Green)
4] 1004, 450, 380, 22 # NEW DETECT(Green)
5] 126, 663, 380, 22 # NEW DETECT(Green)
6] 978, 540, 380, 22 # NEW DETECT(Green)
7] 497, 95, 380, 22 # NEW DETECT(Green)
8] 251, 805, 380, 22 # NEW DETECT(Green)

[JohnDoe]

Currently i use avisynth 2.60 build: Feb 20 2015 03:16:45 with wine in a windows xp bottle with linux.

For your scripts i need some newer version?

Thanks!

[StainlessS]

Avs+ would be better, but you could try with GScript plugin and RT_Stats v2.0 beta 12+ [mandatory]

Try install Avs+, https://forum.doom9.org/showthread.php?t=181351

[JohnDoe]

When i try to use the 2nd avs script with SetMTMode... it no more works. And without it is very slow. Do you have a idea?

Thanks!

[StainlessS]

Dont use SetMTMode [or rather Prefetch], for any of the supplied scripts [things have to be done in order, not simultaneously]
The 2nd script averages 300 input frames for each output frame, and also converts each input to either YV24 or RGB24
and the average is on all three channels of each input frame, so is maybe a bit slow. (Also processed in ScriptClip which is also a factor for speed).
Even on my crap Core Duo it runs in at least real time, ie will take no longer than duration of input clip, dont sweat about it, take the dog/cat/ferret for a walk or something.

[JohnDoe]

Strange. I installed now Avisynth+ ... but again with wine and Linux because i don't use Windows. My PC is powered by an old six core Intel Xeon w3690@4Ghz ... and here i need for one frame with the second avs script about 6 seconds...

Maybe because the old xeon doesn't have AVX extensions?!?
https://www.techsavvy.de/prozessoren/intel_xeon_w3690/

I think this is the problem....

cu

[StainlessS]

ClipBlend is simply C++, no AVX/SSE2/ etc.

300 Frames / 6 seconds is 50 input frames per second, single core.

[JohnDoe]

Is this good? Or is this slow?

[StainlessS]

Its not fast, but not that slow either. If you want it fast, then you're gonna have to look elsewhere.

[ChaosKing]

I would check if the the logo position has a threshold of how close it appears near an edge. You could crop then the video a bit to speed up the process.
Or just simply run multiple instances for multiple videos.

[StainlessS]

Temporary Test, using Uglarm Delogo:- https://forum.doom9.org/showthread.php?t=176891

5A_Delogo_Uglarm.avs

# 5A_Delogo_Uglarm.avs : Temp Test using Uglarm delogo

###### CONFIG ###########################################
# Source file with first BLACK frame with text removed/trimmed out.
FN = RT_GetFullPathName(".\SampleStripped.avi")
DB = RT_GetFullPathName(".\SampleStripped.DB")

# UGLARM Args
SHOWMODE = 2 # UGLARM Show Mode, 0 = Hilite Logo Coords, 2 = Uglarm delogo
BLURPOWER = 3.0 # UGLARM BlurPower 0.0 < BlurPower <= 10.0
SAR = 1.0
#

SUBS = False # Show Coords TLHS

##### END CONFIG ########################################
Avisource(FN)
ConvertToYV24 # Some Coords are ODD
############
ORG=Last

PROC_FLD = 0
GROUP_FLD = 1
START_FLD = 2
END_FLD = 3
LFT_FLD = 4
TOP_FLD = 5
WID_FLD = 6
HIT_FLD = 7
############

PREV_FRAME = -2 # Force Init
GroupNo = -1 # Init for play
GROUP_START = -1 # Init for play
GROUP_END = -1 # Init for play
LogoX = 0
LogoY = 0
LogoW = 0
LogoH = 0
CropX = 0
CropY = 0
CropW = 0
CropH = 0
####

GroupCnt = RT_DBaseRecords(DB)
FC = Framecount

Assert(DetectValid(DB,PROC_FLD),"Some LOGOS NOT found")

SSS="""
n = current_frame
if(PREV_FRAME + 1 != n || n > GROUP_END) {
GroupNo = RT_DBaseFindSeq(DB,START_FLD,END_FLD, n)
GROUP_START = RT_DBaseGetField(DB,GroupNo,START_FLD)
GROUP_END = RT_DBaseGetField(DB,GroupNo,END_FLD)
LogoX = RT_DBaseGetField(DB,GroupNo,LFT_FLD)
LogoY = RT_DBaseGetField(DB,GroupNo,TOP_FLD)
LogoW = RT_DBaseGetField(DB,GroupNo,WID_FLD)
LogoH = RT_DBaseGetField(DB,GroupNo,HIT_FLD)
}
Uglarm(LogoX,LogoY,LogoW,LogoH,CropX,CropY,CropW,CropH,SAR=SAR,BlurPower=BLURPOWER,ShowMode=SHOWMODE,BLANK=False)
(SUBS) ? RT_Subtitle("%d/%d:%d/%d] %d %d %d %d",GroupNo,GroupCnt,n,FC,LogoX,LogoY,LogoW,LogoH) : NOP
PREV_FRAME = n
Return Last
"""
ORG.ScriptClip(SSS)
Return ConverttoYV12

############

Function DetectValid(String DB,Int Field) {
GroupCnt = RT_DBaseRecords(DB)
OK = True
for(i=0,GroupCnt-1) {
if(!RT_DBaseGetField(DB,i,Field)) {
OK = False
RT_DebugF("%d] Group subtitle NOT Found",i)
i=GroupCnt # Break
}
}
Return OK
}


It is slow. EDIT: About 10FPS [EDIT: Actually 11FPS after Ffix] on Core Duo 2.8GHz.

EDIT: Renamed to "5A_ ...avs", reserving 4n... for Manual Override script if necessary.

FIXED:

[StainlessS]

Mod for prev given Uglarm delogo, getting about 22.0 FPS [EDIT: Actually 31FPS after fix] on Core Duo 2.8GHz (about double prev issue).

5B_Delogo_Uglarm.avs

# 5B_Delogo_Uglarm.avs : Test using Uglarm delogo : YV12 not YV24

###### CONFIG ###########################################
# Source file with first BLACK frame with text removed/trimmed out.
FN = RT_GetFullPathName(".\SampleStripped.avi")
DB = RT_GetFullPathName(".\SampleStripped.DB")

SUBS = False # Show Coords TLHS

MARGIN_L = 1 # Horizontal (Left and right) pixel margin around detected subs to ensure we got entire subtitle
MARGIN_R = 1
MARGIN_T = 0 # Vertical margin maybe not necessary (is wide subtitle, plenty of horizontal pixels for vertical detect)
MARGIN_B = 0

# UGLARM Args
SHOWMODE = 2 # UGLARM Show Mode, 0 = Hilite Logo Coords, 2 = Uglarm delogo
BLURPOWER = 3.0 # UGLARM BlurPower 0.0 < BlurPower <= 10.0
SAR = 1.0
#

##### END CONFIG ########################################
Avisource(FN)
############
ORG = Last

PROC_FLD = 0
GROUP_FLD = 1
START_FLD = 2
END_FLD = 3
LFT_FLD = 4
TOP_FLD = 5
WID_FLD = 6
HIT_FLD = 7
############

PREV_FRAME = -2 # Force Init
GroupNo = -1 # Init for play
GROUP_START = -1 # Init for play
GROUP_END = -1 # Init for play

####
GroupCnt = RT_DBaseRecords(DB)
FC = Framecount

Assert(DetectValid(DB,PROC_FLD),"Some LOGOS NOT found")

GRP_Clip = Last.Blankclip(Length=0)

SSS="""
n = current_frame
if(PREV_FRAME + 1 != n || n > GROUP_END) {
GroupNo = RT_DBaseFindSeq(DB,START_FLD,END_FLD, n)
GROUP_START = RT_DBaseGetField(DB,GroupNo,START_FLD)
GROUP_END = RT_DBaseGetField(DB,GroupNo,END_FLD)
ORG_LogoX = RT_DBaseGetField(DB,GroupNo,LFT_FLD)
ORG_LogoY = RT_DBaseGetField(DB,GroupNo,TOP_FLD)
ORG_LogoW = RT_DBaseGetField(DB,GroupNo,WID_FLD)
ORG_LogoH = RT_DBaseGetField(DB,GroupNo,HIT_FLD)
X1 = (ORG_LogoX - MARGIN_L) / 2 * 2 # Coords on 2x2 boundary after adding margins for detect error
X2 = (ORG_LogoX + ORG_LogoW + MARGIN_R + 1) / 2 * 2 # Is actually coord of rightmost pixel + 1 (ie x2+1)
Y1 = (ORG_LogoY - MARGIN_T) / 2 * 2
Y2 = (ORG_LogoY + ORG_LogoH + MARGIN_B + 1) / 2 * 2
GRP_Clip = ORG.Uglarm(X1,Y1,X2-X1,Y2-Y1,SAR=SAR,BlurPower=BLURPOWER,ShowMode=SHOWMODE)
}
Last = GRP_Clip
(SUBS) ? RT_Subtitle("%d/%d:%d/%d : %d/%d] %d %d %d %d",GroupNo,GroupCnt,n-GROUP_START,GROUP_END-GROUP_START+1,n,FC,LogoX,LogoY,LogoW,LogoH) : NOP
PREV_FRAME = n
Return Last
"""
ORG.ScriptClip(SSS)
Return Last

############

Function DetectValid(String DB,Int Field) {
GroupCnt = RT_DBaseRecords(DB)
OK = True
for(i=0,GroupCnt-1) {
if(!RT_DBaseGetField(DB,i,Field)) {
OK = False
RT_DebugF("%d] Group subtitle NOT Found",i)
i=GroupCnt # Break
}
}
Return OK
}


ChaosKing,
I would check if the the logo position has a threshold of how close it appears near an edge. You could crop then the video a bit to speed up the process.
Or just simply run multiple instances for multiple videos.

At least as far as Uglarm is concerned, it already does internally crop on affected area, and also only affects the logo area, ie does affect full frame where/if colorspace change is necessary.

Before:
https://i.postimg.cc/PpDHNFC6/5-B-Delogo-Uglarm-2-00.jpg (https://postimg.cc/PpDHNFC6)

After:
https://i.postimg.cc/kRdmSF87/5-B-Delogo-Uglarm-2-01.jpg (https://postimg.cc/kRdmSF87)

I'll try get it working with VoodooFX InPaintDelogo, but is likely much slower, as big script with disk based mask required.
Might be able to extract the magic bits out of VoodooFX script, so as to be able to only use necessary parts with
non disk based mask.

FIXED:

[StainlessS]

5A_Delogo_Uglarm.avs and 5B_Delogo_Uglarm.avs both updated, BugFix.

5A getting about 11.0 FPS rather than 10.0, and 5B getting 31.0 FPS instead of 22.0 FPS. [Core Duo 2.8GHz, quad using single core]

Forgot to add this line (so was reading DBase for every frame).

PREV_FRAME = n


EDIT: Above timings for x86 in VDub2 saving AVI at UtVideo.
Doing similar for 5B_Delogo_Uglarm.avs in x64 getting 35.5FPS.

Using AvsMeter for same, x86 getting 35.5 FPS, and x64 38.4FPS.

[VoodooFX]

ClipBlend is simply C++, no AVX/SSE2/ etc.

300 Frames / 6 seconds is 50 input frames per second, single core.

I started toying with Sample2.mkv (https://forum.doom9.org/showthread.php?p=1912410#post1912410) and ClipBlend, on the first try I got a clip with a pretty good dynamic mask (a bit different approach than yours), but there is some bug/bottleneck in ClipBlend.
Look what you can make out of this:

Script:

SetMemoryMax(2000)
LoadPlugin("D:\LSMASHSource.dll")
LWLibavVideoSource("D:\Sample2.mkv")
ClipBlend(52)


ClipBlend(52) benchmark (52 - last good setting before a threshold for "bug" is reached):

AVSMeter 3.0.7.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 2.399 | 53.06 | 28.19
Process memory usage (max): 436 MiB
Thread count: 10
CPU usage (average): 42.5%

Time (elapsed): 00:01:17.184
It starts at ~5fps and slowly goes down to ~2fps till ~190th frame (all that at ~90% cpu use on 2core/4threads), then sudden jump to ~45fps (cpu drops to ~40%) and never slows down till the end.


Here is ClipBlend(53) benchmark (I terminated as it goes slower and slower and finishes after an hour or two. Btw, memory usage stays always same when with "ClipBlend(52)" it gradually goes up):

Frame (current | last): 446 | 2175
FPS (cur | min | max | avg): 0.771 | 0.771 | 52.90 | 1.695
Process memory usage: 128 MiB
Thread count: 10
CPU usage (current | average): 94.9% | 83.0%

Time (elapsed | estimated): 00:04:23.179 | 00:21:24.030
^C


Tried 540p resolution and there it's same 52/53 bug, tried 64x32 resolution - same. ClipBlend x64 - same.

Full runs with 64x32:
ClipBlend(52) - 1772fps
ClipBlend(53) - 73fps
ClipBlend(500) - 74fps
ClipBlend(1000) - 113fps


Then I tried classic AvS 2.6, there it's same, except that a threshold is at 99/100 instead of 52/53.
ClipBlend(99) with AvS 2.6:

AVSMeter 3.0.7.0 (x86), (c) Groucho2004, 2012-2020
AviSynth 2.60, build:Mar 31 2015 [16:38:54] (2.6.0.6)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 1.991 | 52.90 | 26.10
Process memory usage (max): 706 MiB
Thread count: 6
CPU usage (average): 42.1%

Time (elapsed): 00:01:23.374

[StainlessS]

@ VoodooFX
Arh, this was the thread you were talking about a little while ago.
I did see your post but did not have a clue what you where talkig about and went to bed and forgot all about it the next day.
[I think I saw another 'awkward post', (one I could not immediately answer) the same day, I think I probably forgot about that one too, I'll go look for it tomorrow].
I dont really have much time but I'll try find a little of it [time] tomorrow, its now 02:21 and I'm off to bed soon.
Chow.
[always confused me why italians invoke the name of a fluffy dog when saying goodbye].

[VoodooFX]

Sorry that I was lazy to search and didn't linked here.
Bugs never walk alone, now I'm dazzled with some AvsInPaint bug processing this dynamic mask... :eek:

Anyway, here is a test when ClipBlend'ed all 2176 frames:


AVSMeter 3.0.7.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 28.52 | 53.23 | 51.54
Process memory usage (max): 128 MiB
Thread count: 10
CPU usage (average): 41.5%

Time (elapsed): 00:00:42.220

[StainlessS]

VX, here some numbers (all Avs+, latest)
Numbers starting at your threshold number 52-1 = 51.

Firstly AVI, UT_Video


D:\VX>avsmeter vxa.avs [Avs+ AVI ClipBlend(51)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 26.25 | 63.05 | 41.22
Process memory usage (max): 372 MiB
Thread count: 9
CPU usage (average): 33.4%

Time (elapsed): 00:00:52.790

D:\VX>avsmeter vxb.avs [Avs+ AVI ClipBlend(52)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 21.68 | 63.74 | 41.50
Process memory usage (max): 379 MiB
Thread count: 9
CPU usage (average): 33.4%

Time (elapsed): 00:00:52.433

D:\VX>avsmeter vxc.avs [Avs+ AVI ClipBlend(53)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 22.92 | 63.13 | 35.95
Process memory usage (max): 72 MiB
Thread count: 9
CPU usage (average): 38.5%

Time (elapsed): 00:01:00.527

D:\VX>avsmeter vxd.avs [Avs+ AVI ClipBlend(54)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 25.15 | 63.03 | 40.92
Process memory usage (max): 378 MiB
Thread count: 9
CPU usage (average): 33.4%

Time (elapsed): 00:00:53.178


Then original MKV, LSmash MKV, Threads=0 (ie auto)


D:\VX>avsmeter vya.avs [Avs+ MKV THREAD=0 ClipBlend(51)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)
Creating lwi index file 100%

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 2.313 | 67.26 | 30.10
Process memory usage (max): 428 MiB
Thread count: 10
CPU usage (average): 45.8%

Time (elapsed): 00:01:12.289

D:\VX>avsmeter vyb.avs [Avs+ MKV THREAD=0 ClipBlend(52)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 2.488 | 69.11 | 29.70
Process memory usage (max): 433 MiB
Thread count: 10
CPU usage (average): 45.5%

Time (elapsed): 00:01:13.274

D:\VX>avsmeter vyc.avs [Avs+ MKV THREAD=0 ClipBlend(53)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 0.683 | 67.80 | 1.388
Process memory usage (max): 129 MiB
Thread count: 10
CPU usage (average): 91.4%

Time (elapsed): 00:26:08.218

D:\VX>avsmeter vyd.avs [Avs+ MKV THREAD=0 ClipBlend(54)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 0.692 | 70.94 | 1.381
Process memory usage (max): 129 MiB
Thread count: 10
CPU usage (average): 91.2%

Time (elapsed): 00:26:16.107


Then original MKV, LSmash MKV, Threads=1


D:\VX>avsmeter vza.avs [Avs+ MKV THREAD=1 ClipBlend(51)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 1.167 | 55.62 | 16.62
Process memory usage (max): 397 MiB
Thread count: 5
CPU usage (average): 24.4%

Time (elapsed): 00:02:10.959

D:\VX>avsmeter vzb.avs [Avs+ MKV THREAD=1 ClipBlend(52)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 1.142 | 55.36 | 16.39
Process memory usage (max): 403 MiB
Thread count: 5
CPU usage (average): 24.4%

Time (elapsed): 00:02:12.726

D:\VX>avsmeter vzc.avs [Avs+ MKV THREAD=1 ClipBlend(53)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 0.198 | 55.69 | 0.427
Process memory usage (max): 99 MiB
Thread count: 5
CPU usage (average): 24.6%

Time (elapsed): 01:25:00.314

D:\VX>avsmeter vzd.avs [Avs+ MKV THREAD=1 ClipBlend(54)]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 0.197 | 55.61 | 0.417
Process memory usage (max): 101 MiB
Thread count: 5
CPU usage (average): 24.6%

Time (elapsed): 01:27:01.902


Twould seem that awkwardness is somehow related to source filter,
ClipBlend() jumps about a lot, and requires Frame Accurate filter, seems that LSmash has problems jumping about,
also, having played a little with AVS 2.6 Std, I would say that version of AVS caching, also plays some part in problem you have pointed out.

Going to the feathers [ie bed, Swiss, or perhaps German or Austrian saying] soon, will make script that simulates ClipBlend action tomorrow,
should easily show similar numbers.

04:00 am now, toodle-pip.
EDIT:
[MKV timings are painfully slow, took a long time, more than enough time for me to be totally pissed right now, Ill sleep ok tonight].
EDIT: MKV timing == LSmash - Reminds me why I pretty much always convert clips to AVI, unless VOB/MPEG where
I use DGIndex. LSmash and FFMPEG generally are unpredicatable probs.
AVI is old, but aint nuttin works better/more predicable [OK, DGindex, MPegSource is faultless too, way to go Don].

[StainlessS]

OK, here AVS+ script demo simulator of approx what Clipblend does. [simulating the LSmash slowdown]


#####################################
# Play FORWARD ONLY [AVS+]
#####################################
DELAY=53 # OK 51 and 52, BAD 53 and 54
THREADS=0
DEBUG=False
#############################
SetMemoryMax(2000)
#Avisource(".\Sample2.mkv.AVI")
LWLibavVideoSource(".\Sample2.mkv",Threads=THREADS)
#############################

QQQ="""
RT_DebugF("%d ] frame fetch",current_frame,name="QQQ: ")
Return Last
"""

(DEBUG) ? ScriptClip(QQQ) : NOP

SSS="""
Want_E = current_frame
Want_S = Max(Want_E - DELAY, 0)
if(WANT_S > 0) {
current_frame = Want_S - 1
Dummy = AverageLuma # Simulate subtract of Previous iteration start of range frame from accumulators.
}
current_frame = Want_E # Restore current_frame [EDIT: This should maybe be inside above if condition]
Dummy = AverageLuma # Simulate addition of current_frame [End of Range] to accumulators.
Return Last # Simulate render average frame from accumulators. [just return frame instead of averageluma]
"""
# Above, Final Return last should be more like return Last.Invert or some other func which creates New frame, but demo still works as required.
# (The frame will in this case be fetched from caching done by previous line Averageluma)

ScriptClip(SSS)


Results, bad one halted prematurely

D:\VX>avsmeter simulated.avs [MKV Clipblend(Delay=52]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 2.500 | 422.8 | 50.85
Process memory usage (max): 419 MiB
Thread count: 10
CPU usage (average): 78.6%

Time (elapsed): 00:00:42.790

D:\VX>avsmeter simulated.avs [MKV Clipblend(Delay=53 - HALTED BEFORE END, was getting slower and slower]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 696 (0 - 695)
FPS (min | max | average): 0.690 | 358.2 | 1.633
Process memory usage (max): 114 MiB
Thread count: 10
CPU usage (average): 85.8%

Time (elapsed): 00:07:06.318


EDIT:
Anyway, here is a test when ClipBlend'ed all 2176 frames:
When blending all frames, does not need to be jumping back and forth, just adds next frame to accumulators and renders average.
[So no weird slow downs].

EDIT:
And here same as bad one above, but using AVI UT_Video codec pre converted via ffmpeg [AviSource instead of LSmash with MKV].

D:\VX>avsmeter simulated.avs [AVI ClipBelnd(delay=53]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 12.46 | 1394 | 81.13
Process memory usage (max): 54 MiB
Thread count: 9
CPU usage (average): 51.6%

Time (elapsed): 00:00:26.822

Maybe could have been a bit faster, I did it whilst showing small Webcam "Always On Top" on desktop.

[VoodooFX]

From your UT_Video test (I assume 5x numbers are correct here now):

D:\VX>avsmeter vxb.avs [Avs+ AVI ClipBlend(52)]
Process memory usage (max): 379 MiB
Time (elapsed): 00:00:52.433

D:\VX>avsmeter vxb.avs [Avs+ AVI ClipBlend(53)]
Process memory usage (max): 72 MiB # why it is much less than with (52)?
Time (elapsed): 00:01:00.527 # ~16% slower. No biggie like with LSMASH, but for some reason it slows here.


Anyway, I added RequestLinear(rlim=201,clim=201) before ClipBlend(200) and there is no "slow mode" anymore with LSMASH.

I think binarizing video before ClipBlend should give better isolation/detection results. Maybe ClipBlend would run faster if it would blend only luma ignoring the rest on binarized clip?

[StainlessS]

(I assume 5x numbers are correct here now)
Yep, I originally posted all the AVI tests 3 posts prior to this, labelled as AVI Clipblend(51), have corrected the 3 wrong ones.
# why it is much less than with (52)?
Dont know.
# ~16% slower. No biggie like with LSMASH, but for some reason it slows here.
Dont know, I think AVS Std and AVS+ have some kind of cache strategy learning thing, and that AVS Std and AVS+ strategy
are different, perhaps ~16% slower something to do with cache strategy, and also the bug threshold related to AVS/AVS+ strategy thingy too.
Anyway, I added RequestLinear(rlim=200,clim=200) before ...
Great stuff, not sure if Ive done that too in the past, I'll hav'ta remember it.
blend only luma
You could try feeding it with ConvertToY8, should work ok.

[StainlessS]

Amendments to simulator,

#####################################
# Play FORWARD ONLY [AVS+]
#####################################
DELAY=53 # OK 51 and 52, BAD 53 and 54
THREADS=0
DEBUG=False
BUGFIX=True
#############################
SetMemoryMax(2000)
#Avisource(".\Sample2.mkv.AVI")
LWLibavVideoSource(".\Sample2.mkv",Threads=THREADS)
#############################

QQQ="""
RT_DebugF("%d ] frame fetch",current_frame,name="QQQ: ")
Return Last
"""

(DEBUG) ? ScriptClip(QQQ) : NOP

(BUGFIX) ? RequestLinear(rlim=200,clim=200) : NOP

SSS="""
Want_E = current_frame
Want_S = Max(Want_E - DELAY, 0)
if(WANT_S > 0) {
current_frame = Want_S - 1
Dummy = AverageLuma # Simulate subtract of Previous iteration start of range frame from accumulators.
current_frame = Want_E # Restore current_frame
}
Dummy = AverageLuma # Simulate addition of current_frame [End of Range] to accumulators.
Return Last # Simulate render average frame from accumulators. [just return frame instead of averageluma]
"""
# Above, Final Return last should be more like return Last.Invert or some other func which creates New frame, but demo still works as required.
# (The frame will in this case be fetced from caching done by previous line Averageluma)
ScriptClip(SSS)


AVI ClipBlend(Delay=53) # Faster, not showing WebCam Always OnTop on desktop [or maybe RequestFrameLinear helped speed up]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 118.8 | 212.8 | 160.9
Process memory usage (max): 650 MiB
Thread count: 9
CPU usage (average): 54.8%

Time (elapsed): 00:00:13.526


LSmash MKV Clipblend(Delay=53), nice improvment [EDIT: THREADS=0, ie default, BUGFIX=True]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 114.5 | 216.2 | 173.2
Process memory usage (max): 704 MiB
Thread count: 10
CPU usage (average): 85.1%

Time (elapsed): 00:00:12.563

[VoodooFX]

Here is a small tweak to the simulator:


#####################################
# Play FORWARD ONLY [AVS+]
#####################################
DELAY=53 # OK 51 and 52, BAD 53 and 54
THREADS=0
DEBUG=False
BUGFIX=True
#############################
SetMemoryMax(2000)
#Avisource(".\Sample2.mkv.AVI")
LWLibavVideoSource(".\Sample2.mkv",Threads=THREADS)
#############################

QQQ="""
RT_DebugF("%d ] frame fetch",current_frame,name="QQQ: ")
Return Last
"""

(DEBUG) ? ScriptClip(QQQ) : NOP

(BUGFIX) ? RequestLinear(rlim=DELAY+1,clim=DELAY+1) : NOP

SSS="""
Want_E = current_frame
Want_S = Max(Want_E - DELAY, 0)
if(WANT_S > 0) {
current_frame = Want_S - 1
Dummy = AverageLuma # Simulate subtract of Previous iteration start of range frame from accumulators.
current_frame = Want_E # Restore current_frame
}
Dummy = AverageLuma # Simulate addition of current_frame [End of Range] to accumulators.
Return Last # Simulate render average frame from accumulators. [just return frame instead of averageluma]
"""
# Above, Final Return last should be more like return Last.Invert or some other func which creates New frame, but demo still works as required.
# (The frame will in this case be fetced from caching done by previous line Averageluma)
ScriptClip(SSS)

[StainlessS]

Magic, will those mods create havoc if eg Delay=10,000 ?

[NOT a prob on this clip as will be too short].

EDIT: Made Delay=2000 [2176 frames in clip], and no problems [EDIT: Delay=1000 also no probs]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 2176
Length (hh:mm:ss.ms): 00:01:12.604
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/100
Colorspace: i420

Frames processed: 2176 (0 - 2175)
FPS (min | max | average): 96.33 | 234.1 | 171.2
Process memory usage (max): 3866 MiB
Thread count: 10
CPU usage (average): 84.5%

Time (elapsed): 00:00:12.712

[VoodooFX]

Magic, will those mods create havoc if eg Delay=10,000 ?

Yes, that's how many frames would want to be buffered. :D

[StainlessS]

Coupla tests at DELAY=10000, multi-Spliced clips to about 21,000 input frames.


D:\VX>avsmeter simulated.avs [ MKV : DELAY=10000]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 21760
Length (hh:mm:ss.ms): 00:12:06.035
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: i420

Frame (current | last): 11090 | 21759
FPS (cur | min | max | avg): 642.0 | 3.805 | 2002 | 29.38
Process memory usage: 3864 MiB
Thread count: 10
CPU usage (current | average): 25.0% | 67.2%

Time (elapsed | estimated): 00:06:17.511 | 00:12:20.724

Could not allocate video frame. Out of memory. memory_max = 2097152000,
memory_used = 3931545600 Request=3110400

AVI with same delay did about the same as above error halt.

AVI : Delay=10000 : BUGFIX=false

D:\VX>avsmeter simulated.avs [ AVI : DELAY=10000 : BUGFIX=False ]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)

Number of frames: 21760
Length (hh:mm:ss.ms): 00:12:06.035
Frame width: 1920
Frame height: 1080
Framerate: 29.971 (29971/1000)
Colorspace: YV12

Frames processed: 21760 (0 - 21759)
FPS (min | max | average): 50.61 | 247.2 | 122.0
Process memory usage (max): 60 MiB
Thread count: 9
CPU usage (average): 62.6%

Time (elapsed): 00:02:58.403

Average FPS was getting slower as time progressed, but as expected as first 10,000 frames would not be subtracting start of range frame from accumulators (ie faster for 1st 10,000 frames).
EDIT: Core Duo Q9550 Quad core, W7 x64, AVS+ x86.

[VoodooFX]

AVSMeter 3.0.2.0 (x86), (c) Groucho2004, 2012-2020
AviSynth+ 3.6.2 (r3328, master, i386) (3.6.2.0)
Process memory usage: 3866 MiB


Shouldn't be there 2GB limit for 32bit AvS? :confused:

[StainlessS]

I would have thought so.
Does RequestFramesLinear alloc its own mem ?
EDIT: Or LWLibavVideoSource.
By the way, simulator ignores chroma, so not exactly as original script (less mem & time overhead).

EDIT: Coffee is starting to work.
Shouldn't be there 2GB limit for 32bit AvS?
Not x86 AVS on x64 machine (And OS) [think 4GB limit, Groucho knows].
(Q9550 is 64 bit machine, as are all Core Duo, I think).

EDIT: Change to my prev post to
Core Duo Q9550 Quad core, W7 x64, AVS+ x86.

EDIT:
By the way, simulator ignores chroma, so not exactly as original script (less mem & time overhead).
Above slightly rubbish, fetches full frame (with chroma), just does not scan chroma so does not totally ignore chroma,
and probably similar mem overhead

[Groucho2004]

Not x86 AVS on x64 machine (And OS) [think 4GB limit, Groucho knows].If the application (in this case AVSMeter) has the LARGEADDRESSAWARE link flag set and is running on a 64bit OS, up to 4G are addressable.