MatteCrop - automatic framing and crop

[Dogway]

MatteCrop - automatic framing and crop


I made this function to fix certain films with varying and asymmetrical borders. You have two options, mode=0 which only recenters the frame within borders, and mode=2 which scales it to fill the width (no side borders).

I find this workflow more straight forward as it doesn't require an analysis pass, but you can scrub through the video and given you choose mode=0, find a minimal or maximal border size to set a fixed crop value (set width and height in DogCrop below original res), in case mode=2 (scaling) is a nono.


mode=0
http://i.imgur.com/2409bmKt.png (https://i.imgur.com/2409bmK.png)http://i.imgur.com/gBRb7EZt.png (https://i.imgur.com/gBRb7EZ.png)
mode=2
http://i.imgur.com/2409bmKt.png (https://i.imgur.com/2409bmK.png)http://i.imgur.com/HHyZEnKt.png (https://i.imgur.com/HHyZEnK.png)


It's not Credits proof, so I recommend you to use trims, to trim out problematic areas.

Download from my Github (https://github.com/Dogway/Avisynth-Scripts)


##############################
### Automatic cropping and/or centering function.
### It works more like a recentering function when borders are asymmetrical or varying between shots.
### Which makes it easier to crop later after inspection (with width/height params) to the minimum or maximal found border.
###
###
### "Width/Height" Destination width/height
### "thr" Threshold, pixel values above this will be considered borders
### "CropMore" In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
### "ScanW/ScanH" Range of pixels for scanning borders, set this to minimum to enhance performance
### "mode" Mode of the function. 0: centers the frame within borders
### 1: crops to minimum found borders
### 2: resize bordered shots to destination width/height (Default)
### "Kernel" Kernel to use for resizing
###
### i.e.
### DogCrop(1920,1080,thr=16.0,cropmore=true,ScanW=90,ScanH=0,mode=2)
###
###############################################################################

function DogCrop(clip c, int "width", int "height", float "thr", bool "CropMore", int "ScanW", int "ScanH", int "mode", string "kernel") {

c
w = width()
h = height()

nw = Default(width,w)
nh = Default(height,h)
addw = Default(ScanW,round((width()/5)))
addh = Default(ScanH,round((height()/4)))
thr = Default(thr, 16.0) # Threshold, pixel values above this will be considered borders
CM = Default(CropMore, False) # In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
mode = Default(mode, 2) # 0: center+pad 1: crop to minimum (WIP) 2: resize to maximum
kernel = Default(kernel, "spline36") # Kernel to use when resizing (mode=2)

sisphbd = AvsPlusVersionNumber > 2294
contoy = sisphbd ? !isy() : !isy8()

ScriptClip("""
contoy ? sisphbd ? converttoy() : converttoy8() : last

x1=0 x2=0
y1=0 y2=0

for (li=2, addw, 2) {
L1=crop(0,0,-w+li,0)
avgLL=AverageLuma(L1)
if (avgLL>thr) {
x1= addw == 0 ? 0 : CM ? li : li-2
li=addw
}
}

for (ri=2, addw, 2) {
R1=crop(w-ri,0,0 ,0)
avgLR=AverageLuma(R1)
if (avgLR>thr) {
x2= addw == 0 ? 0 : CM ? ri : ri-2
ri=addw
}
}

for (ti=2, addh, 2) {
T1=crop(0,0,0,-h+ti)
avgLT=AverageLuma(T1)
if (avgLT>thr) {
y1= addh == 0 ? 0 : CM ? ti : ti-2
ti=addh
}
}

for (bi=2, addh, 2) {
B1=crop(0,h-bi,0 ,0)
avgLB=AverageLuma(B1)
if (avgLB>thr) {
y2= addh == 0 ? 0 : CM ? bi : bi-2
bi=addh
}
}

crop(c,x1,y1,-x2,-y2)
mode == 0 ? padresize(w,h) : \
mode == 1 ? padresize(w,h) : \
RatioResize(float(nw),"adjust2w", kernel=kernel).padresize(w,h)

""",args="c,nw,nh,addw,addh,w,h,thr,CM,sisphbd,contoy,mode,kernel")
padresize(nw,nh)
}



I had two issues in case someone can help me optimize it further.
I could not use eval inside ScriptClip to skip when ScanH/ScanW is 0.
Using nnedi3 for RatioResize() calls nnedi3_resize16(), this causes a memory leak when used inside ScriptClip, is this a nnedi3 issue?

[real.finder]

Using nnedi3 for RatioResize() calls nnedi3_resize16(), this causes a memory leak when used inside ScriptClip, is this a nnedi3 issue?

it's Spline36Resize/Spline36ResizeMT

ColorBars(width=640, height=480, pixel_type="yv12")
ScriptClip("""
Spline36Resize(Width(),Height()*2).Spline36Resize(Width(),Height())
""")

edit: see here https://forum.doom9.org/showthread.php?p=1939782

[Dogway]

Currently I'm encoding with spline36 and if it leaks it seems to be minor, only 4Gb of RAM usage. With nnedi3 it would raise to 31Gb and I had to terminate it.

[real.finder]

nnedi3 =/= nnedi3_resize16

and seems you didn't see this https://forum.doom9.org/showthread.php?p=1939782

it's Dither_resize16 problem since nnedi3_resize16 use Dither_resize16, I did try nnedi3 alone and didn't note any leak

ColorBars(width=640, height=480, pixel_type="yv12")
ScriptClip("""
ConvertBits(16).ConvertToStacked.Dither_resize16(Width(),Height()*2).Dither_resize16(Width(),Height()).ConvertfromStacked.ConvertBits(8)
""")

[real.finder]

I could not use eval inside ScriptClip to skip when ScanH/ScanW is 0.


maybe this method help you https://github.com/Asd-g/AviSynthPlus-Scripts/blob/933fcf70f82b8c83d7efa74a1144151fadb1ba82/AdaptiveGrain.avsi#L66

same method you used in SMDegrain but seems you forget :)

[Dogway]

Oh my gosh... I've been prepending "if" before ternaries lol. Long time away from avs and too much with Python lately.
I wasn't using lsb on my nnedi3_resize16 () call so I didn't expect Dither_resize16 to be used, but I haven't inspected the code closely. Maybe the function needs an update since there are so many alternatives lately.

I'm going to update DogCrop() for moving mattes, that means pixel accuracy (maybe subpixel?) and optimize it a bit further.

[Dogway]

Here is the latest version (renamed to MatteCrop), now it samples the average per row of pixels instead of aggregated rows, this makes it faster. Also I implemented a 1 pixel accuracy for moving mattes.
There's some little jump on moving mattes since after all I have to crop on a mod2 basis, unless I convert to RGB which I want to avoid. I will test with a reencode of my source sample if it is too distracting.

Here before and after:
Source: https://www.mediafire.com/file/ao9zccmqlft8vlm
Processed: https://www.mediafire.com/file/vd8vajaccpl7449

##############################
### Automatic cropping and/or centering function.
### It works more like a recentering function when borders are asymmetrical or varying between shots.
### Which makes it easier to crop later after inspection (with width/height params) to the minimum or maximal found border.
###
###
### "Width/Height" Destination width/height
### "thr" Threshold, pixel values above this will be considered borders
### "CropMore" In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
### "ScanW/ScanH" Range of pixels for scanning borders, set this to minimum to enhance performance
### "Moving" If the matte is moving (sliding) it defaults to 1 pixel accuracy, among other settings (check defaults)
### "mode" Mode of the function. 0: centers the frame within borders
### 1: crops to minimum found borders (Work in Progress)
### 2: resize bordered shots to destination width/height (Default)
### "Kernel" Kernel to use for resizing
###
### i.e.
### MatteCrop(1920,1080,thr=16.0,cropmore=true,ScanW=90,ScanH=0,mode=2)
###
###############################################################################

function MatteCrop(clip c, int "width", int "height", float "thr", bool "CropMore", bool "Moving", int "ScanW", int "ScanH", int "mode", string "kernel") {

c
w = width()
h = height()

nw = Default(width,w)
nh = Default(height,h)
addw = Default(ScanW,int(round((width()/5))))
addh = Default(ScanH,int(round((height()/4))))
Mot = Default(Moving, False) # If the matte is moving (sliding) this enables pixel level accuracy.
CM = Default(CropMore, Mot ? True : False) # In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
thr = Default(thr, Mot ? 16.3 : 16.0) # Threshold, pixel values above this will be considered borders
mode = Default(mode, 2) # 0: center+pad 1: crop to minimum (WIP) 2: resize to maximum
kernel = Default(kernel, "spline36") # Kernel to use when resizing (mode=2)

sisphbd = AvsPlusVersionNumber > 2294
contoy = sisphbd ? !isy() : !isy8()

ScriptClip("""
contoy ? sisphbd ? converttoy() : converttoy8() : last

x1=0 x2=0
y1=0 y2=0

step = Mot ? 1 : 2

addw != 0 ? Eval("
for (li=step, addw, step) {
if (AverageLuma(crop(li-step,0,-w+li,0))>thr) {
x1= CM ? li : li-step
li=addw
}
}

for (ri=step, addw, step) {
if (AverageLuma(crop(w-ri,0,-ri+step,0))>thr) {
x2= CM ? ri : ri-step
ri=addw
}
}") : nop()


addh != 0 ? Eval("
for (ti=step, addh, step) {
if (AverageLuma(crop(0,ti-step,0,-h+ti))>thr) {
y1= CM ? ti : ti-step
ti=addh
}
}

for (bi=step, addh, step) {
if (AverageLuma(crop(0,h-bi,0 ,-bi+step))>thr) {
y2= CM ? bi : bi-step
bi=addh
}
}") : nop()


MotW = Mot ? round(w-x1-x2) : nop()
MotH = Mot ? round(h-y1-y2) : nop()

Mot ? spline36resize(c,MotW+MotW%2,MotH+MotH%2,src_left=x1,src_width=-x2,src_top=y1,src_height=-y2) : \
crop(c,x1,y1,-x2,-y2)

mode == 0 ? padresize(w,h) : \
mode == 1 ? padresize(w,h) : \
RatioResize(float(nw),"adjust2w", kernel=kernel).padresize(w,h)

""",args="c,nw,nh,addw,addh,w,h,thr,CM,sisphbd,contoy,mode,kernel,Mot",local=true)
padresize(nw,nh) }

Is there any speed difference between ScriptClip and gScriptClip?

[StainlessS]

Doggy,
If you have Cabaret [Minnelli/York] on DVD, suggest chop 10 minitue section starting at [I think] "Prairie Oyster" scene
[when Minnelli & York first meet] and test on that 10 mins.
I know that AutoCrop totally butchers it with its default settings. [just to get an idea of how well it performs in bad situations, maybe].

[Dogway]

There are some false-positives in the section where she changes the turntable, but nothing else otherwise (is the DVD bordered?).
Autocrop would overcrop a little overall. In any case I designed this for the source at OP, like digital mattes, for DVDs and some crappy borders I think maybe I would need to average a selectevery range of frames and maybe SC on top. Makes things more difficult. But if anyone can provide a sample I can give it a look.

[StainlessS]

(is the DVD bordered?)
I think is borders left and right, and maybe AR of something like 1.60.

According to IMDB
Runtime 2 hr 4 min (124 min)
Sound Mix Mono
Color Color (Technicolor)
Aspect Ratio 1.85 : 1
Laboratory Technicolor (as Technicolor®)
Negative Format 35 mm (Eastman 100T 5254)
Cinematographic Process Spherical
Printed Film Format 35 mm
My copy was freebie with newspaper, years ago, maybe was edited or something, I'll take another peek.
autocrop on my copy chopped off a good third off left edge I think, almost nothing left.

EDIT: By default, [original] autocrop only samples 5 frames.

[real.finder]

I wasn't using lsb on my nnedi3_resize16 () call so I didn't expect Dither_resize16 to be used, but I haven't inspected the code closely. Maybe the function needs an update since there are so many alternatives lately.

yes even without lsb Dither_resize16 still needed in nnedi3_resize16, I did update it but the update only for HBD case since there are no full replacement for Dither_resize16 unless someone backport https://github.com/EleonoreMizo/fmtconv

also there are SmoothAdjust that has no alternatives and the worst is close source so for HBD there are many things that will not work as I said before (but I will put it here with some edit)
new basic HBD support for nnedi3_resize16.avsi and ContraSharpen_mod.avsi, with HBD you will need ResizeX and most conversion parameters in nnedi3_resize16 will not do anything (since the input color formats = the output color formats), the output also is different in HBD (maybe it will be similar with FastNnediHBD=true)

so maybe you can try edi_rpow2() instead of nnedi3_resize16

Is there any speed difference between ScriptClip and gScriptClip?

in your case no, both will use GRunT since you use args

[Dogway]

Yeah, as you might expect I have been giving a glimpse to nnedi3_resize16() to find out what makes it so special about Dither.
I'm very tempted to simplify it but I'm aware that you are more apt to the task than me, if you haven't tried already. Can't you use cube instead of fmtconv? HBD on 420 is what most need in any case, to start things up.

[real.finder]

IIRC cube is RGB only, also I don't think anything can fully replace Dither_resize16 aside from fmtconv since fmtconv is like dither tools but for vs (it's made by same developer)

[Dogway]

AFAIK Dither tools also required conversion to RGB for color matrix operations, not sure about fmtconv but I doubt it's not the same, color space operations are always performed on RGB.

Updated MatteCrop to fix edges on moving mattes, and also added support for HBD.

##############################
### https://forum.doom9.org/showthread.php?t=182678
###
### Automatic cropping and/or centering function.
### It works more like a recentering function when borders are asymmetrical or varying between shots.
### Which makes it easier to crop later after inspection (with width/height params) to the minimum or maximal found border.
###
###
### "Width/Height" Destination width/height
### "thr" Threshold, pixel values above this will be considered borders
### "CropMore" In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
### "ScanW/ScanH" Range of pixels for scanning borders, set this to minimum to enhance performance
### "Moving" If the matte is moving (sliding) it defaults to 1 pixel accuracy, among other settings (check defaults)
### "mode" Mode of the function. 0: centers the frame within borders
### 1: crops to minimum found borders (Work in Progress)
### 2: resize bordered shots to destination width/height (Default)
### "Kernel" Kernel to use for resizing
###
###
### Dependencies (for Moving=true):
### ------------
### EdgeFixer (http://avisynth.nl/index.php/EdgeFixer)
###
###
### i.e.
### MatteCrop(1920,1080,thr=16.0,cropmore=true,ScanW=90,ScanH=0,mode=2)
###
###############################################################################

function MatteCrop(clip c, int "width", int "height", float "thr", bool "CropMore", bool "Moving", int "ScanW", int "ScanH", int "mode", string "kernel") {

c
w = width(c)
h = height(c)

sisphbd = AvsPlusVersionNumber > 2294
bdpth = sisphbd ? pow(256.,c.BitsPerComponent()/8.)/256. : 1.
contoy = sisphbd ? !isy() : !isy8()
fullchr = contoy ? UtoY().width() == w : true

nw = Default(width,w)
nh = Default(height,h)
addw = Default(ScanW,int(round((w/5))))
addh = Default(ScanH,int(round((h/4))))
Mot = Default(Moving, False) # If the matte is moving (sliding) this enables pixel level accuracy.
CM = Default(CropMore, Mot || fullchr ? True : False) # In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
thr = Default(thr, Mot ? 16.3 : 16.0) # Threshold, pixel values above this will be considered borders
mode = Default(mode, 2) # 0: center+pad 1: crop to minimum (WIP) 2: resize to maximum
kernel = Default(kernel, "spline36") # Kernel to use when resizing (mode=2)

thr = thr*bdpth


pclip=contoy ? sisphbd ? converttoy() : converttoy8() : last

ScriptClip("""

x1=0 x2=0
y1=0 y2=0

step = Mot ? 1 : 2

pclip
addw != 0 ? Eval("
for (li=step, addw, step) {
if (AverageLuma(crop(li-step,0,-w+li,0))>thr) {
x1= CM ? li : li-step
li=addw
}
}

for (ri=step, addw, step) {
if (AverageLuma(crop(w-ri,0,-ri+step,0))>thr) {
x2= CM ? ri : ri-step
ri=addw
}
}") : nop()


addh != 0 ? Eval("
for (ti=step, addh, step) {
if (AverageLuma(crop(0,ti-step,0,-h+ti))>thr) {
y1= CM ? ti : ti-step
ti=addh
}
}

for (bi=step, addh, step) {
if (AverageLuma(crop(0,h-bi,0 ,-bi+step))>thr) {
y2= CM ? bi : bi-step
bi=addh
}
}") : nop()


MotW = Mot ? round(w-x1-x2) : nop()
MotH = Mot ? round(h-y1-y2) : nop()

Mot ? spline36resizeMT(c,fullchr?MotW:MotW+MotW%2,fullchr?MotH:MotH+MotH%2,src_left=x1,src_width=-x2,src_top=y1,src_height=-y2) : \
crop(c,x1,y1,-x2,-y2)

mode == 0 ? padresize(w,h) : \
mode == 1 ? padresize(w,h) : \
RatioResize(float(w),"adjust2w", kernel=kernel).padresize(w,h,mirror=!CM)

""",args="c,nw,nh,addw,addh,w,h,thr,CM,sisphbd,contoy,mode,kernel,Mot,fullchr,pclip",local=true)
padresize(nw,nh)
(!CM || Mot) && mode==2 ? ContinuityFixer(left=addw>0?2:0, top=addh>0?2:0, right=addw>0?2:0, bottom=addh>0?2:0, radius=CM && w>720?0:1) : last }

[StainlessS]

Doggy, this is what it looks like, borders on all 4 sides [have not as yet ascertained aspect ratio]
https://i.postimg.cc/ZW7B3R35/c-00.jpg (https://postimg.cc/ZW7B3R35)
EDIT:
OK, DAR = about 1.817687, or thereabouts, not 1.60 like I said, musta been the side borders that put me off the scent.
https://i.postimg.cc/6y75S0Gt/c-01.jpg (https://postimg.cc/6y75S0Gt)


# Mpeg2Source(...)
DAR=4.0/3.0 # full frame
SAR=Last.RT_GetSAR(DAR) # Sample ie pixel aspect ratio
X=RoboCrop() # auto cropped
DAR=X.RT_GetDAR(SAR) # DAR after crop
RoboCrop(Show=true) # Now re-do crop but keep borders, show where cropped
Subtitle("DAR="+String(DAR),Align=2) # Show DAR


EDIT: Above image showing the RoboCrop cursors I was talking about, ie the crop indicators.
Double or wiggly lines [top, right, bottom] are where extra cropping was applied due to colorspace
or default xmod/ymod requirements [I just used robocrop without specifying WMod/HMod and whether interlaced]
EDIT: Convert to YV24 and WMod=1,HMod=1,Laced=False, DAR=1.802020, so as exact detected edges

ConvertToYV24
DAR=4.0/3.0
SAR=Last.RT_GetSAR(DAR)
X=RoboCrop(WMod=1,HMod=1,Laced=false)
DAR=X.RT_GetDAr(SAR)
RoboCrop(WMod=1,HMod=1,Laced=false,Show=true)
Subtitle("DAR="+String(DAR),Align=2)

https://i.postimg.cc/v4tBLmVV/c-02.jpg (https://postimg.cc/v4tBLmVV)

[Dogway]

The script works better than autocrop as in it doesn't butcher the left side, but it's not designed for these kind of sources with dirty mattes nor even fixed mattes (just manually insert crop()? ) but varying border mattes or border sizes, that's why the code is in runtime. In any case I crafted some kind of prefilter and got very stable results with some occasional 1 pixel matte resize, which in my book isn't allowed neither. Ideally one would want to create equally weighted temporal averages of shots, I had a look yesterday, I'm sure it can be done using DBSC or some of your tools.

One main problem with your sample is that there's a bright column right in the middle of the border, which triggers the detection and forces you to raise the thr therefore having that jumping matte as result (I tried to remove it with median in prefilter).

https://i.postimg.cc/WDvmxH9k/c000812.png (https://i.postimg.cc/QMB6KL10/c000812.png)

As for Robocrop it detects the furthermost border, so for my source it will leave lots of innermost borders. Yesterday I gave a thorought look to DBSC following your guide for dynamic cropping and I managed to get it somewhere, but there were some borders left untouched, some overcropping, some anamorphic scaling, and no work done on moving mattes.

This is what I get out of Robocrop with the following settings:

######## Temp OUTER RoboCrop args (to remove outermost/common border)
RC_SAMPLES = Max(Round(FrameCount/(FrameRate*1.0)),12) # Sample 1 Frame every 1 seconds,
RC_IGNORE = 0.04
RC_WMOD = 2 # Width multiple of this
RC_HMOD = 2 # Height multiple of this
RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 12
RC_RIGHTSKIP = 0
RC_BOTSKIP = 12

CropLimit_Left=85 CropLimit_Top=12 CropLimit_Right=0 CropLimit_Bot=12 # EDIT: The dynamic border cropping max's (also see RLBT)

https://i.postimg.cc/WD8ZCbFB/New-File-8-004817.png (https://i.postimg.cc/LXKVqJH8/New-File-8-004817.png)

And this is the result after removing autocontrast (vertical stretched):

https://i.postimg.cc/G9CfCXfd/New-File-8-004817-2.png (https://i.postimg.cc/MZMJ14bM/New-File-8-004817-2.png)

The function name might sound misleading, I wanted to call it DynaCrop but the seat was not vacant : P

[hello_hello]

I tried DBSC a while back and was on the verge of suicide before I managed to get it to work almost properly (there was still some tweaking needed for it not to over-crop in a couple of places). I just love StainlessS's help files. :)

For the DynaCrop stretching problem, I'll throw in a plugin for my CropResize function.
https://forum.doom9.org/showthread.php?t=176667
If the source is anamorphic you could specify input and/or output sample aspect ratios (not an input display aspect ratio in this case though).

Some silly cropping and adding of borders for testing.

A = Crop(20,20,-20,-20).AddBorders(20,20,20,20).Trim(30000,30099)
B = Crop(100,20,-10,0).AddBorders(100,20,10,0).Trim(30000,30099)
A + B

Frame #50

https://i.postimg.cc/fVSxV09r/B1.jpg (https://postimg.cc/fVSxV09r)

Frame #150

https://i.postimg.cc/fkMj8LGV/B2.jpg (https://postimg.cc/fkMj8LGV)

DBSC_DynaCrop("D:\New.ScanDB", "D:\New.SpliceDB", 640,480, CropLimit=100, CropThresh=16)

Frame #50

https://i.postimg.cc/Yj8NZ77b/1.jpg (https://postimg.cc/Yj8NZ77b)

Frame #150

https://i.postimg.cc/nCP4V1Cm/2.jpg (https://postimg.cc/nCP4V1Cm)

DBSC_DynaCrop("D:\New.ScanDB", "D:\New.SpliceDB", 640,480, Resizer="CropResize(_W_,_H_)", CropLimit=100, CropThresh=16)

Frame #150

https://i.postimg.cc/w7v5drTH/3.jpg (https://postimg.cc/w7v5drTH)

DBSC_DynaCrop("D:\New.ScanDB", "D:\New.SpliceDB", 640,480, Resizer="CropResize(_W_,_H_, Borders=true)", CropLimit=100, CropThresh=16)

Frame #150

https://i.postimg.cc/bdhXwQKs/4.jpg (https://postimg.cc/bdhXwQKs)

DBSC_DynaCrop("D:\New.ScanDB", "D:\New.SpliceDB", 640,480, Resizer="CropResize(_W_,_H_, Borders=true, Frosty=true)", CropLimit=100, CropThresh=16)

Frame #150

https://i.postimg.cc/n9QH4ZGp/5.jpg (https://postimg.cc/n9QH4ZGp)

While I was messing around I made a new wrapper function for Dogway's SpliceResize.

CropResize(640,480, Resizer="CR_SpliceResize")

# ===============================================================================
# ========== CR_SpliceResize() - Bicubic & Spline64 Wrapper Function ============
# ===============================================================================

# Requires MaskTools2 http://avisynth.nl/index.php/Masktools2

# SpliceResize by Dogway https://github.com/Dogway/Avisynth-Scripts/blob/master/ResizersPack4.7.3.avsi
# Resizer which gets the best from 2 worlds, the "sharpness" of spline64 and the "ringing-free" catmull-rom,
# biased by a threshold.
# Slightly mod from javlak's function http://forum.doom9.org/showthread.php?p=1504678#post1504678

function CR_SpliceResize(clip Source, int target_width, int target_height, \
float "src_left", float "src_top", float "src_width", float "src_height", float "B", float "C", int "Threshold") {

Threshold = default (Threshold, 8)
B = default(B, 0.0)
C = default(C, 0.5)

mt_lutxy(BicubicResize(Source, target_width, target_height, \
src_left=src_left, src_top=src_top, src_width=src_width, src_height=src_height, b=B, c=C), \
Spline64Resize(Source, target_width, target_height, \
src_left=src_left, src_top=src_top, src_width=src_width, src_height=src_height), \
"x y - abs "+string(threshold)+" > y x ?") }

# ===============================================================================

[Dogway]

Yes, with CropResize it doesn't stretch anymore but still lots of overcropping and left out borders, (it tends to prefer cropping vertically than horizontally), not to talk about sliding mattes. Overcropping on digital borders seems like a big mistake to oversee, or probably I'm doing it wrong.
For clean borders such as BluRays I'm content with my function 99%, and to nail it to extend it to DVDs would be per shot averages... now I recall RT_Stats providing such info... I should investigate. EDIT: Yes, RT_AverageLuma() but doesn't have SC.


Answering below:
Yes, it's set as LACED=false as indicated in the client script

OCROP = ORG.RoboCrop(SAMPLES=RC_SAMPLES,ignore=RC_IGNORE,WMod=RC_WMOD,HMod=RC_HMOD,
\ LeftSkip=RC_LEFTSKIP,TopSkip=RC_TOPSKIP,RightSkip=RC_RIGHTSKIP,botskip=RC_BOTSKIP,
\ LogFn=TmpCropLog,LogAppend=false,Align=True,laced=False)

[StainlessS]

One thing I will say is that you should specify LACED=False [where so], otherwise YMOD [default gotten from natural colorspace croppable granularity]
will be doubled if default is left at LACED=true.
I may come back again later.

EDIT:
In the provided DBSC sample script

RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 2
RC_RIGHTSKIP = 0
RC_BOTSKIP = 2

2 values above should probably have been 0, my test clip had half scanline black, top and bottom [I dont like that].

As for Robocrop it detects the furthermost border, so for my source it will leave lots of innermost borders. Yesterday I gave a thorought look to DBSC following your guide for dynamic cropping and I managed to get it somewhere, but there were some borders left untouched, some overcropping, some anamorphic scaling, and no work done on moving mattes.
Just to be clear, in DBSC, it is necessary that initial RoboCrop is done to get rid of outer/common border, before it creates DBase of scene changes, otherwise everything will be screwed.
CropLeft_limit etc should be set to [ideally] same or 1 pixel more than greatest thickness border [which is alread pre-Robocropped].
DBSC_ShowCrop() should NOT be called separately without pre-RoboCrop.

CropThresh=-40.0,

Above -ve is auto thresholding, using eg +16 is absolute thresholding and maybe best avoided.
Auto thresholding, measures min and max luma in [already pre-robocropped] source, and in case of -40, sets threshold
to minimum Y level + 40. [Old default was -32]. +16 is probably way too low, but -16 might be OK if you know what you are doing.

Pre-Robocrop, gets rid of dark common border which removes those numbers, better measurements.

EDIT: I've never played with blueray stuff, I've no idea what its like.

[hello_hello]

DynaCrop cropped just the borders for my example, except for maybe the 10 pixel border. I haven't looked closely at that one. The over-cropping in one screenshot was done by my script to prevent aspect error, and I think it defaults to mod4 even when adding borders, so any extra cropping when resizing and adding borders would've been my fault too.
I assume you could tell DynaCrop to leave the top and bottom borders alone if you want to keep them, but I was experimenting with removing them all, as that's what I'd normally do.

StainlessS,
I forgot about the Laced argument, although except for one, the borders were all mod4 anyway.

[Dogway]

I would test with no mod borders, in my case it crops more than 4 pixels anyway. I also did a second test with more strict settings to proof myself wrong but I get both mentioned issues.
It easily fails on dark scenes, yours is well lit at borders I think.

RC_IGNORE = 0.00
RC_WMOD = 2 # Width multiple of this
RC_HMOD = 2 # Height multiple of this
RC_LEFTSKIP = 0 # Crop at least these
RC_TOPSKIP = 0
RC_RIGHTSKIP = 0
RC_BOTSKIP = 0

[StainlessS]

If your borders are SOLID single color, the yes RC_IGNORE=0.0 sounds good.

I'm gonna go play with FredAverage, and add mask like I said earlier today.

[Dogway]

I have been trying to come up with solutions for longer variable lifetime set at runtime functions, like inside scriptclip, the next is a proof of concept:

function MatteCrop2(clip c, int "width", int "height", float "thr", bool "CropMore", bool "Moving", int "ScanW", int "ScanH", int "mode", string "kernel", float "SC_thr") {

c
w = width(c)
h = height(c)

sisphbd = AvsPlusVersionNumber > 2294
bdpth = sisphbd ? pow(256.,c.BitsPerComponent()/8.)/256. : 1.
contoy = sisphbd ? !isy() : !isy8()
fullchr = contoy ? sisphbd ? ExtractU().width() == w : UToY8().width() == w : true

nw = Default(width,w)
nh = Default(height,h)
addw = Default(ScanW,int(round((w/5))))
addh = Default(ScanH,int(round((h/4))))
Mot = Default(Moving, False) # If the matte is moving (sliding) this enables pixel level accuracy.
CM = Default(CropMore, Mot || fullchr ? True : False) # In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
thr = Default(thr, Mot ? 16.3 : 16.0) # Threshold, pixel values above this will be considered borders
mode = Default(mode, 2) # 0: center+pad 1: crop to minimum (WIP) 2: resize to maximum
SC_thr = Default(SC_thr, 3.5)
kernel = Default(kernel, "spline36") # Kernel to use when resizing (mode=2)

thr = thr*bdpth

avg = contoy ? sisphbd ? converttoy() : converttoy8() : last

avg = avg.RatioResize(640.0,"adjust2w",kernel="bicubic")

blk_b = avg.blankclip(pixel_type="Y8",fps=framerate(c),color=$000000).killaudio()
blk_w = avg.blankclip(pixel_type="Y8",fps=framerate(c),color=$FFFFFF).killaudio()

avg = avg.TemporalSoften(10,255,255,10,2)
avg = avg.TemporalSoften(10,255,255,10,2)
avgsc= RatioResize(float(w),"adjust2w",kernel="bicubic").converttoyv12()
SC = SCSelect_HBD(avg,blk_w,blk_b,blk_b,dfactor=SC_thr,mindif=1.0)

ScriptClip("""

isSC=YPlaneMax(SC)>128

# Obviously Defined() doesn't work here
Defined(x1) || Defined(y1) ? Eval("
for (LB=0, 1440, 1) {
if (YPlaneMax(trim(SC,current_frame-LB,current_frame-LB))>128) {
LBF=current_frame-LB
LB=1440
}
}
") : nop()

trim(avgsc,LBF,LBF)


x1=0 x2=0
y1=0 y2=0

step = Mot ? 1 : 2

addw != 0 ? Eval("
for (li=step, addw, step) {
if (AverageLuma(crop(li-step,0,-w+li,0))>thr) {
global x1= CM ? li : li-step
li=addw
}
}

for (ri=step, addw, step) {
if (AverageLuma(crop(w-ri,0,-ri+step,0))>thr) {
global x2= CM ? ri : ri-step
ri=addw
}
}") : nop()


addh != 0 ? Eval("
for (ti=step, addh, step) {
if (AverageLuma(crop(0,ti-step,0,-h+ti))>thr) {
global y1= CM ? ti : ti-step
ti=addh
}
}

for (bi=step, addh, step) {
if (AverageLuma(crop(0,h-bi,0 ,-bi+step))>thr) {
global y2= CM ? bi : bi-step
bi=addh
}
}") : nop()

#subtitle(string(y2),x=30,y=10,align=5)

MotW = Mot ? round(w-x1-x2) : nop()
MotH = Mot ? round(h-y1-y2) : nop()
Mot ? spline36resizeMT(c,fullchr?MotW:MotW+MotW%2,fullchr?MotH:MotH+MotH%2,src_left=x1,src_width=-x2,src_top=y1,src_height=-y2) : \
crop(c,x1,y1,-x2,-y2)

mode == 0 ? padresize(w,h) : \
mode == 1 ? padresize(w,h) : \
RatioResize(float(w),"adjust2w", kernel=kernel).padresize(w,h,mirror=false)

""",args="c,SC,avgsc,addw,addh,w,h,thr,CM,Mot,w,h,fullchr,kernel,mode",local=true, after_frame=false)


padresize(nw,nh)
(!CM || Mot) && mode==2 ? ContinuityFixer(left=addw>0?2:0, top=addh>0?2:0, right=addw>0?2:0, bottom=addh>0?2:0, radius=CM && w>720?0:1) : last
}


I'm having a hard time setting and holding x1,x2,y1 and y2 variables available as long as there's no a new scene change. Indeed Global doesn't cut it, so for the time being I'm shifting frames in a for loop until a scene change is detected but this is very slow. I tried Gavino's solution here (https://forum.doom9.org/showthread.php?p=1486364#post1486364)without success though.

[Dogway]

Unfortunately I still don't know how to manage variable lifetime of runtime functions, but needed to finish the script for some projects so I went down the rabbit hole and found a method, it does the same and it's very fast, false positives are reduced to a minimum. I still need to do some slight fine tuning but this is pretty much the script done. I even added a hybrid method where IMAX scenes would center in -mode 0- (so content is not cropped) but 2.40 scenes are upscaled to fit -mode 2-.



function MatteCrop(clip c, int "width", int "height", float "thr", bool "CropMore", bool "Moving", int "ScanW", int "ScanH", int "mode", string "kernel", float "SC_thr") {

c
w = width(c)
h = height(c)

sisphbd = AvsPlusVersionNumber > 2294
bdpth = sisphbd ? pow(256.,c.BitsPerComponent()/8.)/256. : 1.
contoy = sisphbd ? !isy() : !isy8()
fullchr = contoy ? sisphbd ? ExtractU().width() == w : UToY8().width() == w : true

nw = Default(width,w)
nh = Default(height,h)
addw = Default(ScanW,int(round((w/8))))
addh = Default(ScanH,int(round((h/4))))
Mot = Default(Moving, False) # If the matte is moving (sliding) this enables pixel level accuracy.
CM = Default(CropMore, Mot || fullchr ? True : False) # In case of odd cropping, either crop 1 pixel out or leave 1 pixel of the border
thr = Default(thr, Mot ? 16.3 : 16.0) # Threshold, pixel values same or below this will be considered borders
mode = Default(mode, 2) # 0: center+pad 1: crop to minimum (WIP) 2: resize to maximum 3:hybrid 0-2 (auto)
SC_thr = Default(SC_thr, 3.5)
kernel = Default(kernel, "spline36") # Kernel to use when resizing (mode=2)

addw==0 ? Assert( addh>0, "You need to scan borders for at least one of the dimension") : nop()
addh==0 ? Assert( addw>0, "You need to scan borders for at least one of the dimension") : nop()

thr = thr*bdpth

avg = contoy ? sisphbd ? converttoy() : converttoy8() : last

avg = avg.RatioResize(320.0,"adjust2w",kernel="bicubic")

blk_b = avg.blankclip(width=16,height=16,pixel_type="Y8",fps=framerate(c),color=$000000).killaudio()
blk_w = avg.blankclip(width=16,height=16,pixel_type="Y8",fps=framerate(c),color=$FFFFFF).killaudio()

avg = avg.TemporalSoften(10,255,255,10,2)
avg = avg.TemporalSoften(10,255,255,10,2)
# need to turn converttoyv12 into other formats as source
avg = avg.RatioResize(float(w),"adjust2w",kernel="bicubic").MatchColorFormat(c)
avgsc= Overlay(c.greyscale(),avg,mask=BoxMaskf(addw,w-addw,addh,h-addh,show=true),greymask=true, mode="blend", opacity=1.0)

SC = SCSelect_HBD(avg,blk_w,blk_b,blk_b,dfactor=SC_thr,mindif=1.0)

ScriptClip("""

step = Mot ? 1 : 2
LBF=current_frame
x1=0 x2=0
y1=0 y2=0

# 1440 frames -1 min on 24fps- lookback
YPlaneMax(SC)<128 ? Eval("
for (LB=0, 1440, 1) {
if (YPlaneMax(trim(SC,current_frame-LB,-1))>128) {
LBF=current_frame-LB
LB=1440
}
}") : nop()


trim(avgsc,LBF+10,-1)


addw > 0 ? Eval("
for (li=step, addw, step) {
if (AverageLuma(crop(li-step,0,-w+li,0,align=true))>thr) {
x1= CM ? li : li-step
li=addw
}
}

for (ri=step, addw, step) {
if (AverageLuma(crop(w-ri,0,-ri+step,0,align=true))>thr) {
x2= CM ? ri : ri-step
ri=addw
}
}") : nop()


addh > 0 ? Eval("
for (ti=step, addh, step) {
if (AverageLuma(crop(0,ti-step,0,-h+ti,align=true))>thr) {
y1= CM ? ti : ti-step
ti=addh
}
}

for (bi=step, addh, step) {
if (AverageLuma(crop(0,h-bi,0 ,-bi+step,align=true))>thr) {
y2= CM ? bi : bi-step
bi=addh
}
}") : nop()


MotW = Mot ? round(w-x1-x2) : nop()
MotH = Mot ? round(h-y1-y2) : nop()
Mot ? spline36resizeMT(c,fullchr?MotW:MotW+MotW%2,fullchr?MotH:MotH+MotH%2,src_left=x1,src_width=-x2,src_top=y1,src_height=-y2) : \
crop(c,x1,y1,-x2,-y2,align=true)

mode == 0 ? padresize(w,h) : \
mode == 1 ? padresize(w,h) : \
mode == 2 ? RatioResize(float(w),"adjust2w", kernel=kernel).padresize(w,h,mirror=false) : \
YPlaneMax(crop( addh > 0 ? width()-16 : 0,0,0, addw > 0 ? -height()+16 : 0)) <= thr ? \
RatioResize(float(w),"adjust2w", kernel=kernel).padresize(w,h,mirror=false) : padresize(w,h)

""",args="c,SC,avgsc,addw,addh,w,h,thr,CM,Mot,fullchr,kernel,mode",local=true)


padresize(nw,nh)
(!CM || Mot) && mode==2 ? ContinuityFixer(left=addw>0?2:0, top=addh>0?2:0, right=addw>0?2:0, bottom=addh>0?2:0, radius=CM && w>720?0:1) : last
}

I plan to fix nnedi3_resize16() to simplify it and remove the dither dependency so borders are upscaled sharply without subpixel blending. The filter is designed towards blurays/UHDs, you can still use it with garbage borders found in DVDs/VHSs but YMMV.

[StainlessS]

Doggy, can you supply a short testclip and client script, thanks.
[I've found your ResizersPack4.7.4.avsi]

EDIT: As a very temporary hack for global survival, you could use RT_Stats DBase or array,
and preset fields/elements with eg -1, not set. Then you only need to use a fixed DBase name
to read or write, vars can even survive a reboot.

EDIT:

DB = "My.DB".RT_GetFullPathName

X1_FLD = 0
Y1_FLD = 1
X2_FLD = 2
Y2_FLD = 3

RECORD_0 = 0

TypeString="iiii" # 4 fields of type int

RT_DBaseAlloc(DB,1,TypeString) # Single pre-existing record [int field defaults are set to 0]

RT_DBaseSet(DB,RECORD_0,-1,-1,-1,-1) # PreSet all vars -1

X1 = RT_DBaseGetField(DB, RECORD_0, X1_FLD)
Y1 = RT_DBaseGetField(DB, RECORD_0, Y1_FLD)
X2 = RT_DBaseGetField(DB, RECORD_0, X2_FLD)
Y2 = RT_DBaseGetField(DB, RECORD_0, Y2_FLD)


# ...

RT_DBaseSetField(DB, RECORD_0, X1_FLD, X1)
RT_DBaseSetField(DB, RECORD_0, Y1_FLD, Y1)
RT_DBaseSetField(DB, RECORD_0, X2_FLD, X2)
RT_DBaseSetField(DB, RECORD_0, Y2_FLD, Y2)

# OR

RT_DBaseSet(DB, RECORD_0, X1,Y1,X2,Y2)


EDIT: Even though file based, DBase is fast from harddrive, certainly faster than Global access on v2.60, not sure about Avs+ as
Avs+ implemented hash table for Globals access.

EDIT: DBase is also a way to return multiple results from a function, supply the function with DBase name, and function
returns results in DB, with perhaps success status as actual function return.

[Dogway]

I sent you a PM. I still need to refine code for sliding mattes, repurpose width and height arguments depending on mode (pad or resize)

ffvideosource("title_t00_trans1.mkv")

MatteCrop(1920,1080,thr=17.0,cropmore=true,scanH=0,scanW=50,mode=3,kernel="spline36",moving=false)

Use the function from my post, as I still haven't updated ResizersPack.


As for variable lifetime, I'm precisely trying to avoid creating intermediary files or analysis passes, but I will have a look deeper in case it helps.

[StainlessS]

Thanks Dog, I got the clip.
I wanted the resize pack for the other functions, RatioResize, PadResize and PadMirror.

EDIT:
The DBase thing could be used in realtime, not multiple pass thingy, and you can even delete the Dbase on clip closure using CallCmd().
EDIT: This part would be done before entering scriptclip, ie create and preset DBase,
And supply DB name as arg to scriptclip,

DB = "My.DB".RT_GetFullPathName

X1_FLD = 0
Y1_FLD = 1
X2_FLD = 2
Y2_FLD = 3

RECORD_0 = 0

TypeString="iiii" # 4 fields of type int

RT_DBaseAlloc(DB,1,TypeString) # Single pre-existing record [int field defaults are set to 0]

RT_DBaseSet(DB,RECORD_0,-1,-1,-1,-1) # PreSet all vars -1

Using the RT_DBaseSet and RT_DBaseGet within Scriptclip.
The RECORD_0 and X1_FLD stuff is just for self documentation, you would likely use constants instead of variables.
During script developement, it is sometimes useful to use vars like X1_FLD etc so things eg field order can be changed around
without changing entire script. When completed, then can simply do a mass text Replace of eg X1_FLD with
the final chosen constant.

[Dogway]

I will try to play with RT_DBaseSet() soon when I recover from yesterday lol, at first sight it looks promising although currently the biggest help would be from finding a method to average ranges of frames based on SCSelect_HBD. Currently I'm doing a convoluted method of stacking TemporalSoften() (with its own SC) and shifting the parsing frame 10 frames into the future from SC for better consistency.

EDIT: just updated ResizersPack (https://github.com/Dogway/Avisynth-Scripts/blob/master/ResizersPack4.7.9.avsi).

[StainlessS]

finding a method to average ranges of frames
Yeah, what you want is tricky.
If you can find range then below might be useful.
ClipBlend() :- https://forum.doom9.org/showthread.php?t=168048&highlight=clipblend

[Dogway]

I tested your workflow a few months ago without realizing it also involved writing to disk.
I updated the function today. I used a mix of frame properties for scene change (modded your SCSelect_HBD function) and also your ClipBoard plugin for variable life. The script is now like 50% faster.
I wanted to also use properties for the x1,x2,y1,y2 coords, but frame properties didn't cut it, I would need clip properties, and update them every time there's a SC. I read you can use frame 0 as a placeholder for clip properties, might be useful, need to try.