SmartCrop.avsi — per-frame adaptive border removal via ScriptClip

[shurik_pronkin]

SmartCrop.avsi — per-frame adaptive border removal via ScriptClip

The problem

Film-based sources (telecine from physical reels) and analog broadcast recordings often have black borders that vary: different thickness on different sides, shifting between reel changes or across the recording. A static Crop() doesn't work — you either leave borders in or eat into content on other segments.

The idea

AviSynth's Resize functions accept src_left, src_top, src_width, src_height parameters that define a source window within the input frame. The output dimensions remain fixed. This means we can vary the crop region per frame inside ScriptClip while the filter graph sees constant dimensions — no recompilation, single pass, fully transparent to downstream filters.

How it works

For each frame, scan from the edge inward, one row/column at a time. Measure average luma via RT_AverageLuma. If it's at or below the threshold — that's border, keep going. First line above the threshold — stop, that's content.

Detection order: top/bottom first (full-width rows), then left/right within the detected content vertical range, so horizontal borders don't contaminate column averages.

Dark scene safeguard: if the scan reaches the depth limit without finding content, it assumes the dark area is part of the scene, not a border, and returns zero for that side. This prevents dark scenes from being clipped. Set scan depth with some margin above your expected maximum border width.

Per-side margins (T/B/L/R) add extra pixels beyond what was detected — useful for gradient borders from analog sources where the transition isn't a hard edge. Margins are only applied when a border is actually detected. Result is always rounded up to even.

Requirements


AviSynth+ (ScriptClip with closures)
RT_Stats by StainlessS (RT_AverageLuma)


Functions

SmartCropDebug(clip c, int "threshold", int "T", int "B", int "L", int "R",
int "scan_v", int "scan_h")

SmartCrop(clip c, int target_w, int target_h, string "mode", int "threshold",
int "T", int "B", int "L", int "R",
int "scan_v", int "scan_h", string "resizer")

Parameters

threshold avg luma ≤ this = border default 32
T,B,L,R extra pixels added per side default 0
scan_v max pixels to scan from top/bottom default 30
scan_h max pixels to scan from left/right default 30
resizer resize kernel default "Spline36Resize"
mode "stretch" — fill target exactly default "stretch"
"fit" — preserve AR, center, pad black

Examples

Import("SmartCrop.avsi")
src = LWLibavVideoSource("film.mkv")

# 1. Tune — watch overlay while scrubbing through reel transitions
SmartCropDebug(src, threshold=40, R=4)

# 2. Apply — crop borders, stretch to target
SmartCrop(src, 720, 576, threshold=40, R=4)

# 3. Fit mode — preserve AR, center content, pad with black
SmartCrop(src, 1920, 1080, mode="fit", threshold=40, L=2, R=4)

# 4. Analog broadcast — wide sidebars, narrow top/bottom
SmartCrop(src, 960, 720, mode="fit", scan_v=30, scan_h=220)

Source

###############################################################################
# SmartCrop.avsi — single-pass adaptive crop via ScriptClip
#
# Scan from edge inward. Dark (avg luma ≤ threshold) → border.
# First bright line → stop. If scan hits depth limit without finding
# content → dark scene, not border → no crop on that side.
# Per-side margin (T/B/L/R). Round up to even.
# Resize src_ → output always constant.
#
# Dependencies: RT_Stats (RT_AverageLuma)
###############################################################################


function SC_Top(clip c, int thr, int sd, int mg) {
w = c.Width()
n = current_frame
b = 0
s = false
for (y = 0, sd - 1, 1) {
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=0, y=y, w=w, h=1)
s = s ? true : (a > thr)
b = s ? b : y + 1
}
b = (b >= sd) ? 0 : b
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}

function SC_Bot(clip c, int thr, int sd, int mg) {
w = c.Width()
h = c.Height()
n = current_frame
b = 0
s = false
for (dy = 0, sd - 1, 1) {
y = h - 1 - dy
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=0, y=y, w=w, h=1)
s = s ? true : (a > thr)
b = s ? b : dy + 1
}
b = (b >= sd) ? 0 : b
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}

function SC_Left(clip c, int thr, int sd, int mg, int top, int bot) {
h = c.Height()
cy = top
ch = Max(1, h - top - bot)
n = current_frame
b = 0
s = false
for (x = 0, sd - 1, 1) {
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=x, y=cy, w=1, h=ch)
s = s ? true : (a > thr)
b = s ? b : x + 1
}
b = (b >= sd) ? 0 : b
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}

function SC_Right(clip c, int thr, int sd, int mg, int top, int bot) {
w = c.Width()
h = c.Height()
cy = top
ch = Max(1, h - top - bot)
n = current_frame
b = 0
s = false
for (dx = 0, sd - 1, 1) {
x = w - 1 - dx
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=x, y=cy, w=1, h=ch)
s = s ? true : (a > thr)
b = s ? b : dx + 1
}
b = (b >= sd) ? 0 : b
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}


function SmartCrop(clip c, int target_w, int target_h, \
string "mode", int "threshold", \
int "T", int "B", int "L", int "R", \
int "scan_v", int "scan_h", string "resizer") {
mode = Default(mode, "stretch")
thr = Default(threshold, 32)
mT = Default(T, 0)
mB = Default(B, 0)
mL = Default(L, 0)
mR = Default(R, 0)
sv = Default(scan_v, 30)
sh = Default(scan_h, 30)
resizer = Default(resizer, "Spline36Resize")

Assert(target_w % 2 == 0 && target_h % 2 == 0, "SmartCrop: target must be mod-2")

base = Eval(resizer + "(c, target_w, target_h)")

return (mode == "fit") \
? SC_Fit(base, c, target_w, target_h, thr, mT, mB, mL, mR, sv, sh, resizer) \
: SC_Stretch(base, c, target_w, target_h, thr, mT, mB, mL, mR, sv, sh, resizer)
}

function SC_Stretch(clip base, clip src, int tw, int th, \
int thr, int mT, int mB, int mL, int mR, \
int sv, int sh, string resizer) {
ScriptClip(base, function [src, tw, th, thr, mT, mB, mL, mR, sv, sh, resizer] () {
top = SC_Top(src, thr, sv, mT)
bot = SC_Bot(src, thr, sv, mB)
lft = SC_Left(src, thr, sh, mL, top, bot)
rgt = SC_Right(src, thr, sh, mR, top, bot)
Eval(resizer + "(src, tw, th, " \
+ "src_left=" + String(lft) + ".0" \
+ ", src_top=" + String(top) + ".0" \
+ ", src_width=" + String(Max(16, src.Width()-lft-rgt)) + ".0" \
+ ", src_height=" + String(Max(16, src.Height()-top-bot)) + ".0)")
})
}

function SC_Fit(clip base, clip src, int tw, int th, \
int thr, int mT, int mB, int mL, int mR, \
int sv, int sh, string resizer) {
ScriptClip(base, function [src, tw, th, thr, mT, mB, mL, mR, sv, sh, resizer] () {
top = SC_Top(src, thr, sv, mT)
bot = SC_Bot(src, thr, sv, mB)
lft = SC_Left(src, thr, sh, mL, top, bot)
rgt = SC_Right(src, thr, sh, mR, top, bot)

sw = Float(Max(16, src.Width() - lft - rgt))
ssh = Float(Max(16, src.Height() - top - bot))

content_ar = sw / ssh
target_ar = Float(tw) / Float(th)

rw = (content_ar > target_ar) ? tw : Round(Float(th) * content_ar)
rh = (content_ar > target_ar) ? Round(Float(tw) / content_ar) : th
rw = Max(2, Min(rw - (rw % 2), tw))
rh = Max(2, Min(rh - (rh % 2), th))

resized = Eval(resizer + "(src, rw, rh, " \
+ "src_left=" + String(lft) + ".0" \
+ ", src_top=" + String(top) + ".0" \
+ ", src_width=" + String(sw) \
+ ", src_height=" + String(ssh) + ")")

pl = (tw - rw) / 2
pt = (th - rh) / 2
pr = tw - rw - pl
pb = th - rh - pt

(pl > 0 || pt > 0 || pr > 0 || pb > 0) \
? resized.AddBorders(pl, pt, pr, pb) : resized
})
}


function SmartCropDebug(clip c, int "threshold", \
int "T", int "B", int "L", int "R", \
int "scan_v", int "scan_h") {
thr = Default(threshold, 32)
mT = Default(T, 0)
mB = Default(B, 0)
mL = Default(L, 0)
mR = Default(R, 0)
sv = Default(scan_v, 30)
sh = Default(scan_h, 30)

ScriptClip(c, function [thr, mT, mB, mL, mR, sv, sh] () {
top = SC_Top(last, thr, sv, mT)
bot = SC_Bot(last, thr, sv, mB)
lft = SC_Left(last, thr, sh, mL, top, bot)
rgt = SC_Right(last, thr, sh, mR, top, bot)

Subtitle(last, \
"SmartCrop T=" + String(top) + " B=" + String(bot) \
+ " L=" + String(lft) + " R=" + String(rgt) \
+ "\nContent: " + String(last.Width()-lft-rgt) \
+ "x" + String(last.Height()-top-bot) \
+ " Frame: " + String(current_frame) \
+ "\nThr=" + String(thr) \
+ " +T=" + String(mT) + " +B=" + String(mB) \
+ " +L=" + String(mL) + " +R=" + String(mR) \
+ " ScanV=" + String(sv) + " ScanH=" + String(sh), \
size=20, text_color=$FFFFFF, halo_color=$000000, \
y=8, lsp=5)
})
}

Changelog

v2:

Split scan_depth into scan_v (top/bottom) and scan_h (left/right) for sources with wide sidebars but narrow vertical borders.
Dark scene safeguard: if scan reaches the depth limit without finding content, assumes dark scene (not border) and returns 0 for that side. Set scan depth above your expected maximum border width.


Notes


Designed for film scans with variable borders across reel changes and analog broadcast recordings with wide sidebars.
The key trick: Resize's src_ parameters accept per-frame values inside ScriptClip, while output dimensions stay fixed — AviSynth sees a stable filter graph.
Left/right detection runs only within the content vertical range (after top/bottom are found), so horizontal borders don't contaminate column averages.
If no border is detected on a given side, margin is not applied — no accidental clipping of borderless frames.


Feedback and edge cases welcome.

[StainlessS]

Maybe post in"New Plugs" etc,
I know you know where it is.

[Genji]

That's fantastic!
It's sure to come in handy when encoding old analog recordings!
I have one request, though.
Could you add a mode that centers the image while maintaining the aspect ratio after cropping and resizing it to the target dimensions?
This would mean filling in the missing space with black if the top and bottom or the left and right sides don't quite reach the target size.
I would appreciate it if you could consider this.

[Genji]

This is a demonstration of how it works.

Source: 1440x1080 (displayed at 1920x1080)

A scene from somewhere

SmartCropDebug(scan_depth=220)

Screen display
SmartCrop T=6 B-0 L-204 R=186
Content: 1050x1074 Frame: 13308
Processing
Crop(204,6,-186,0)
AddBorders(12,0,12,0) *Add padding to maintain aspect ratio
Spline36Resize(960x720)

[Genji]

Sorry for the back-to-back posts.
One more point.
I would like to set different values for `scan_depth` in the vertical and horizontal directions.
Since this is for analog broadcasting, I set `scan_depth` to 220 in the horizontal direction; however, doing so causes unintended clipping in the vertical direction during dark scenes.
Please consider this request ,too.

[shurik_pronkin]

=== Reply to StainlessS ===

Good point, will do. Thanks for RT_Stats — the whole thing is built on RT_AverageLuma.


=== Reply to Genji ===

Thanks for testing and for the detailed feedback! Both requests are addressed:

1. Centering with aspect ratio preservation

This already exists — it's the mode="fit" parameter. It crops the detected borders, resizes the content to fit within the target dimensions while preserving AR, and pads the remainder with black (centered). Your example would be:

SmartCrop(src, 960, 720, mode="fit", scan_h=220)

2. Separate scan depth for vertical and horizontal

Good catch. The single scan_depth parameter has been replaced with scan_v (top/bottom) and scan_h (left/right). Now you can scan deep horizontally for wide analog sidebars without eating into dark scenes vertically:

# scan 220px from left/right, only 30px from top/bottom
SmartCropDebug(src, threshold=32, scan_v=30, scan_h=220)

Updated script below.


SmartCrop.avsi — per-frame adaptive border removal via ScriptClip

The problem

Film-based sources (telecine from physical reels) and analog broadcast recordings often have black borders that vary: different thickness on different sides, shifting between reel changes or across the recording. A static Crop() doesn't work — you either leave some borders in or eat into content on other segments.

The idea

AviSynth's Resize functions accept src_left, src_top, src_width, src_height parameters that define a source window within the input frame. The output dimensions remain fixed. This means we can vary the crop region per frame inside ScriptClip while the filter graph sees constant dimensions — no recompilation, single pass, fully transparent to downstream filters.

How it works

For each frame, scan from the edge inward, one row/column at a time. Measure average luma via RT_AverageLuma. If it's at or below the threshold — that's border, keep going. First line above the threshold — stop, that's content. The count of consecutive dark lines from the edge = border width.

Detection order: top/bottom first (full-width rows), then left/right within the detected content vertical range, so horizontal borders don't contaminate column averages.

Per-side margins (T/B/L/R) add extra pixels beyond what was detected — useful for gradient borders from analog sources where the transition isn't a hard edge. Result is always rounded up to even.

If no border is detected on a given side, margin is not applied on that side.

Requirements


AviSynth+ (ScriptClip with closures)
RT_Stats by StainlessS (RT_AverageLuma)


Functions

SmartCropDebug(clip c, int "threshold", int "T", int "B", int "L", int "R",
int "scan_v", int "scan_h")

SmartCrop(clip c, int target_w, int target_h, string "mode", int "threshold",
int "T", int "B", int "L", int "R",
int "scan_v", int "scan_h", string "resizer")

Parameters

threshold avg luma ≤ this = border default 32
T,B,L,R extra pixels added per side default 0
scan_v max pixels to scan from top/bottom default 30
scan_h max pixels to scan from left/right default 30
resizer resize kernel default "Spline36Resize"
mode "stretch" — fill target exactly default "stretch"
"fit" — preserve AR, center, pad black

Examples

Import("SmartCrop.avsi")
src = LWLibavVideoSource("film.mkv")

# 1. Tune — watch overlay while scrubbing through reel transitions
SmartCropDebug(src, threshold=40, R=4)

# 2. Apply — crop borders, stretch to target
SmartCrop(src, 720, 576, threshold=40, R=4)

# 3. Fit mode — preserve AR, center content, pad with black
SmartCrop(src, 1920, 1080, mode="fit", threshold=40, L=2, R=4)

# 4. Analog broadcast — wide sidebars, narrow top/bottom
SmartCrop(src, 960, 720, mode="fit", scan_v=30, scan_h=220)

Source

###############################################################################
# SmartCrop.avsi — single-pass adaptive crop via ScriptClip
#
# Scan from edge inward. Dark (avg luma ≤ threshold) → border.
# First bright line → stop. Add per-side margin (T/B/L/R).
# Round up to even. Resize src_ → output always constant.
#
# Dependencies: RT_Stats (RT_AverageLuma)
###############################################################################


function SC_Top(clip c, int thr, int sd, int mg) {
w = c.Width()
n = current_frame
b = 0
s = false
for (y = 0, sd - 1, 1) {
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=0, y=y, w=w, h=1)
s = s ? true : (a > thr)
b = s ? b : y + 1
}
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}

function SC_Bot(clip c, int thr, int sd, int mg) {
w = c.Width()
h = c.Height()
n = current_frame
b = 0
s = false
for (dy = 0, sd - 1, 1) {
y = h - 1 - dy
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=0, y=y, w=w, h=1)
s = s ? true : (a > thr)
b = s ? b : dy + 1
}
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}

function SC_Left(clip c, int thr, int sd, int mg, int top, int bot) {
h = c.Height()
cy = top
ch = Max(1, h - top - bot)
n = current_frame
b = 0
s = false
for (x = 0, sd - 1, 1) {
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=x, y=cy, w=1, h=ch)
s = s ? true : (a > thr)
b = s ? b : x + 1
}
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}

function SC_Right(clip c, int thr, int sd, int mg, int top, int bot) {
w = c.Width()
h = c.Height()
cy = top
ch = Max(1, h - top - bot)
n = current_frame
b = 0
s = false
for (dx = 0, sd - 1, 1) {
x = w - 1 - dx
a = s ? 255.0 : RT_AverageLuma(c, n=n, x=x, y=cy, w=1, h=ch)
s = s ? true : (a > thr)
b = s ? b : dx + 1
}
b = (b > 0) ? b + mg : 0
return b + (b % 2)
}


function SmartCrop(clip c, int target_w, int target_h, \
string "mode", int "threshold", \
int "T", int "B", int "L", int "R", \
int "scan_v", int "scan_h", string "resizer") {
mode = Default(mode, "stretch")
thr = Default(threshold, 32)
mT = Default(T, 0)
mB = Default(B, 0)
mL = Default(L, 0)
mR = Default(R, 0)
sv = Default(scan_v, 30)
sh = Default(scan_h, 30)
resizer = Default(resizer, "Spline36Resize")

Assert(target_w % 2 == 0 && target_h % 2 == 0, "SmartCrop: target must be mod-2")

base = Eval(resizer + "(c, target_w, target_h)")

return (mode == "fit") \
? SC_Fit(base, c, target_w, target_h, thr, mT, mB, mL, mR, sv, sh, resizer) \
: SC_Stretch(base, c, target_w, target_h, thr, mT, mB, mL, mR, sv, sh, resizer)
}

function SC_Stretch(clip base, clip src, int tw, int th, \
int thr, int mT, int mB, int mL, int mR, \
int sv, int sh, string resizer) {
ScriptClip(base, function [src, tw, th, thr, mT, mB, mL, mR, sv, sh, resizer] () {
top = SC_Top(src, thr, sv, mT)
bot = SC_Bot(src, thr, sv, mB)
lft = SC_Left(src, thr, sh, mL, top, bot)
rgt = SC_Right(src, thr, sh, mR, top, bot)
Eval(resizer + "(src, tw, th, " \
+ "src_left=" + String(lft) + ".0" \
+ ", src_top=" + String(top) + ".0" \
+ ", src_width=" + String(Max(16, src.Width()-lft-rgt)) + ".0" \
+ ", src_height=" + String(Max(16, src.Height()-top-bot)) + ".0)")
})
}

function SC_Fit(clip base, clip src, int tw, int th, \
int thr, int mT, int mB, int mL, int mR, \
int sv, int sh, string resizer) {
ScriptClip(base, function [src, tw, th, thr, mT, mB, mL, mR, sv, sh, resizer] () {
top = SC_Top(src, thr, sv, mT)
bot = SC_Bot(src, thr, sv, mB)
lft = SC_Left(src, thr, sh, mL, top, bot)
rgt = SC_Right(src, thr, sh, mR, top, bot)

sw = Float(Max(16, src.Width() - lft - rgt))
ssh = Float(Max(16, src.Height() - top - bot))

content_ar = sw / ssh
target_ar = Float(tw) / Float(th)

rw = (content_ar > target_ar) ? tw : Round(Float(th) * content_ar)
rh = (content_ar > target_ar) ? Round(Float(tw) / content_ar) : th
rw = Max(2, Min(rw - (rw % 2), tw))
rh = Max(2, Min(rh - (rh % 2), th))

resized = Eval(resizer + "(src, rw, rh, " \
+ "src_left=" + String(lft) + ".0" \
+ ", src_top=" + String(top) + ".0" \
+ ", src_width=" + String(sw) \
+ ", src_height=" + String(ssh) + ")")

pl = (tw - rw) / 2
pt = (th - rh) / 2
pr = tw - rw - pl
pb = th - rh - pt

(pl > 0 || pt > 0 || pr > 0 || pb > 0) \
? resized.AddBorders(pl, pt, pr, pb) : resized
})
}


function SmartCropDebug(clip c, int "threshold", \
int "T", int "B", int "L", int "R", \
int "scan_v", int "scan_h") {
thr = Default(threshold, 32)
mT = Default(T, 0)
mB = Default(B, 0)
mL = Default(L, 0)
mR = Default(R, 0)
sv = Default(scan_v, 30)
sh = Default(scan_h, 30)

ScriptClip(c, function [thr, mT, mB, mL, mR, sv, sh] () {
top = SC_Top(last, thr, sv, mT)
bot = SC_Bot(last, thr, sv, mB)
lft = SC_Left(last, thr, sh, mL, top, bot)
rgt = SC_Right(last, thr, sh, mR, top, bot)

Subtitle(last, \
"SmartCrop T=" + String(top) + " B=" + String(bot) \
+ " L=" + String(lft) + " R=" + String(rgt) \
+ "\nContent: " + String(last.Width()-lft-rgt) \
+ "x" + String(last.Height()-top-bot) \
+ " Frame: " + String(current_frame) \
+ "\nThr=" + String(thr) \
+ " +T=" + String(mT) + " +B=" + String(mB) \
+ " +L=" + String(mL) + " +R=" + String(mR) \
+ " ScanV=" + String(sv) + " ScanH=" + String(sh), \
size=20, text_color=$FFFFFF, halo_color=$000000, \
y=8, lsp=5)
})
}

[Genji]

> 1. Centering while preserving the aspect ratio

I believe `mode=“fit”` behaves as follows:
Crop(204,6,-186,0)
AddBorders(194,0,196,0)
Spline36Resize(960,720)
What I want is to preserve the aspect ratio of the image after cropping.
Crop(204,6,-186,0)
AddBorders(12,0,12,0)
Spline36Resize(960,720)

> 2. Separate scan depth for vertical and horizontal

Thank you for providing `scan_v` and `scan_h`!

[StainlessS]

Post #1 of #2

Hi shurik,
v1.43 RT_Stats had some AVS scripts that I removed in v2.0Beta,
https://www.mediafire.com/file/zsev7a1mkb3p63o/RT_Stats_AVS_From_V143.7z/file

In above 7z, "QueryBorderCrop.avs" might be of interest (then again, maybe it aint).
RT_QueryBorderCrop() is based on QueryBorderCrop.avs script, and RoboCrop plugin is based on both.

Here is doc from the script.


Function QueryBorderCrop(clip c,int "Samples",Float "Thresh",bool "Laced",int "XMod",int "YMod",int "WMod",int "HMod", \
bool "Relative", String "Prefix",int "RLBT",bool "DEBUG",float "Ignore",int "Matrix",Float "ScanPerc",int "Baffle", \
bool "ScaleAutoThreshRGB",bool "ScaleAutoThreshYUV",Float "ATM",int "Start",int "End") {
VERS="QueryBorderCrop v1.31 - 01 Jan 2014" # By StainlessS, Requires GScript & RT_Stats v1.31 Plugins.
/*

Prescan function to get coords for eg cropping black borders by sampling Samples frames. Planar, YUY2, RGB.
Borders are detected by sampling at Samples frames, at scanlines (h/v) using AverageLuma (RGB is converted to Luma Y at either TV or
PC levels, See Matrix). This sampling is done on all 4 sides of the frame.
If a scanline Average luma is below or equal to Thresh, then is considered possible black border, above considered possible image,
if Baffle [default=4] adjacent scanlines above Thresh, then it IS image.

Simultaneously returns 4 sets of strings holding crop coords, here is one set: "QBCropX=8 QBCropY=8 QBCropW=640 QBCropH=480".
String sets are Chr(10) separated, the 'exact found' coords set as above, not rounded at all and possibly throwing an error if
you try to crop, although could use in eg resize.
Second set using eg "QBCropXL=8", which is CropLess("L" trailer), ie when rounding for Xmod,WMod etc may leave some black border.
Third set using eg "QBCropXM=8", which is CropMore, ie when rounding for Xmod,WMod etc may crop some image.
Forth set using eg "QBCropXP=8", which is CropPlus, moves border positions an extra 2 pixels inwards on each edge and then as CropMore.
The non-exact coords try to center the coords when cropping for larger WMod/HMod's so as to evenly crop from eg both left and right instead of
just one side. Also returned in the return string is the used Threshold, perhaps set by AutoThresh, as eg "QBCropThresh=32.0"
You can use eg Eval(QueryBorderCrop()) to set values into QBCropX, QBCropY,QBCropW,QBCropH etc variables.

Function QueryBorderCrop(clip c,int "Samples"=32,Float "Thresh"=-32,bool "Laced"=true, \
int "XMod",int "YMod",int "WMod"=XMod,int "HMod"=YMod, \
bool "Relative"=false, String "Prefix"="QBCrop",int "RLBT"=15,bool "DEBUG"=false,float "Ignore"=0.2, \
int "Matrix" = (c.Width > 1100 || c.Height > 600 ? 3 : 2), \
Float "ScanPerc"=49.0,int "Baffle"=4, bool "ScaleAutoThreshRGB"=true,bool "ScaleAutoThreshYUV"=false,Float "ATM"=4.0, \
int "Start"=Undefined,int "End"=Undefined)

Args:-

Samples=32, Number of frames to sample from source clip c.
As an observation, most clips have good border recogition within the 1st 2 sampled frames using the default -32.0 AUTOTHRESH
although have noticed some dark clips that required ~8 samples (maybe more) for full recognition @ default Thresh = -32.0
We use a default Samples=32, because we are intrinsically paranoid.
If number of frames between frame at 5% of framecount and frame at 90% of framecount is greater than 250 and is also greater
than Samples, will ignore the first 5% and last 10% of frames when both auto-thresholding and crop sampling to try to negate
effects of artificial black in titles and end credits. Can override the Auto Credits skipping by setting Start and/or End frame
of range to sample.
Samples = 0, will be converted to Samples = FrameCount - 1, ie auto credits skipping disabled and ALL FRAMES SAMPLED,
of use for very short scenes, not for general full movie clips.

Thresh: Default= -32.0 (==DEFAULT AUTOTHRESH, any -ve Thresh is AUTOTHRESH where Thresh adjustments are automatic).

Thresh > 0: (Explicit Threshold)
A user supplied +ve Thresh should be at correct TV/PC levels for the the clip being worked on ie 16->235 for TV levels and
0->255 for PC Levels (RGB, as appropriate for matrix being used).

Thresh <= 0: (AUTOTHRESH)
When Thresh <= 0, the clip will be sampled over Samples frames to find the minimum YPlaneMin (using Matrix if RGB) which
we will call MINLUMA and an Explicit Threshold calculated by adding MINLUMA to abs(Thresh), after that it is processed
as for Thresh > 0: (Explicit Threshold) as noted above, but, before adding MINLUMA, some AUTOTHRESH massaging and scaling occurs.

Here AUTOTHRESH Thresh massaging and scaling occurs in sequence:-
1 ) if (Thresh == DEFAULT AUTOTHRESH && ATM < 32.0) (DEFAULT AUTOTHRESH = exactly -32.0, defaulted OR user supplied):
Let sampstart and sampend, be starting and ending frames numbers after any Auto Credits skipping and/or user set Start or End.
Let Samples be limited to sampend-sampstart+1.
Let SampRange (Sample Range) = SampEnd-SampStart+1.
samples_massage =(Samples>=16) ? 1.0 : (Samples-1) * (1.0/(16-1)) # No massaging if Samples >= 16
range_massage =(SampRange >= (250*16)) ? 1.0 : (SampRange-1) * (1.0/((250*16)-1)) # No massaging if SampRange >= 4000
Both samples_massage and range_massage will be in range 0.0 to 1.0.
Thresh = -(((samples_massage * range_massage) * (32.0 - ATM)) + ATM)
This adjustment to Auto Thresh is to reduce the possibility of overcropping on eg a dark low 'Samples' clip, or where
source SampRange (ie temporal frame set) is too small to take a reliable sample from.
Resulting massaged Thresh will range between -ATM and -32.0.
Although massaging is intended to reduce overcropping, it could result in not cropping enough border (less disastrous),
its a bit of a balancing act really. See also ATM.
2 ) If RGB AND PC matrix(default) AND ScaleAutoThreshRGB==True(default) then
Thresh= Thresh*(255.0/(235-16))
3 ) If YUV AND ScaleAutoThreshYUV==True(default=false) then
Thresh= Thresh*(255.0/(235-16))

Steps 2) and 3) above, by default treat a -ve AUTOTHRESH as being @ TV Levels and so Scale RGB Thresh to PC levels but not YUV.
If you want to supply a PC levels AUTOTHRESH Thresh for RGB, then simply set ScaleAutoThreshRGB=false to inhibit scaling.
Note, if a TV levels Matrix is supplied for RGB, then scaling will always be inhibited.
If your clip is YUV at PC levels and you want to use eg DEFAULT AUTOTHRESH (-32.0, which is considered to be @ TV levels),
then set ScaleAutoThreshYUV=True to enable Thresh scaling.
If your clip is YUV at PC levels and you want to use a PC levels AUTOTHRESH (-ve) then leave ScaleAutoThreshYUV at default false
which does not scale Thresh.
After any scaling, MINLUMA is then added to abs(Thresh) and processed as for +ve Explicit Threshold as noted above.
NOTE, Above QueryBordeCrop step 1) 'massages' DEFAULT AUTOTHRESH (exactly -32.0) if low samples count or if short clip. Reason being to
avoid overcropping when insufficient data available for reliable cropping. It is considered better to not crop enough or at all,
than to overcrop. You can override by simply setting an explicit threshold (+ve) of eg 40.0, or setting a NON-DEFAULT auto thresh
(-ve) eg -16.0 or -32.1, where YPlaneMin is established for the sampled frames and then abs(thresh) is added to that value which
is then used as an explicit thresh.

[StainlessS]

Post #2 of #2




Laced:, Default=true, true for Interlaced. (Modifies defaults for YMod,HMod, explicit YMod/HMod will override).
QueryBordeCrop automatically deduces colorspace cropping restrictions and sets internal XMod and YMod,
eg XMod and YMod would both be set to 2 for YV12, for YUY2 Xmod=2, YMod=1, etc.
If Laced==true, then internal YMod is doubled so as not to destroy chroma.
Below WMod and HMod are both defaulted to internal XMod and YMod respectively after the Laced hint is applied to YMod.

XMod:, Default=The natural cropping XMod for clip colorsapace, eg YV411=4, YV12=2, YUY2=2, RGB=1
YMod:, Default=The natural cropping YMod for clip colorsapace, eg YV411=1, YV12=2, YUY2=1, RGB=1: BUT, Doubled if laced=true.
NOTE, XMod, YMod, If overridden must all be multiples of the colorspace natural cropping XMod,YMod else throws error.
NOTE, As We now use natural XMod,Ymod, might be best to never alter defaults for XMod,YMod, suggest only change
Laced, WMod and HMod as required, XMod and YMod left in-situ so as not to break scripts.

WMod:, Default=The natural chroma cropping restriction of the colorspace concerned, eg 2 for YV12.
HMod:, Default=The natural chroma cropping restriction of the colorspace concerned, BUT, doubled if laced=true.
The above WMod,HMod sets rounding for legal cropping coords for colorspace concerned.
WMod MUST be a multiple of internal XMod as described under Laced above, or it will throw an error.
HMod MUST be a multiple of internal YMod as described under Laced above, or it will throw an error. If eg colorspace is
YV12 then YMod would be set to 2, and if Laced, then YMod would be doubled to 4, so HMod MUST be a multiple of 4.
Some encoders may require an WMod/HMod of eg 8 or 16, and if set thus, would crop off more or less depending upon
which CropMode is used, if later resizing will be done, then encoder requirements can be satisfied during the resize.
NOTE, Some players and VirtualDubMod (Helix YV12 decoder) dont like WMOD less than 4 (Vdub latest, OK, internal decoder).
If eg VDMod show blank frame, OR eg player halts saying eg "No combination of filters cound be found to render frame"
then set WMod to a multiple of 4. We do not do this by default as Current VDub and some player/encoders may work just fine.
If you dont care about possibility of losing a couple of pixels then always supply WMod=4 to avoid display problems.

Relative:=false, False returns Width and Height, true returns Width/Height relative eg QBCropW=-6 QBCropH=-4.

Prefix:="QBCrop", string for returned variable names, only use valid variable name characters eg NO SPACES. Default returns eg "QBCropX".

RLBT:=15=All Borders, Bitflags of edges to crop, 15 ($0F) crops all four. Each letter in the name 'RLBT' represents an edge and bit position
starting with 'R' in bit 3 representing the value 8 (2^3=8). 'L' = bit 2=4 (2^2=4), 'B' = bit 1=2 (2^1=2), 'T' = bit 0=1 (2^0=1).
To calculate the RLBT bitflags, for 'R'(Right) add 8, for 'L'(Left) add 4, for 'B'(Bottom) add 2, and for 'T'(Top) add 1.
Add all of the bit flags together 8+4+2+1 (=15) crops all four edges, 8+4 crops only Right & Left, and 2+1 crops only Bottom & Top.

DEBUG:=False=No Debug. Set True for debugging info, need DebugView: http://technet.microsoft.com/en-gb/sysinternals/bb545027
The debug info output shows eg range limiting of Samples and sample info and resultant auto set Thresh. You are encouraged
to use debug to see the eg the auto Thresh massaging in action, it may help to understand usage of the function. MS DebugView
can also be used to view output from other plugins and programs that can also be useful.

Ignore:=0.2, Percentage of darkest pixels to ignore during AutoThresh scan to detect minimum luma pixel of all sampled frames.
(ignores extreme values ie noise, as for Threshold arg in YPlaneMin).

Matrix:, RGB ONLY. For conversion of RGB to YUV-Y, 0 = Rec601, 1 = Rec709, 2 = PC601, 3 = PC709
Default for RGB is:- If clip Width > 1100 OR clip Height > 600 Then 3(PC709) , else 2(PC601) : YUV not used
The defaults are for PC601 & PC709 range 0-255.
So as to not require different AutoThresh for RGB, if clip c is RGB AND matrix is PC Levels AND Thresh < 0.0 and ScaleAutoThreshRGB=true,
then Thresh will be scaled to RGB full range ie Thresh = Thresh * (255.0/(235.0-16.0)) ONLY when AutoThresh (ie Thresh < 0.0,
YPlaneMin relative).
When +ve Thresh is explicitly supplied (Thresh > 0.0) it is not scaled and assumed to be already correct range TV or PC levels.

ScanPerc:=49.0=Scan only 49 percent width and height (left, right, top bot) when detecting borders, special use only. Range 1.0 -> 99.0.
ScanPerc implemented because of this post by Jmac698: http://forum.doom9.org/showthread.php?p=1604004#post1604004

Baffle:=4, Number of scanlines (h/v) that must break threshold to be considered image (can avoid noise around frame edges).
Does not usually need changing but might be of use to avoid some kind of eg teletext data at top of frame in broadcast video.

ScaleAutoThreshRGB: bool default True. If true and RGB and Matrix at PC levels, and Thresh -ve (autothresh) then thresh will (after any
auto Thresh massaging) be scaled to PC levels. By default, -ve Auto Thresh is considered to be at TV levels, and so will be scaled to
to PC levels to match the matrix setting. If ScaleAutoThreshRGB is False Then autothresh is considered to be at PC levels and not scaled.

ScaleAutoThreshYUV: bool default False. If true and YUV and Thresh -ve (autothresh) then Thresh will (after any auto Thresh massaging),
be scaled to PC levels. By default, -ve Auto Thresh is considered to be at TV levels, this allows you to change that assumption when
source clips are at PC levels. If supplying PC levels clip and PC levels autothresh then leave this setting at false (no scaling).
The above seemingly awkward settings are due to having to deal with YUV @ TV levels and RGB @ PC levels with the possibility of
YUV @ PC levels.

ATM: Float default 4.0 (0.5 -> 32.0). Silently limited to valid range.
ATM allows user to set the DEFAULT AUTOTHRESH massaging minimum. When eg samples = 1, then auto thresholding is of course quite
unreliable and so auto Thresh would be 'massaged' to -ve(ATM), other values of Samples below 16 will likewise have auto thresh
massaged but to a lesser degree, linearly between -ve(ATM) for Samples == 1 and -ve(32.0) for Samples == 16.
Auto Thresh massaging also takes into account the frame range of samples (first sampled frame to last sampled frame inclusive)
and this is mixed together with any samples massaging then applied to auto Thresh.
Previous (Fixed) default for ATM was 0.5, for maximum safety so that a single Samples scan would NOT overcrop. The new ATM arg default
of 4.0 is a less paranoid safety setting which should in most cases work well but with a very short clip or eg single image then
user might be better off giving the minimum ATM of 0.5. An ATM of 32.0 will switch OFF default auto thresh massaging.
So long as sample range is about 4000+ frames and Samples at least 16(default 32), then there will be no auto thresh massaging and
current default is unlikely to need changed, with very short clips or reduced Samples count, then you might want to reduce ATM, for
maximum paranoia, set ATM=0.5 to 1.0, especially in an app that processes single images. See Thresh.

Start: Default Undefined. Start frame of scan area. Overrides Auto Intro credits skipping.
For Auto Intro and End Credits Skipping to be set to 5% (of FrameCount for Intro Skipping) and 90% (for End Skipping), the number of
frames between them must be greater or equal to 250 frames, and MUST also be greater than Samples, otherwise Auto skipping ignored and
the Start and End frame numbers are set to 0 and Framecount - 1. If a user supplies eg a Start frame number ONLY, then End Skipping
has to comply with the same conditions, range between End Skip frame and user supplied Start has to be at least 250 frames and greater
than Samples, otherwise End frame set to FrameCount - 1.
After either via Auto Credits skipping, or user supplied Start/End, or defaulted to 0 & FrameCount -1, we have a sample scan range.

End: Default Undefined. End frame for scan area. Overrides Auto End credits skipping. 0 (or less) will be converted to Framecount - 1.
See previous Start setting.

Cropping will likely fail on END CREDITS ONLY clips, where on black background, and will probably crop off the sides that are no
different in average luma to any letterbox borders, if you cannot see the borders, then neither can QueryBordeCrop(), even setting the
auto Thresh to eg -1.0 or 0.0 is quite likely to fail. (See RLBT edge bitflags).

If cropping too much border, then increase Samples or reduce Thresh or lower ATM if short clip.
If not cropping enough border then border is being seen as image, increase Thresh (for -ve Thresh , make more -ve).

To speed up, you may want to leave Auto-Thresh alone and reduce samples, but there is danger that it will not detect all borders (overcrop).
Suggest you do not go below Samples=16, although early default for samples was 12, we use a paranoid setting of 32 by default.
However, QueryBordeCrop is quite sprightly and you will probably not need to reduce Samples, but if you are interested in how long it takes to
do it's auto thresh scanning, the time taken is output via the debug arg to DebugView. Clips that have no letter
boxing are quickest dealt with and those with largest borders take most time as it has to scan the borders of all Samples frames, looking for
image.
NOTE, The plugin AutoCrop() uses a Thresh of 40.0 and samples == 5 by default (No AutoThresh), but the base logic is not too dissimilar.

Final NOTE, the args to especially note are:- WMod, you might want to supply WMod=4 if using VDubMod (some players might try to download codec),
And Laced=False if you always process progressive or deinterlaced, And ATM set between 0.5 and 4.0 if very short clips/single-image.

*/


I have not really looked at your script, but thought that you might like to see the removed script.
(perhaps they serve two different purposes [EDIT: Yes they do serve different purposes], mine gets outermost borders for entire clip.)

[StainlessS]

Here is a post with a BorderTorture() function, I think IanB posted original idea for a border torture clip for testing, could not find it.
https://forum.doom9.org/showthread.php?p=1622434#post1622434


Function BorderTorture(clip c,String "fn",int "sz") {
c
myName="BorderTorture:"
fn=Default(fn,"BorderTorture.txt") RT_FileDelete(fn)
sz=Default(sz,100)
WW=Width HH=Height Frames=FrameCount
nTrims=(Frames+sz-1)/sz c2=0 oldcx=0 oldcy=0 oldcw=0 oldch=0
wr_s="# BorderTorture: ("+String(WW)+"x"+String(HH)+") Creating output clip "+String(WW+8)+"x"+String(HH+8)+" Scene Start End X Y W H"
RT_TxtWriteFile(wr_s,fn,append=true) RT_Debug(myName,wr_s,false)
GSCript("""
For(n=0,nTrims-1) {
b=true
while(b || (cx==oldcx && cy==oldcy && cw==oldcw && ch==oldch)) { # Make sure some change at every trim
cx=Rand(4+1)*2 cy=Rand(4+1)*2 cw=Rand(4+1)*2 ch=Rand(4+1)*2
b=false
}
e = ((n+1)*sz-1>=Frames) ? Frames-1 : (n+1)*sz-1
wr_s = \
RT_StrPad(String(n+1)+")",5) + " " + \
RT_StrPad(String(n*sz),7) + " " + \
RT_StrPad(String(e),7) + " " + \
RT_StrPad(String(cx+4),4) + " " + \
RT_StrPad(String(cy+4),4) + " " + \
RT_StrPad(String(WW-cx-cw)+"("+String(-(cw+4))+")",11) + " " + \
RT_StrPad(String(HH-cy-ch)+"("+String(-(ch+4))+")",11)
RT_Debug(myName,wr_s,false)
RT_TxtWriteFile(wr_s,fn,append=true)
t=Trim(n*sz,-sz).Crop(cx,cy,-cw,-ch).AddBorders(cx+4,cy+4,cw+4,ch+4)
c2 = (c2.isClip) ? c2++t : t
old_cx=cx oldcy=cy oldcw=cw oldch=ch
}
""")
return c2
}


EDIT:
I think IanB posted original idea for a border torture clip for testing, could not find it
Actually, it was a Pitch torture tests for plugin devs.

The YV12 path of ConvertToYUY2() outputs mod 32 pitch so it can do 16 pixels per loop. Line 132 of convert/convert_yuy2.cpp

And must not assume any relationship between pitch of the luma planes and chroma planes for the planar formats.

You must use the GetPitch() method on each and every PVideoFrame you get.

Pitch torture test for filters....
A=SelectEvery(4, 0)
B=SelectEvery(4, 1).AddBorders(0,0,8,0).Crop(0,0,-8,0)
C=SelectEvery(4, 2).AddBorders(0,0,16,0).Crop(0,0,-16,0)
D=SelectEvery(4, 3).AddBorders(2,0,22,0).Crop(2,0,-22,0)
Interleave(A,B,C,D)
FilterUnderPitchStressTest(...)
...

[shurik_pronkin]

SmartCrop v1.0 — adaptive border removal for film sources

Stable release. Completely rewritten from scratch.

What it does:
Per-frame detection of black borders from the film gate (DVD, telecine transfers). Borders vary between reels but stay constant within each reel. SmartCrop detects them automatically and resizes to a fixed output via Resize src_ parameters — AviSynth sees a stable filter graph.

Algorithm:
The clip is divided into blocks of N frames (default 400). For each block:

Find the brightest frame (most reliable for detection)
Scan inward from each edge — rows/columns with avg luma ≤ threshold are border
Baffle filter: require N consecutive bright lines to confirm content (avoids noise)
Snap result to grid (default 4px) — eliminates sub-pixel jitter
Compare with previous block — within tolerance → inherit (same reel), exceeds → reel change
Dark blocks propagate from nearest bright block

All frames in the same block get identical values. Fully stateless — safe for AvsPmod seeking.

Usage:

# Debug — shows detection overlay
SmartCrop(debug=true, threshold=25, snap=4, baffle=4, block_size=400, tolerance=4, min_luma=40.0)

# Production — crop + resize to target
SmartCrop(720, 576, threshold=25, snap=4, baffle=4, block_size=400, tolerance=4, min_luma=40.0)

# With margins
SmartCrop(720, 576, threshold=25, R=4, T=2)

# Fit mode — preserve aspect ratio with padding
SmartCrop(720, 576, mode="fit", threshold=25)


Parameters:

target_w, target_h Output size (mod-2). Omit for debug mode.
mode "stretch" (default) or "fit" (preserve AR with padding)
threshold =32 Max avg luma for border row/column
T, B, L, R =0 Extra margin pixels per side
scan_v =30 Max scan depth vertical
scan_h =30 Max scan depth horizontal
snap =4 Round to nearest multiple
baffle =4 Consecutive bright lines to confirm content
block_size =400 Frames per analysis block
tolerance =4 Max difference to inherit previous block
min_luma =40.0 Min brightness to trust detection (dark → propagate)
resizer ="Spline36Resize"
debug =false


Debug overlay shows:

Raw T=9 B=10 L=17 R=0 <- raw detection current frame
Result T=8 B=12 L=16 R=0 INHERITED <- final values + status
Block=2 [800..1199] DetF=965 DetL=98.1 <- block info
Frame=894 Luma=72.3

Status: DETECTED (new reel), INHERITED (same reel), PROPAGATED (dark block).

Requirements:

AviSynth+ x64 (v3.7.3+)
Planar input (YV12, YV24, etc.)
Works with Prefetch()


Download:
GitHub: https://github.com/schpuppa-art/SmartCrop-avisynth-

Build from source:
x64 Native Tools Command Prompt:
cl /LD /EHsc /O2 SmartCrop.cpp /link /OUT:SmartCrop.dll

Thanks to StainlessS for the baffle concept from QueryBorderCrop, and to Genji for feedback on fit mode and scan_v/scan_h.

[tormento]

SmartCrop v1.0 — adaptive border removal for film sources
Perhaps out of the purpose but it would be nice to have real adaptive crop, where some slightly unstable video, such as old scanned films, could be stabilized before cropping, as the process usually moves the picture around the frame.

[Genji]

Congratulations on the release of v1.0!
I can't open the GitHub link—could you please check it?

I realize my explanation was incomplete.
The source image is 1440x1080 (3:2), but it's displayed at 1920x1080 (16:9).
When I process this with SmartCrop using mode="fit", it seems to be processing it based on the 1440x1080 (3:2) aspect ratio.
If it could return the result processed at 1920x1080 (16:9), I believe that would produce the output I was expecting.

[Selur]

the link should be: https://github.com/schpuppa-art/SmartCrop-avisynth-

[real.finder]

in case you didnt see the github https://github.com/schpuppa-art/SmartCrop-avisynth-/issues/1

I think it worth adding 'mode "mask"' to output as a mask for detected borders so we can use that mask in some plugin/filter that will fill those borders instead of using resizers internally to fill them

and it also worth adding another option (not as a mode but as new parameter) for YUVA/RGBA(P) to invert the alpha channel in the detected borders and that will help in cases like the borders is not clean so with the help of the mask that come from the alpha (A) of YUVA/RGBA we can clean them in other filters

[real.finder]

Perhaps out of the purpose but it would be nice to have real adaptive crop, where some slightly unstable video, such as old scanned films, could be stabilized before cropping, as the process usually moves the picture around the frame.

you mean filters like stab() that add a borders? I have some idea https://github.com/pinterf/mvtools/issues/67

if this feature added then I will made new clean version of my stab3()

[DTL]

"resized.AddBorders(pl, pt, pr, pb) "

To save from halo/ringing at the display scaler new AVS+ supports filtering of the new created transients (with either AddBorders or borders filling with LetterBox) - https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/corefilters/addborders.html . It is good to use in both scripts and make some implementation in the compiled filter.

[real.finder]

in case you didnt see the github https://github.com/schpuppa-art/SmartCrop-avisynth-/issues/1

I think it worth adding 'mode "mask"' to output as a mask for detected borders so we can use that mask in some plugin/filter that will fill those borders instead of using resizers internally to fill them

and it also worth adding another option (not as a mode but as new parameter) for YUVA/RGBA(P) to invert the alpha channel in the detected borders and that will help in cases like the borders is not clean so with the help of the mask that come from the alpha (A) of YUVA/RGBA we can clean them in other filters

I did it myself https://github.com/realfinder/AVS-Stuff/raw/refs/heads/master/for%20AVSPlus/AutoBorderFiller.avsi

so far so good https://forum.doom9.org/showpost.php?p=2030585&postcount=909