Auto cropping script

[alfrednorman]

Hi all!

I have a source where the main frame bounces around from left to right, but the frame itself is consistently the same width. This is the code I landed on that seems to work pretty well, I was wondering if anyone sees any issues with it. I'm very new to everything so it'd be nice to learn. Thanks!

import vapoursynth as vs
import functools

core = vs.core

clip = core.dgdecodenv.DGSource(source)
clip = core.wwxd.WWXD(clip)
template = core.std.BlankClip(clip, width=708, height=480)

class Selector:
def __init__(self):
self.avg = 0
self.maxavg = 0
self.maxavg_l = 0
self.total_avg = 0
self.total_frames = 1

def __call__(self, n, f, clip):
if n == 0 or f.props['Scenechange'] == 1:
self.avg = 0
self.maxavg = 0
self.maxavg_l = 0
self.total_avg = 0
self.total_frames = 1
for l in range(0, 12, 2):
cropped = core.std.CropAbs(clip, width=708, height=480, left=l).std.PlaneStats()
self.avg = cropped.get_frame(n).props['PlaneStatsAverage']
self.total_avg = self.avg
for frame in range(n + 1, n + 40):
while f.props['Scenechange'] == 0:
self.avg = cropped.get_frame(frame).props['PlaneStatsAverage']
self.total_avg += self.avg
self.total_frames += 1
avg_avg = self.total_avg / self.total_frames
if avg_avg > self.maxavg:
self.maxavg = avg_avg
self.maxavg_l = l
return core.std.CropAbs(clip, width=708, height=480, left=self.maxavg_l)
else:
return core.std.CropAbs(clip, width=708, height=480, left=self.maxavg_l)

selector = Selector()
clip = core.std.FrameEval(template, functools.partial(selector, clip=clip), prop_src=clip, clip_src=clip)
clip.set_output()

[_Al_]

just tried to figure out what it does, and:
while f.props['Scenechange'] == 0:
self.avg = cropped.get_frame(frame).props['PlaneStatsAverage']
self.total_avg += self.avg
self.total_frames += 1
f is not changing in that block, so it should run technically forever, if scene change is zero

Have you tried vapoursynth autocrop? Does it not work on frame bases, checking every frame?
https://github.com/Irrational-Encoding-Wizardry/vapoursynth-autocrop
I tested it recently and it left some 2pixel strips on sides, or maybe I should play with its color and color_second arguments. But if that is consistent (that bug), it could be applied and then just checking result per frame, that might be easier somehow.

[alfrednorman]

That's a good point! Using multiple clips as arguments got me confused, I guess I should use
while clip.get_frame(frame).props['Scenechange'] == 0

As I recall, Autocrop doesn't incorporate scene changes so it has the potential to bounce around. I think it's also based around trying to detect the black bars around the frame not unlike Avisynth's Robocrop. I just was thinking it made more sense that if you know how wide the frame is and it's consistent, the crop with the highest PlaneStatsAverage (averaged across 40 frames) is necessarily the one without black bars. Please let me know if I made a mistake!! Thank you!!!

[alfrednorman]

Ugh that's wrong, but that's what I'm going for

[_Al_]

scene detect is in those miscellaneous filters in MiscFilters.dll , not sure if it is distributed with vapoursynth or not, if not, could be downloaded here (https://github.com/vapoursynth/vs-miscfilters-obsolete/releases/tag/R2)
it creates _SceneChangeNext and _SceneChangePrev properties instead of _SceneChange

clip = core.misc.SCDetect(clip)

[alfrednorman]

This is what I ended up going with. Since the actual content part of the frame is consistently 708 px wide, every scene change I do AverageFrames and find the crop 708 px wide that has the highest PlaneStatsAverage. I was having issues with certain scenes that were especially dark, so I made the exception where if the plane average was below a certain threshold I would use FrameBlender so I could average more frames to go off of. So generally it ends up processing pretty fast, I haven't had issues yet. I tried SCDetect (thanks _Al_) but I was getting too fussy setting a threshold, I ended up using the MVTools scene detection so I didn't have to think about it.

import vapoursynth as vs
core = vs.core
import functools

clip = core.dgdecodenv.DGSource(source).std.PlaneStats()
super = core.mv.Super(clip)
vectors = core.mv.Analyse(super)
clip = core.mv.SCDetection(clip, vectors)
template = core.std.BlankClip(clip,width=708)

def selector(n,f,clip):
global maxavg
global maxavg_l
global weights
if n == 0 or f.props['_SceneChangePrev'] == 1:
maxavg = 0
maxavg_l = 0
weights = []
if f.props['PlaneStatsAverage'] >= 0.2:
weights = [1] * 31
for l in range(0,13,2):
avg = core.std.CropAbs(core.std.AverageFrames(clip, weights=weights, scenechange=True),width=708,height=480,left=l).std.PlaneStats().get_frame(n).props['PlaneStatsAverage']
if avg > maxavg:
maxavg = avg
maxavg_l = l
else:
for frame in range(n,n+127):
if clip.get_frame(frame).props['_SceneChangePrev'] == 1:
if (frame - n) % 2 == 0:
weights = [1] * (frame - n - 1)
else:
weights = [1] * (frame - n)
break
if weights == []:
weights = [1] * 127
for l in range(0,13,2):
avg = core.std.CropAbs(core.frameblender.FrameBlend(clip,weights),width=708,height=480,left=l).std.PlaneStats().get_frame(n).props['PlaneStatsAverage']
if avg > maxavg:
maxavg = avg
maxavg_l = l
return core.text.Text(core.std.CropAbs(clip,width=708,height=480,left=maxavg_l),maxavg_l)
clip = core.std.FrameEval(template, functools.partial(selector,clip=clip), prop_src=clip, clip_src=clip)
clip.set_output()