Duplicate aware resizing,..

[Selur]

Based on a discussion a while ago the idea was to write a script so that if you have a clip with duplicate frames in it, instead of applying the frame based resizer (without a temporal component) to each frame of a row of duplicates,
you just apply it to the first frame and than duplicate that frame to replace the duplicates.

This was meant to speed up using non temporal resizering like done with for example vsgan and similar.
Taking the idea I started writing a script which at first should work for the core.resize (http://www.vapoursynth.com/doc/functions/video/resize.html) resizers:
import vapoursynth as vs
from vapoursynth import core

'''
DubplicateAwareResizing:
The idea of DubplicateAwareResizing is that if you have a clip with duplicate frames in it,
instead of applying the frame based resizer (without a temporal component) to each frame of a row of duplicates,
you just apply it to the first frame and than duplicate that frame to replace the duplicates.

call using:
from DubplicateAwareResizing import DAResizer
sr = DAResizer(clip, thresh=0.001, method='XXX')
clip = sr.out

'''

# tWidth, tHeight: target resolution
# thresh: threshold for duplicate detection
# method: resizing method to use
# model: model parameter for vsgan
# debug: add debug info to frame
# device_index: set device to be used for gpu based resizing
class DAResizer:
# constructor
def __init__(self, clip: vs.VideoNode, tWidth: int, tHeight: int, thresh: float=0.001, method: str='Bicubic', model: str='', debug: bool=False, device_index: int=0):
self.clip = core.std.PlaneStats(clip, clip[0]+clip)
self.thresh = thresh
self.debug = debug
self.method = method
self.model = model
self.device_index = device_index
self.tWidth = tWidth
self.tHeight = tHeight
if self.method == 'VSGAN' and model == '':
raise ValueError(f'DAResizer: "method" \'{self.method}\' called while not setting a model!')

def daResize(self, n, f):
out = self.resize(n)
if self.debug:
return out.text.Text(text="avg: "+str(f.props['PlaneStatsDiff']),alignment=8)
return out

def resize(self, n):
if self.is_duplicate(n):
return self.previous

# TODO: add VSGAN

# apply generig resizer by method
resize_method = getattr(self.clip[n].resize, self.method, None)
if resize_method is None:
raise ValueError(f'DAResizer: Unknown "method" \'{self.method}\' called!')
resized = resize_method(width=self.tWidth, height=self.tHeight)
self.previous = resized
return resized

def is_duplicate(self, n):
# first frame can't be a duplicate, after that check agains the threshold
return n != 0 and self.clip.get_frame(n).props['PlaneStatsDiff'] <= self.thresh

@property
def out(self):
# TODO: needs to be adjusted for non YUV input
self.clip = core.std.PlaneStats(self.clip, self.clip[0] + self.clip)
return core.std.FrameEval(self.clip, self.daResize, prop_src=self.clip)

Problem is, I made a mistake there since when calling the script:
from DuplicateAwareResizing import DAResizer
sr = DAResizer(clip, tWidth=1280, tHeight=704, thresh=0.001, method='Bicubic', debug=True)
clip = sr.out

I get:
Error: Failed to retrieve frame 0 with error: FrameEval: Returned frame has wrong dimensions
I'm probably overlooking something obvious, but since I don't see it, I thought that may be someone here has an idea. :)

Cu Selur

[WolframRhodium]

return core.std.FrameEval(self.clip, self.daResize, prop_src=self.clip)

in the last line should be

return core.std.FrameEval(self.clip.std.BlankClip(width=self.tWidth, height=self.tHeight), self.daResize, prop_src=self.clip)

[Selur]

Thanks!
Got it working with that!
for those interested, added the script to: https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/DuplicateAwareResizing.py

Cu Selur

[_Al_]

I know I introduced that clip.out return from class into script, but it migh freak out users, they expect to use schemes like:
clip = func(....)
In case of this latest DAResize , DA Bears! :-), for example I'd include this function at the end of your posted script (on github):

def da_resizer(clip: vs.VideoNode,
tWidth: int,
tHeight: int,
thresh: float=0.001,
method: str='Bicubic',
vsgan_models: List[str]=None,
debug: bool=False,
device_index: int=0):

sr = DAResizer(clip, tWidth, tHeight, thresh, method, vsgan_models, debug, device_index)
return sr.out
this way DAResizer class is hidden and function can be called from scripts like:

from DuplicateAwareResizing import da_resizer
clip = da_resizer(clip, ... )
This addition does not brake functionality, so whomever used scrips before with that class instantiating first, it would still work

[Selur]

added a wrapper for easier usage to the code