Why is FrameEval Crashing

[MysteryX]

Why is this method crashing when I add FrameEval? Memory out of bound exception.

# BM3D denoising method
def BM3D(clip, sigma, radius, gpuid, chroma, ref, outbits):
clean = clip.resize.Bicubic(format=vs.RGBS, matrix_in_s="709") if chroma else clip.fmtc.bitdepth(bits=32, dmode=1)
ref = ref.resize.Bicubic(format=vs.RGBS, matrix_in_s="709") if chroma else ref.fmtc.bitdepth(bits=32, dmode=1)
clean = clean.bm3dcuda.BM3D(sigma=[sigma,sigma,sigma], radius=radius, ref=ref, device_id=gpuid, fast=False)
if radius > 0:
clean = clean.bm3d.VAggregate(radius=radius)

if clip.format.subsampling_w == 1 and clip.format.subsampling_h == 1:
fmt = vs.YUV420P16
elif clip.format.subsampling_w == 1 and clip.format.subsampling_h == 0:
fmt = vs.YUV422P16
else:
fmt = vs.YUV444PS if outbits == 32 else vs.YUV444P16
#return clean.resize.Bicubic(format=fmt, matrix_s="709", range = 1)

# Convert back while respecting _ColorRange of each frame. Note that setting range=1 sets _ColorRange=0 (reverse)
def ConvertBack(n, f, clean, format):
fullRange = '_ColorRange' in f.props and f.props['_ColorRange'] == 0
return clean.resize.Bicubic(format=format, matrix_s="709", range = 1 if fullRange else 0)
return clean.std.FrameEval(functools.partial(ConvertBack, clean=clean, format=fmt), prop_src=clip)


If I want to do this correctly, I'd have to run a FrameEval before and after to resize using the right ChromaLocation, ColorRange, Primaries, Matrix, Transfer? Or it uses them automatically while converting YUV -> RGB but need to be set manually when converting back RGB -> YUV? Can someone help me write part in the right way?

[Myrsloik]

It shouldn't crash. Which VS version did you use? And can you paste the full error message.

[MysteryX]

I found out that it can crash due to 3 reasons

1) If the sub-function has same clip parameter name as the function -- it doesn't like that

2) If the first parameter doesn't have the same format as the output of FrameEval, it crashes with memory exception like that. Doing it like this works.


return core.std.BlankClip(src, format=fmt).std.FrameEval(functools.partial(ConvertBack, clip=c, format=fmt), prop_src=src)


3) If there's an error within FrameEval, it might just throw a memory exception instead of telling you what's going on

[_Al_]

I know I cannot use this for example, I was scratching my head for a while to figure out, why FrameEval was not working:
clip = clip.std.FrameEval(.......)
it has to be:
other_name_clip = clip.std.FrameEval(.......)

output format difference, I'd expect that is not going work