Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd October 2021, 17:55   #1  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Why is FrameEval Crashing

Why is this method crashing when I add FrameEval? Memory out of bound exception.

Code:
# BM3D denoising method
def BM3D(clip, sigma, radius, gpuid, chroma, ref, outbits):
    clean = clip.resize.Bicubic(format=vs.RGBS, matrix_in_s="709") if chroma else clip.fmtc.bitdepth(bits=32, dmode=1)
    ref = ref.resize.Bicubic(format=vs.RGBS, matrix_in_s="709") if chroma else ref.fmtc.bitdepth(bits=32, dmode=1)
    clean = clean.bm3dcuda.BM3D(sigma=[sigma,sigma,sigma], radius=radius, ref=ref, device_id=gpuid, fast=False)
    if radius > 0:
        clean = clean.bm3d.VAggregate(radius=radius)

    if clip.format.subsampling_w == 1 and clip.format.subsampling_h == 1:
        fmt = vs.YUV420P16
    elif clip.format.subsampling_w == 1 and clip.format.subsampling_h == 0:
        fmt = vs.YUV422P16
    else:
        fmt = vs.YUV444PS if outbits == 32 else vs.YUV444P16
    #return clean.resize.Bicubic(format=fmt, matrix_s="709", range = 1)

    # Convert back while respecting _ColorRange of each frame. Note that setting range=1 sets _ColorRange=0 (reverse)
    def ConvertBack(n, f, clean, format):
        fullRange = '_ColorRange' in f.props and f.props['_ColorRange'] == 0
        return clean.resize.Bicubic(format=format, matrix_s="709", range = 1 if fullRange else 0)
    return clean.std.FrameEval(functools.partial(ConvertBack, clean=clean, format=fmt), prop_src=clip)
If I want to do this correctly, I'd have to run a FrameEval before and after to resize using the right ChromaLocation, ColorRange, Primaries, Matrix, Transfer? Or it uses them automatically while converting YUV -> RGB but need to be set manually when converting back RGB -> YUV? Can someone help me write part in the right way?
MysteryX is offline   Reply With Quote
Old 4th October 2021, 09:04   #2  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,555
It shouldn't crash. Which VS version did you use? And can you paste the full error message.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 11th October 2021, 16:59   #3  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
I found out that it can crash due to 3 reasons

1) If the sub-function has same clip parameter name as the function -- it doesn't like that

2) If the first parameter doesn't have the same format as the output of FrameEval, it crashes with memory exception like that. Doing it like this works.

Code:
return core.std.BlankClip(src, format=fmt).std.FrameEval(functools.partial(ConvertBack, clip=c, format=fmt), prop_src=src)
3) If there's an error within FrameEval, it might just throw a memory exception instead of telling you what's going on

Last edited by MysteryX; 11th October 2021 at 17:17.
MysteryX is offline   Reply With Quote
Old 11th October 2021, 17:28   #4  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 321
I know I cannot use this for example, I was scratching my head for a while to figure out, why FrameEval was not working:
clip = clip.std.FrameEval(.......)
it has to be:
other_name_clip = clip.std.FrameEval(.......)

output format difference, I'd expect that is not going work
_Al_ is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.