FrameRateConverter (Official)

[Selur]

Nice! Thanks!

[Selur]

Hmm, when using FrameRateConverter in Vapoursynth I get:

vapoursynth.Error: BlankClip: nodes foreign to this core passed as input, improper api usage detected

script I used was:

# Imports
import os
import sys
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/FrameFilter/FramerateConverter/FrameRateConverter-x64.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# Import scripts
import havsfunc
import FrameRateConverter
# source: 'G:\TestClips&Co\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading source using FFMS2
clip = core.ffms2.Source(source="G:/TestClips&Co/test.avi",cachefile="E:/Temp/avi_9dec25d3f707eb4813d42334c7f1a8d6_853323747.ffindex",fpsnum=25,format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# adjusting frame count&rate with FramerateConverter
clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8, output="auto", skipOver=210) # new fps: 50
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model (i420@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 25.000fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Output
clip.set_output()

-> Any idea what could cause this?

Okay, replacing:

core = vs.get_core()

with:

from vapoursynth import core

in FrameRateConverter.py (and deleting the Python cache files) fixed the issue.

Cu Selur

Ps.: Should such thinks better be reported here, over at github or at both places?

[MysteryX]

I cannot reproduce your problem. This works for me

import os
import sys
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
import FrameRateConverter as frc
import vapoursynth as vs
core = vs.get_core()

core.std.LoadPlugin(path=r'C:\GitHub\FrameRateConverter\Src\x64\release\FrameRateConverter.dll')
core.avs.LoadPlugin(path=r'C:\GitHub\FrameRateConverter\masktools2.dll')
#file='E:\AVSMeter\Motion Estimation Torture Clip.avi'
file="E:\AVSMeter\MotionTest2.mp4"
#file="E:\NaturalGrounding\Girl's Day\Original\Female President.mp4"
video = core.ffms2.Source(source=file, format=vs.YUV420P8, alpha=False)
clip = video.std.SetFrameProp("_FieldBased", intval=0)

clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
clip = frc.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8, output="auto", skipOver=210) # new fps: 50
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
clip.set_output()

To keep note:

- Need to validate that _FieldBased == 0 or throw an error. The weird resizing issue I had was due to this flag being improperly set.

- Need to call AssumeFPS or validate that the input is already fixed frame rate

[MysteryX]

Better report issues here. vs.get_core or import core, why is it making a difference?

[Selur]

My guess is:
get_core(..) -> seems to create a new core-instance
import core -> takes the existing core-instance
compared how havsfunc got the core and there 'import ..' was used.

[MysteryX]

I'm doing vs.get_core() and it doesn't crash.

AH! -- libraries may need to specifically import core. Is this documented anywhere? Even there... I'm not getting your error.

[Selur]

Might be related to that I use a portable Vapoursynth version,...

[Selur]

A few questions regarding FrameRateConverter in Vapoursynth:

• Is it correct/intended that FrameRateConverter always outputs RGB24?
  Using:
  

  # Imports
  import os
  import sys
  import vapoursynth as vs
  core = vs.get_core()
  # Import scripts folder
  scriptPath = 'I:/Hybrid/64bit/vsscripts'
  sys.path.append(os.path.abspath(scriptPath))
  # Loading Plugins
  core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
  core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
  core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/FrameFilter/FramerateConverter/FrameRateConverter-x64.dll")
  core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
  # Import scripts
  import havsfunc
  import FrameRateConverter
  
  clip = core.ffms2.Source(source="G:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_6c441f37d9750b62d59f16ecdbd59393_853323747.ffindex",fpsnum=25,format=vs.YUV420P8,alpha=False)
  # making sure input color matrix is set as 470bg
  clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
  # making sure frame rate is set to 25
  clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
  # Setting color range to TV (limited) range.
  clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
  
  # adjusting frame count&rate with FramerateConverter
  clip = FrameRateConverter.FrameRateConverter(clip, newNum=25, newDen=1, blkSize=8, blkSizeV=8, output="over", maskThr=100, blendOver=65, skipOver=255) # new fps: 25
  
  # Output
  clip.set_output()

  the output is RGB24 (I was expecting YUV420P8, same as the input).
• If this is correct, does it respect the color matrix and luma range durint the yuv->rgb conversion?
• any plans for high bit depth support?

Cu Selur

[MysteryX]

That's because the Overlay function in VapourSynth only behaves as expected when done in the RGB space; gives weird result in YUV. I guess I could convert it back to the original format afterwards so that there's no matrix issue; and I should do that YUV->RGB->Overlay->YUV conversion in 16-bit.

There shouldn't be much issue with HBD, just need to try and see where it blocks.

[poisondeathray]

Originally Posted by MysteryX

That's because the Overlay function in VapourSynth only behaves as expected when done in the RGB space; gives weird result in YUV.

In what way is the Overlay result "weird" in YUV ?

[Selur]

I guess I could convert it back to the original format afterwards so that there's no matrix issue; and I should do that YUV->RGB->Overlay->YUV conversion in 16-bit.

Looking forward to it.

There shouldn't be much issue with HBD, just need to try and see where it blocks.

clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8)

with a yuv420p10 clip produces:

Traceback (most recent call last):
  File "src\cython\vapoursynth.pyx", line 2242, in vapoursynth.vpy_evaluateScript
  File "src\cython\vapoursynth.pyx", line 2243, in vapoursynth.vpy_evaluateScript
  File "C:\Users\Selur\Desktop\test.vpy", line 30, in <module>
    clip = FrameRateConverter.FrameRateConverter(clip, frameDouble=True, blkSize=8, blkSizeV=8) # new fps: 50
  File "I:\Hybrid\64bit\vsscripts\FrameRateConverter.py", line 178, in FrameRateConverter
    EM = ToGray(C.mv.Mask(bak, ml=255, kind=1, gamma=1/gam, ysc=255, thscd2=skipOver))
  File "src\cython\vapoursynth.pyx", line 2067, in vapoursynth.Function.__call__
vapoursynth.Error: Mask: input clip must be GRAY8, YUV420P8, YUV422P8, YUV440P8, or YUV444P8, with constant dimensions.

comes from https://github.com/dubhater/vapoursy.../MVMask.c#L328
Seems like the documentation (https://github.com/dubhater/vapoursy...ter/readme.rst) and functionality of mvtools don't match. :/
known 'missing feature' -> https://github.com/dubhater/vapoursy...ools/issues/16

[MysteryX]

Actually a 8-bit mask is perfectly fine. Just need to convert it to Gray16 to make it compatible.

[MysteryX]

As an update on the discussion about Pixel Addressing, it is now supported in this VapourSynth plugin.

[kedautinh12]

Time to replace masktool with ex_ family in Vapoursynth
https://github.com/Dogway/Avisynth-S...Converter.avsi

[MysteryX]

ex_ scripts haven't been ported to VapourSynth AFAIK. Plus the VPY replacement methods are highly optimized for HBD.

[kedautinh12]

About BicubicResize
https://forum.doom9.org/showthread.p...23#post1946623
https://forum.doom9.org/showthread.p...22#post1946622

[MysteryX]

DogWay, to replace Blur(.6) with a Convolution like you're doing, this convolution matrix gives a better result. If the blur is too wide, it lets artefacts leak through.

Convolution(matrix=[0, 1, 0, 1, 3, 1, 0, 1, 0])

for StripeMask giving "worse" results, I get similar results as before if doing it on PC range instead of Full range. Playing with the color range... seems like you can get better results by reducing the range. I might add a parameter to adjust the value range on top of the threshold.

[wonkey_monkey]

Wouldn't something like

Convolution(matrix=[1, 3, 1, 3, 9, 3, 1, 3, 1])

be more accurate? Ignoring the corner pixels seems wrong to me.

[Dogway]

I run some tests and yes Mysterys's kernel is more similar to either blur(0.6) or blur(0.5), actually a practical match to blur(0.5) besides the kernel is shorter so faster to compute.

Dog:      Expr("x[-1,1] x[0,1] x[1,1] x[-1,0] x[0,0] 6 * x[1,0] x[-1,-1] x[0,-1] x[1,-1] + + + + + + + + 0.071428571 *","")
monkey:   Expr("x[-1,1] x[0,1] 3 * x[1,1] x[-1,0] 3 * x[0,0] 9 * x[1,0] 3 * x[-1,-1] x[0,-1] 3 * x[1,-1] + + + + + + + + 0.04 *","")
Mystery:  Expr("x[0,1]  x[-1,0] x[0,0] 3 * x[1,0] x[0,-1] + + + + 0.1428571428 *","")
mergeluma(blur(0.6))
mergeluma(blur(0.5))

[MysteryX]

It's not about accuracy (of a pure white/black mask), but about having the right shape: not too wide, and not too square.