Vapoursynth

[Jukus]

How can make a smooth transition between different filters? For example:
clip1 = core.std.Trim(clip, 6223, 8361)
clip1 = adjust.Tweak(clip1, hue=0.0, sat=1.3, bright=8.0, cont=1.1, coring=True)

clip2 = core.std.Trim(clip, 8362, 9887)
clip2 = adjust.Tweak(clip2, hue=0.0, sat=1.4, bright=15.0, cont=1.3, coring=True)
That is, it would not be an instant change, but a smooth transition within 100 frames.

[poisondeathray]

How can make a smooth transition between different filters? For example:
clip1 = core.std.Trim(clip, 6223, 8361)
clip1 = adjust.Tweak(clip1, hue=0.0, sat=1.3, bright=8.0, cont=1.1, coring=True)

clip2 = core.std.Trim(clip, 8362, 9887)
clip2 = adjust.Tweak(clip2, hue=0.0, sat=1.4, bright=15.0, cont=1.3, coring=True)
That is, it would not be an instant change, but a smooth transition within 100 frames.

You can use CrossFade from kagefunc

[Jukus]

You can use CrossFade from kagefunc
No, need a smooth transition from some adjust settings to others, not frame blending.

[poisondeathray]

No, need a smooth transition from some adjust settings to others, not frame blending.

That's what it does, a smooth transition, if the base clip is the same "clip"

Think of it like this:

You have 1 clip. But 2 filtered versions.

1st clip fades out, but 2nd clip fades in , so the filter transition is applied smoothly over period "x"



If that doesn't do what you want, another way would be to write an animation helper function in python, using std.FrameEval . I'm not strong in python , someone can help you with that

[poisondeathray]

Here is an example of an parameter animation helper function, using tweak's sat .

Interpolation is linear from startframe to endframe , from sat0 to sat1 . (In this example from frame 100 to frame 160, sat=1 to sat=3)

You can add other parameters, but linear interpolation and animation does not necessarily work with all parameters for all types of filters


import vapoursynth as vs
import adjust
import functools
core = vs.get_core()

c = core.colorbars.ColorBars(format=vs.YUV444P10)
c = core.std.SetFrameProp(c, prop="_FieldBased", intval=0) # progressive
c = core.resize.Point(c,format=vs.YUV444P8)
c = c * 300
c = core.std.AssumeFPS(c, fpsnum=30000, fpsden=1001)
c = core.text.FrameNum(c, 7)

def satanim(n, sat0, sat1, startframe, endframe):
if n < startframe:
return adjust.Tweak(c, sat=sat0)
elif n > endframe:
return adjust.Tweak(c, sat=sat1)
else:
return adjust.Tweak(c, sat=round(n-endframe)*(-1*sat0)/(endframe-startframe) + round(n-startframe)/(endframe-startframe) * sat1)


ani = core.std.FrameEval(c, functools.partial(satanim, sat0=1, sat1=3, startframe=100, endframe=160))

ani.set_output()

[Jukus]

@poisondeathray
Thanks, I'll try again later.
I tried CrossFade and it really works as I suggested, that is, it does blending, while it loads the CPU very much. It also changes the total number of frames.

[StainlessS]

Note, that Jukus problem as posed, only applies correction 6223, 8361, and 8362, 9887, [ie below 6223 and above 9887 not touched]
whereas PDR script satanim alters entire clip [I assume that CrossFade does too].
This would seem to be correct, pointing this out to Jukus, that will not adjust only frames 6223 -> 9887. [CPU very much comment]

Of course the blending thing would not be appropriate where tweening eg HUE [circular].

EDIT:
It also changes the total number of frames.
Typical with any kind of dissolve, eg CrossFade.

Try the PDR Satanim() thing, I think will preserve length.

EDIT: Ignore this
EDIT: Use trim to only process required range with satanim, then splice back into original clip.

[poisondeathray]

The helper function returns original clip, unless it's within startframe, endframe range. Length is unchanged . It's similar to animate() in avisynth, where parameters of a function are animated. It's a PITA IMO to setup. It's 100000x (or maybe more like 10000000x) easier to do any type of keyframe interpolation in a NLE or a GUI . And you can easily use other types of animation interpolation (not just linear), and some can GUIs have curves

Instead of using a dissolve function, the other way is to create an alpha channel control clip (white fade to black, or vice versa) where you use Overlay() with the alpha channel mask. This way the clip length is unchanged. This is essentially an animated blend between 2 or more layers.

[StainlessS]

The helper function returns original clip, unless it's within startframe, endframe range.
Oops, yes.

[Jukus]

I've already thought about some kind of graphic editor, but I still need filters that only Vapoursynth / Avisynth have.
Сan quickly encode the video in a graphics editor, and only then do it good in Vapoursynth, but that's bd 90 minutes video.

[poisondeathray]

I've already thought about some kind of graphic editor, but I still need filters that only Vapoursynth / Avisynth have.


Which filters ?

Your example used Tweak; - You have far more control over color manipulation, saturation, contrast, levels , hue etc.. in other programs. All keyframeable. Davinci Resolve, or almost any NLE's.

[Jukus]

Which filters ?
Ideally, I would like to use QTGMC with InputType = 1, but can only get by with a noise removal filter.

[poisondeathray]

Ideally, I would like to use QTGMC with InputType = 1, but can only get by with a noise removal filter.

That's OK , avs/vpy are great for some things, bad for others. Pros/Cons like anything

But color manipulation - especially animated parameters, or animated masks/roto - are not avisynth/vapoursynth strong points. If it's something simple , maybe you can stay in avisynth/vapoursynth. But many types of operations cannot be done (or would take years to do) in avs/vpy .

Nothing wrong with combining tools/workflows either - use what works for your needs

Writing a helper function, and defining each parameter is a PITA. And you can't even do realtime adjustments (you can't "see" what you're doing as you're changing), you have to refresh, change, refresh . It's too slow for serious work.

IMO, if staying withing avs/vpy, the overlay blend is the fastest / easiest way to control animation parameters (mix and matching filtered clip states, and the interpolation/transition is controlled by the animated luma mask)

[_Al_]

Using filters by frame number , that question was already here, it is a good idea to use it within FrameEval().
I would even push it even further, so it is more general, using some MAP table, split functions into separate function.

So it is visually very easy to fix, easy to add functions, and functions could be chained (more filters for the same frame):
#just demo, filters do not make sense, just to see how it works

import functools

def func1(clip):
return clip.std.Expr(['x 50 -','','']) #darken

def func2(clip):
return clip.std.Expr(['x 50 +','','']) #brighten

MAP = { #frame intervals, inclusive #filters
( 0, 256) : [func1],
(257, clip.num_frames-1) : [func1, func2]
#other intervals and functions could be added
}

def distribute_filters(n, clip):
for (lower, upper), funcs in MAP.items():
if lower <= n <= upper:
clip = functools.reduce(lambda r, f: f(r), funcs, clip) #this chains filters, example: clip=func2(func1(clip))
return clip

fixed = core.std.FrameEval(clip, functools.partial(distribute_filters, clip=clip))
fixed.set_output()

[_Al_]

To include crossfades it gets more tricky because filter needs frame number n and interval frame numbers, the whole script with that question about adjust.tweak():

import functools
import adjust

source_clip = ...... use your source filter

def tweak1(clip, *args):
return adjust.Tweak(clip, hue=0.0, sat=1.3, bright=8.0, cont=1.1, coring=True)

def tweak2(clip, *args):
return adjust.Tweak(clip, hue=0.0, sat=1.4, bright=15.0, cont=1.3, coring=True)

def crossfade1(clip, n, lower, upper):
return core.std.Merge(tweak1(clip[n]), tweak2(clip[n]), weight=(n-lower)/(upper-lower))

MAP = {
(6223, 8311) : [tweak1],
(8312, 8412) : [crossfade1],
(8413, 9887) : [tweak2]
}

def distribute_filters(n, clip):
for (lower, upper), funcs in MAP.items():
if lower <= n <= upper:
clip = functools.reduce(lambda r, f: f(r, n, lower, upper), funcs, clip)
return clip

out_clip = core.std.FrameEval(source_clip, functools.partial(distribute_filters, clip=source_clip))
out_clip.set_output()

crossfades is just using core.std.Merge() so down to basics, it could be called just like this,
and again, filters could be chained in that MAP dictionary, like [tweak1, my_filter, my_other filter] , so more filters could be used on the same frame

[_Al_]

It's 100000x (or maybe more like 10000000x) easier to do any type of keyframe interpolation in a NLE or a GUI . Lets make it better, something like 100x easier in NLE than in vs :-)

To animate all arguments in a filter (if possible) instead of crossfade could be done like this:
import vapoursynth as vs
from vapoursynth import core
import functools
import adjust
source_clip = .....

TWEAK1 = dict(hue=0.0, sat=1.3, bright=8.0, cont=1.1, coring=True)
TWEAK2 = dict(hue=0.0, sat=1.4, bright=15.0, cont=1.3, coring=True)
TWEAK_TYPES = dict(hue=None, sat= float, bright=float, cont=float, coring=None) #int, float or None to not animate argument

TWEAK_DIFF = dict()
for key, value in TWEAK1.items():
if TWEAK_TYPES[key] is not None:
TWEAK_DIFF[key] = round(TWEAK1[key]-TWEAK2[key]) if TWEAK_TYPES[key] == int else round(TWEAK1[key]-TWEAK2[key],2)

def tweak1(clip, *args):
return adjust.Tweak(clip, **TWEAK1)

def tweak2(clip, *args):
return adjust.Tweak(clip, **TWEAK2)

def crossfade1(clip, n, lower, upper):
tweak_out = dict()
for key, value in TWEAK2.items():
if TWEAK_TYPES[key] is not None:
tweak_out[key] = TWEAK1[key] - ((n-lower)/(upper-lower) * TWEAK_DIFF[key])
else:
tweak_out[key] = value
#print(tweak_out) #debug to see argument values for a frame
return adjust.Tweak(clip, **tweak_out)

MAP = {
(6223, 8311) : [tweak1],
(8312, 8412) : [crossfade1],
(8413, 9887) : [tweak2]
}

def distribute_filters(n, clip):
for (lower, upper), funcs in MAP.items():
if lower <= n <= upper:
clip = functools.reduce(lambda r, f: f(r, n, lower, upper), funcs, clip)
return clip

out_clip = core.std.FrameEval(source_clip, functools.partial(distribute_filters, clip=source_clip))
out_clip.set_output()
Not sure about performance, but it should behave as all in FrameEval() hopefully

[_Al_]

i did not took care if there is int type of value for that tweak_out[key] in crossfade1(), if it is 'int', it needs to be rounded up to integer, but anyway ...

[Selur]

Small question: How to handle 32bit Tiff with Levels properly?

I wanted to load a single 32bit TIFF with vsImageReader and apply levels on it.
Problem is I have no clue how to do it properly.
I assumed that the color space should be RGBS with 32bit precision and thus analog to limiting to 16-235 I used min_in = 16 << (32-8), max_in=235 << (32-8), min_out= 16 << (32-8), max_out = 235 << (32-8).
That gave me a black preview and I'm not sure whether this is due to a limitation of the preview, if I did something wrong or if it's a limitation somewhere else.
So any help is welcome.

# Imports
import vapoursynth as vs
core = vs.get_core()
# source: 'C:/Users/Selur/Desktop/images v2/TIFF 32-bit.tif'
# current color space: RGBS, bit depth: 32, resolution: 640x480, fps: 25, color matrix: 709, yuv luminance scale: full, scanorder: progressive
# Loading C:\Users\Selur\Desktop\images v2\TIFF 32-bit.tif using vsImageReader
clip = core.imwri.Read(["C:/Users/Selur/Desktop/images v2/TIFF 32-bit.tif"])
clip = core.std.Loop(clip=clip, times=100)
# Input color space is assumed to be RGBS
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
# Color Adjustment
clip = core.std.Levels(clip=clip, min_in=268435456, max_in=18446744073357230080, min_out=268435456, max_out=18446744073357230080)
# Output
clip.set_output()


Cu Selur

Ps.: I know that there's not many filters and output formats that support RGBS, but that's okay.

[feisty2]

is it settled that the new API will be placed in "VapourSynth4.h" and "VapourSynth.h" is always the old API?

[poisondeathray]

Small question: How to handle 32bit Tiff with Levels properly?

I assumed that the color space should be RGBS with 32bit precision


Depends on what you have, where it's from

Usually 32bpc float would be loaded with float_output=True

imwri.Read(....float_output=True)

RGBS would "normal" values be 0 to 1 , but values can be negative or >>1 .

eg. it's normal for a HDR sequence have values 10 or 20 with usable data

[Myrsloik]

is it settled that the new API will be placed in "VapourSynth4.h" and "VapourSynth.h" is always the old API?

Yes. Only minor changes are planned from now on.

[Selur]

@poisondeathray: thanks :) (not tackling HDR atm.)

[neo_sapien]

Can someone please check my script? I took an Avisynth script and turned it into a Vapoursynth version, and I want to make sure that they're comparable.

Avisynth script:

TFM()
TDecimate()
QTGMC2 = QTGMC(Preset="Very Slow", SourceMatch=3, TR2=4, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC3 = QTGMC(preset="Very Slow", inputType=3, TR2=4)
Repair(QTGMC2,QTGMC3, 9)


Vapoursynth script:

clip = core.std.SetFieldBased(clip, 2) # 1 = BFF, 2 = TFF
clip = core.vivtc.VFM(clip, 1)
clip = core.vivtc.VDecimate(clip)
QTGMC1 = havsfunc.QTGMC(clip, TFF = True, Preset="Very Slow", SourceMatch=3, TR2=4, InputType=2, Lossless=2, MatchEnhance=0.75, Sharpness=0.5, MatchPreset="Very Slow", MatchPreset2="Very Slow")
QTGMC2 = havsfunc.QTGMC(clip, TFF = True, Preset="Very Slow", InputType=3, TR2=4)
clip = core.rgvs.Repair(QTGMC1,QTGMC2, 9)


I'm also using this script to try and upscale my clip, doubling the height and width, after it's made progressive. This is just enlarging, right? It's not trying to re-deinterlace? Since I noticed the field bit.


clip = core.nnedi3cl.NNEDI3CL(clip, field = 1, pscrn=2, nsize=4, qual =2, nns =4, dh=True, dw=True)

[HuBandiT]

Small question: How to handle 32bit Tiff with Levels properly?

I assumed that the color space should be RGBS with 32bit precision and thus analog to limiting to 16-235 I used min_in = 16 << (32-8), max_in=235 << (32-8), min_out= 16 << (32-8), max_out = 235 << (32-8).
So any help is welcome.


PlaneStats is your friend: http://www.vapoursynth.com/doc/functions/planestats.html

And so is a histogram: https://github.com/dubhater/vapoursynth-histogram

Edit: no, actually PlaneStats is not good for this case - according to the docs at least it does some kind of normalization on the input.

[feisty2]

Do frame properties exist for audio frames as well? (I still don’t quite understand what a frame means for audio...)

[Myrsloik]

Do frame properties exist for audio frames as well? (I still don’t quite understand what a frame means for audio...)

Yes. They exist but I'm not sure what they're good for. One audio frame is up to 3000 samples. No idea what you'd actually want to flag that way though.

[feisty2]

Is it legal to instantly fetch a frame (via getFrameFilter) after requesting it (in the arInitial branch)?

[Myrsloik]

Is it legal to instantly fetch a frame (via getFrameFilter) after requesting it (in the arInitial branch)?

I think so. Any frame that isn't ready yet should simply return null.

But why would you do that?

[feisty2]

I guess to eliminate the need to call GetFrame() on video nodes. After a frame has been requested, it could be instantly fetched and stored in an associative container held by the video node, so the user may access the frame via Clip[n] instead of Clip.GetFrame(n, FrameContext).

[_Al_]

Having this code:
clip = ... interlaced 25 fps clip
clip2 = havsfunc.QTGMC(clip, Preset="Fast", TFF=True)
print(clip2)
correctly shows:
Format: YUV420P8
Width: 1920
Height: 1080
Num Frames: 600
FPS: 50
Flags: NoCache
but
clip = ... interlaced 25 fps clip
def deint(n, clip):
return havsfunc.QTGMC(clip, Preset="Fast", TFF=True)
clip2 = clip.std.FrameEval(functools.partial(deint, clip=clip))
print(clip2)
shows wrong number of frames:
Format: YUV420P8
Width: 1920
Height: 1080
Num Frames: 300
FPS: 25
Flags: NoCache IsCache

if its within FrameEval() it passes originals clip fps for print function. clip2 is properly bobed, but evaluation prints original clips values.


Now looking at it, it actually is correct behavior, how to fix it though so it bobs all frames?

another EDIT:
assuming that first argument is just a placeholder so:
clip2 = core.std.FrameEval(clip.std.AssumeFPS(fpsnum=50, fpsden=1)*2, functools.partial(deint, clip=clip)) seams to give good result, hopefully nothing is broken

[_Al_]

It does not work. I guess frame number in must be total out, its how it works.
Or maybe is there a way how to manipulate that n using lambda somehow?
clip2 = clip.std.FrameEval(lambda n: deint(n, clip=clip))
To do something with that n to force FrameEval pass more frames out than in?

[_Al_]

Just tried it again, yes it works with that placeholder with doubled fps and doubled lenght,
not sure what went wrong yesterday, it seams to return clip correctly from FrameEval().
It works with both, partial and lambda function as well.

[feisty2]

are clips with varying format no longer supported in API v4?
I notice that "format" is no longer a pointer in the "info" struct.

[Myrsloik]

are clips with varying format no longer supported in API v4?
I notice that "format" is no longer a pointer in the "info" struct.

Nope, still supported. It's when the format is pfNone (0).

[feisty2]

is it possible for a filter to return multiple nodes, and some of them are vnodes while others are anodes?

[feisty2]

also, the documentation on calling python functions is a bit unclear, like it was not documented that the return value binds to the "val" key, if the function has multiple return values, will they all bind to "val"?

[Myrsloik]

is it possible for a filter to return multiple nodes, and some of them are vnodes while others are anodes?

Filters can't but functions can. A single VSMap key can only hold anodes or vnodes.

[feisty2]

Filters can't but functions can.

okay, then how do I retrieve the return values from such function in a C++ plugin? I know the first return value binds to "val", what about other return values?

[feisty2]

I played with some pretty ugly hacks


// C++ plugin
auto ret = VSFuncRef_stuff();
auto msg = ""s;
auto m = ret.map_pointer;

for (auto x : Range{ vsapi->propNumKeys(m) }) {
msg += vsapi->propGetKey(m, x);
msg += "\n";
}

throw msg;

# Python tests
def f():
return 'aaa', 'bbb'

def g():
return 'aaa', 123

core.???.Test(f) # okay, error message says "val"
core.???.Test(g) # error message says "not all values are of the same type in val"


and observed from the error log of vsedit that:
1) it is allowed to call a python function with multiple return values of the same type
2) all return values bind to the same key called "val"
3) it is not allowed to call a python function with multiple return values of different types because of 2)

@Myrsloik
technical details like this should really be documented rather than leave us wild guessing and poking around.

[feisty2]

is paTouch removed from api v4?

[Myrsloik]

is paTouch removed from api v4?

Yes, there's the mapSetEmpty() instead.

[feisty2]

is there an example for createFunc(), it seems like a way to call arbitrary C++ functions in py scripts?

[Myrsloik]

is there an example for createFunc(), it seems like a way to call arbitrary C++ functions in py scripts?

I think it's only used in python code (see vapoursynth.pyx for c-ish code) and then lut/lut2 for a callFunc() example.

[Selur]

Is there some known issue with fmtc and R52?
using:
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="I:/workspace/Hybrid/debug/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/workspace/Hybrid/debug/64bit/vsfilters/SourceFilter/FFMS2/ffms2.dll")
# source: 'F:\TestClips&Co\files\test.avi'
# current color space: YUV420P8, bit depth: 8, resolution: 640x352, fps: 25, color matrix: 470bg, yuv luminance scale: limited, scanorder: progressive
# Loading source using FFMS2
clip = core.ffms2.Source(source="F:/TestClips&Co/files/test.avi",cachefile="E:/Temp/avi_078c37f69bb356e7b5fa040c71584c40_853323747.ffindex",format=vs.YUV420P8,alpha=False)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
original = clip
clip = core.fmtc.resample(clip=clip, kernel="gaussian", w=1280, h=704, interlaced=False, interlacedd=False)
original = core.fmtc.resample(clip=original, kernel="bicubic", w=1280, h=704, interlaced=False, interlacedd=False)
# adjusting output color from: YUV420P16 to YUV420P8 for FFvHuffModel (i420@8)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")
# adjusting for FilterView
if (original.format.id != clip.format.id):
if (original.format.color_family == vs.RGB and clip.format.color_family != vs.RGB):
original = core.resize.Bicubic(clip=original, format=clip.format.id, matrix_s="470bg", range_s="limited")
elif (original.format.color_family == clip.format.color_family):
original = core.resize.Bicubic(clip=original, format=clip.format.id, range_s="limited")
else:
original = core.resize.Bicubic(clip=original, format=clip.format.id, matrix_in_s="470bg", range_s="limited")
stacked = core.std.StackHorizontal([original,clip])
# set output frame rate to 25.000fps
stacked = core.std.AssumeFPS(clip=stacked, fpsnum=25, fpsden=1)
# Output
stacked.set_output()
both Vapoursynth Editor and vspipe both sometimes crash without an error, while other times vspipe crashes after a few hundred frames and then both run fine.

Cu Selur

[feisty2]

is copyFrameProps removed from API v4?

[Myrsloik]

Is there some known issue with fmtc and R52?
using:
...
both Vapoursynth Editor and vspipe both sometimes crash without an error, while other times vspipe crashes after a few hundred frames and then both run fine.

Cu Selur

Hard to tell what causes it without some kind of crash dump for debugging. I haven't seen any reported issues and nothing has changed in VS in frame/filter handling.

[Myrsloik]

is copyFrameProps removed from API v4?

I think I did. It was replaced with a map copy functions so you have to go the long way around and use that instead.

[feisty2]

there's no preset for audio formats?

[feisty2]

is it possible to change the API for releaseFrameEarly to void(const VSFrameRef*, VSFrameContext*) (release directly from a frame object rather than from a node object)?
the current API kind of breaks RAII for C++, a frame object always keeps the ownership of its underlying frame reference, if a frame reference is released from a node, there's no way to inform the frame object who's keeping the ownership of the reference that the reference has expired, and there will be a "double free" error after getFrame() returns.

[feisty2]

or maybe releaseFrameEarly() could set a frame to a "zombie" mode, the most memory consuming part (the image content of the frame) is instantly released after calling this function, but the frame header, particularly the part that's keeping the reference count is still alive in the memory, so there won't be a "double free" error when any frame object goes out of scope.