Vapoursynth

[Myrsloik]

there's no preset for audio formats?

No, trivial to query so I didn't see any real need. Also they can't be properly serialized into a single int.

[Myrsloik]

is it possible to change the API for releaseFrameEarly to void(const VSFrameRef*, VSFrameContext*) (release directly from a frame object rather than from a node object)?
the current API kind of breaks RAII for C++, a frame object always keeps the ownership of its underlying frame reference, if a frame reference is released from a node, there's no way to inform the frame object who's keeping the ownership of the reference that the reference has expired, and there will be a "double free" error after getFrame() returns.

No, releaseFrameEarly() is more or less deprecated and useless for 99.99% of filters. Write a wrapper that caters to the masses instead.

[feisty2]

how do I obtain the return value of the function that is passed to createFunc() in python?


// C++ code, inside getFrame()
auto f = [](auto in, auto out, auto, auto, auto) {
vsapi->propSetFloat(out, "val", 2.71, VSPropAppendMode::paReplace);
vsapi->logMessage(VSMessageType::mtWarning, "aaaaaaa");
};

auto fp = Function{ vsapi->createFunc(f, nullptr, [](auto) {}, core, vsapi) };
ProcessedFrame["test"] = fp;


#Python script
clip = core.test.Test(clip)
x = clip.get_frame(0).props['test']() # prints "aaaaaaa" as expected, however x is of type None rather than float

[feisty2]

it seems the problem is caused by Func.__call__() discarding its return value: https://github.com/vapoursynth/vapoursynth/blob/doodle1/src/cython/vapoursynth.pyx#L636
this "ret" thing, after receiving whatever stored in the out map, was never returned by __call__ and was simply discarded.
is this a bug?

[feisty2]

is the message handler attached to each core and no longer global in API v4?

[Myrsloik]

is the message handler attached to each core and no longer global in API v4?

Correct

[feisty2]

what's the use of removeMessageHandler()? it seems I can remove the current handler by simply passing a NULL handler to addMessageHandler()?
who owns the char pointer that the handler receives? is it the same pointer passed to logMessage() or a pointer to some internal deep copy of what's passed to logMessage()?

[Myrsloik]

what's the use of removeMessageHandler()? it seems I can remove the current handler by simply passing a NULL handler to addMessageHandler()?
who owns the char pointer that the handler receives? is it the same pointer passed to logMessage() or a pointer to some internal deep copy of what's passed to logMessage()?

addMessageHandler() simply adds one more handler, it never removes the current one
you then remove the handler by passing the handle from addMessageHandler() to removeMessageHandler()
Ownership is also optional since all handlers obviously will be removed automatically when a core is destroyed. Calling removeMessageHandler() with an invalid/already freed handle is safe and does nothing.

[feisty2]

interesting, I didn't realize there could be multiple message handlers. so logMessage() will send the message to all registered handlers?

[Myrsloik]

interesting, I didn't realize there could be multiple message handlers. so logMessage() will send the message to all registered handlers?

Yes

Filler here

[feisty2]

I can't decide which is the correct design for message handlers.
should I bind the lifetime of a message handler to its descriptor, like a file descriptor?

auto md = Core.AddMessageHandler([](auto...) {});
// md is a stateful object
// automatically calls removeMessageHandler() in md's destructor when it goes out of scope

or should I let the user manage the handler's lifetime manually?

auto md = Core.AddMessageHandler([](auto...) {});
// md is a stateless integer ID / pointer

Core.Eject(md);
// explicitly ejects the handler when no longer needed.

[feisty2]

does getFrameAsync() have any special behavior (creating new threads, etc.)?
it seems the same asynchronous behavior could be achieved by simply calling getFrame() with std::async?

auto f = std::async(std::launch::async, [&] { return vsapi->getFrame(n, node, nullptr, 0); });

... // do something else while waiting for f

auto frame = f.get(); // block the current thread and retrieve the frame when needed


why should I use getFrameAsync() instead of something like the code above?

[Selur]

I use a jpeg as source:
General
Complete name : c:\Users\Selur\Desktop\test.jpg
Format : JPEG
File size : 465 KiB

Image
Format : JPEG
Width : 1 280 pixels
Height : 534 pixels
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Compression mode : Lossy
Stream size : 465 KiB (100%)
which I open using:
clip = core.imwri.Read(["C:/Users/Selur/Desktop/test.jpg"])
clip = core.std.Loop(clip=clip, times=100)
what confuses me is that the file is reported as RGB24.
Does ImageMagick Writer-Reader (http://www.vapoursynth.com/doc/plugins/imwri.html) always return RGB or am I missing something?
In case it always returns RGB, it would be nice if this could be added to the documentation.

Cu Selur

[Myrsloik]

I use a jpeg as source:
General
Complete name : c:\Users\Selur\Desktop\test.jpg
Format : JPEG
File size : 465 KiB

Image
Format : JPEG
Width : 1 280 pixels
Height : 534 pixels
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Compression mode : Lossy
Stream size : 465 KiB (100%)
which I open using:
clip = core.imwri.Read(["C:/Users/Selur/Desktop/test.jpg"])
clip = core.std.Loop(clip=clip, times=100)
what confuses me is that the file is reported as RGB24.
Does ImageMagick Writer-Reader (http://www.vapoursynth.com/doc/plugins/imwri.html) always return RGB or am I missing something?
In case it always returns RGB, it would be nice if this could be added to the documentation.

Cu Selur

It probably always returns RGB... maybe.

[feisty2]

some elaboration on the issue mention at #4187 would be nice...
it's hard to determine the design of the async interface without knowing the technical details of the C API. also it's a lot harder to convert a callback kind of stuff to std::future which involves locks, condition variable and other unnecessary complexity.

[Myrsloik]

some elaboration on the issue mention at #4187 would be nice...
it's hard to determine the design of the async interface without knowing the technical details of the C API. also it's a lot harder to convert a callback kind of stuff to std::future which involves locks, condition variable and other unnecessary complexity.

Internally getFrame() is implemented as a function that waits for getFrameAsync() to return. So I guess wrapping getFrame() in a future would be close at least. But with a lot of unnecessary locking and bookkeeping added.

[feisty2]

I think "id" should be a member of VSVideoFormat in API v4, otherwise it requires access to the core to determine if a clip has a constant format which is not very convenient

[feisty2]

or at least make the core parameter optional for queryVideoFormatID()

[feisty2]

it seems API v4 uses the colorfamily to determine if a format is constant, that's a bit weird...

[Selur]

Is there an "autowhite" filter which is more clever than the example over at http://www.vapoursynth.com/doc/functions/frameeval.html ?

[feisty2]

What do you mean by "more clever"? Like adding temporal consistency?

[Selur]

Yes, exactly. Temporal temporal consistency while taking scene changes into account. I was looking for some 'auto' filters like AutoAdjust, AutoContrast, Autolevels, HDRAGC and similar for Vapoursynth and the frameeval example of autowhite was the only thing I found. Seeing that it was a 'simple per frame auto white balance', I was wondering if there are mor complex/clever solutions out there. :)

[feisty2]

Yes, exactly. Temporal temporal consistency while taking scene changes into account. I was looking for some 'auto' filters like AutoAdjust, AutoContrast, Autolevels, HDRAGC and similar for Vapoursynth and the frameeval example of autowhite was the only thing I found. Seeing that it was a 'simple per frame auto white balance', I was wondering if there are mor complex/clever solutions out there. :)
here: https://github.com/IFeelBloated/AutoWB/blob/master/src.cxx, basically the same as that FrameEval stuff, but with temporal functionalities. writing vaporsynth filters is really easy, it's really no different from scripting in Python. you should consider writing a filter yourself next time you wanna add some basic extensions to an existing script/plugin.

[Selur]

Nice. Thanks! :)

[Myrsloik]

I think "id" should be a member of VSVideoFormat in API v4, otherwise it requires access to the core to determine if a clip has a constant format which is not very convenient

Only id = 0 mean variable format. This is strictly enforced by all the api functions. The reason the api function need the core pointer is to maintain V3 api compatibility where the id isn't a simple serialization so a V4 filter won't choke if it's fed a V3 id.
If it truly bothers you look at the macro used to create ids and parse them on your own, the only advantage the api gives you is that invalid combinations will be rejected instead of possibly passed on.

[feisty2]

I've unified the memory layout of VideoFormat and VideoInfo in my wrapper (which currently still relies on API v3) to match that of API v4:
https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Node.vxx#L109
https://github.com/IFeelBloated/vsFilterScript/blob/master/include/Metadata.vxx#L18
it should be fully compatible with API v4 at the source code level.

[stax76]

Guys, please up vote my request for direct VapourSynth support for x265:

https://github.com/msg7086/x265-Yuuki-Asuna/issues/11

[lansing]

I have started noticing this memory hungry behavior on Vapoursynth on preview editors like vseditor and virtualdub2. Whenever a script was loaded, Vapoursynth seems to allocate a big chunk of memory before any frame was even requested. For example, I have a script with 1080p video and QTGMC in vseditor, on script load the memory jumped straight to 4.5 GB of RAM, this is both unnecessary and unsustainable for multi tab editor, where it will be running out of memory for most people just by opening four or five similar scripts.

[edcrfv94]

Can filter once, multiple output to simultaneous x265 encoding? e.g. different filter stage or resolution 720p 1080p 2160p.

[feisty2]

yes, see: https://github.com/IFeelBloated/Oyster/blob/alpha/Alpha.py#L49

[stax76]

I'm not understanding why line 1+2 are needed here, isn't that exactly what altsearchpath=True in line 3 is supposed to do? Without line 1+2 it's not working.


import os.path
os.environ["PATH"] = r"C:\Anime4KCPP VapourSynth" + os.pathsep + os.environ["PATH"]
core.std.LoadPlugin(r"C:\Anime4KCPP VapourSynth\Anime4KCPP_VapourSynth.dll", altsearchpath=True)
clip = core.anime4kcpp.Anime4KCPP(clip, GPUMode = 1, ACNet = 1, zoomFactor = 2, HDN = 1, HDNLevel = 2)

[Are_]

Doesn't documentation say that's for dependencies?

[stax76]

I think I understand it now, what altsearchpath=True does is telling the LoadLibrary function to search dependencies in PATH.

[_Al_]

I have started noticing this memory hungry behavior on Vapoursynth on preview editors like vseditor and virtualdub2. Whenever a script was loaded, Vapoursynth seems to allocate a big chunk of memory before any frame was even requested. For example, I have a script with 1080p video and QTGMC in vseditor, on script load the memory jumped straight to 4.5 GB of RAM, this is both unnecessary and unsustainable for multi tab editor, where it will be running out of memory for most people just by opening four or five similar scripts.
Nowadays memory should not be an issue. Or you just limit it. For my Preview I use that below, first free memory is evaluated and if not enough, it is limited in vapoursynth. No more memory problems. Sure it might be not optimal, but on the other hand it never freezes, there is no other way if not enough memory, like using QTGMC for HD etc, and it is automatic, no more guessing or setting max cache size.


import vapoursynth as vs
from vapoursynth import core
try:
import psutil
except ImportError:
pass
import os

def freeRAM():
'''
getting free RAM
first it uses non standard library cross platform psutil
if modul is not installed, it falls back using Linux or Windows ways to get free RAM
so for Mac it needs psutil: to install psutil: pip3 install psutil
'''
available_RAM = None

#cross platform try if psutil is installed
try:
mem = psutil.virtual_memory()
available_RAM = int(mem.available/1024/1024)
if available_RAM and isinstance(available_RAM, int):
return available_RAM
except:
pass

#windows fallback
try:
proc = os.popen('wmic.exe OS get FreePhysicalMemory')
l = proc.readlines() #l should be a list
proc.close()
except:
pass
else:
for i, item in enumerate(l):
try:
available_RAM = int(l[i])
available_RAM = int(available_RAM/1024)
if available_RAM and isinstance(available_RAM, int):
return available_RAM
except:
pass

#linux fallback
try:
meminfo = dict((i.split()[0].rstrip(':'),int(i.split()[1])) for i in open('/proc/meminfo').readlines())
available_RAM = int(meminfo['MemAvailable']/1024)
if available_RAM and isinstance(available_RAM, int):
return available_RAM
except:
pass

#failed to get free RAM
return None

def limit_core_cache(core):
'''
Returns tuple (core, log).
Sets core.max_cache_size for less than available RAM,
log is string type ready for print.
'''

log = []
available = None
vs_cache = core.max_cache_size
log.append(f'vapoursynth cache is set to: {vs_cache}MB')
available_RAM = freeRAM()
if available_RAM is not None:
log.append(f'free RAM: {available_RAM}MB')
deduct = 0
if available_RAM < 200:
deduct = 50
log.append('almost no RAM, system likely to freeze')
elif 200 <= available_RAM < 400:
deduct = 120
log.append('not much RAM at all, freezing likely, lagish performance')
elif 400 <= available_RAM < 1024:
deduct = 220
log.append('more RAM would give better performance')
elif 1024 <= available_RAM < 1536:
deduct = 280
else:
deduct = 350
new_cache = max(50, available_RAM - deduct)
if new_cache < vs_cache:
log.append(f'setting Vapoursynth cache to: {new_cache}MB')
core.max_cache_size = new_cache
else:
log.append('\nWARNING, failed to get available free RAM,')
log.append(' Vapoursynth cache was not limited if needed,')
log.append(' RAM overrun or freeze possible\n')
return core, '\n'.join(log)
So it might be put as some extra module and just calling it:

import my_module
core, info = my_module.limit_core_cache(core)
if info: print(info) #or log or something

[lansing]

Nowadays memory should not be an issue. Or you just limit it. For my Preview I use that below, first free memory is evaluated and if not enough, it is limited in vapoursynth. No more memory problems. Sure it might be not optimal, but on the other hand it never freezes, there is no other way if not enough memory, like using QTGMC for HD etc, and it is automatic, no more guessing or setting max cache size.


I think for editor, the limiting of cache size should be frame based rather than memory based. The same QTGMC filter running in avspmod only takes 915 MB on playback because avs+(I think) only caches 3 frames.

[_Al_]

I'm no developer, grateful for I got available.
Some ideas, but sure they could be dead wrong, lacking proper developer background and experience:

-Saying 3 frames sounds like an arbitrary solution, because frame could be any size, SD, HD, 4k. Folks have 32GB nowadays. I have 20GB in "cheaper" laptop. 16GB extra for something like $30.

-Couple of lines above this, feisty2 is storing RAW video so script does not run for more things. We got 4k on the horizon that's the way I guess.

-I mentioned that before. Having more instances of script and switching between them. I think much better is to have clip instances and switch between them and so adjust workflows and habits to do so. Python literally wants you to. Tons of filters above some point, denoise, QTGMC is already rendered and some clips are forked at the bottom of the script, below, to have many filters absolutely the same. If nothing, then at least source plugin is the same. No need to calculate something 5x because having 5 scripts. To get rid of avisynth workflows might help. A red-neck-faulty-spaghetti-code preview I did, switching between clips, not scripts, is very fast comparing clips, instant during playback.

-Also Qt should remember frames, some decent number, depending on size, same as opencv so switching and comparing frames ones were rendered should be instant.

[feisty2]

if an installed logger is never manually removed by removeLogHandler(), will it be automatically removed when the core is freed?

[Myrsloik]

if an installed logger is never manually removed by removeLogHandler(), will it be automatically removed when the core is freed?

Yes yes yes!

[poisondeathray]

I think for editor, the limiting of cache size should be frame based rather than memory based. The same QTGMC filter running in avspmod only takes 915 MB on playback because avs+(I think) only caches 3 frames.

But _Al_ does have some good points...

It's often faster for me to preview in avspmod using VSImport() for me when using multiple tabs and navigating, even with the overhead. It's less sluggish and it seems more frames are held in memory

[Soliloquy]

I'm having troubles running portable python / vapoursynth on win10 x64, version 1909.
I got python 3.9.1 and placed it in a folder, then extracted vapoursynth r52 in the same folder. Lastly i exctracted vapoursynth editor and upon running, it simply prints "Failed to initialize VapourSynth environment!".
Not sure what to try next - googled the issue and didn't really find anything to try besides typing import sys; sys.executable and see what it outputs, which is 'E:\\Encoding\\vapoursynth\\python.exe'.
What do i try next?

[ChaosKing]

R52 needs Python 3.8 on windows.

[Soliloquy]

that did it, thanks man!

[unix]

Is there a direct VapourSynth support for x264?

[DJATOM]

If you can apply patch and compile, my patch is here (https://github.com/DJATOM/x264-aMod-patches/blob/master/06-vpy-input.diff).

[unix]

I think I will stuck with ffmpeg because I'm not familiar with compile x264 ^^"
anyways I found a topic and I think it could help me https://forum.doom9.org/showthread.php?t=148615

Thank you

[feisty2]

@Myrsloik
what are some of the other things I need to do to make my project the preferred C++ wrapper for VS over vsxx?

[Myrsloik]

@Myrsloik
what are some of the other things I need to do to make my project the preferred C++ wrapper for VS over vsxx?

Find 3 people who prefer it I guess? I don't use either so I have no opinion about which one is better...

[videoh]

We don't need no steenkin' wrappers. Imagine having a bug in your filter and having to go to feisty2 for support. :rolleyes:

[stax76]

But it's pythonic and C++20, don't you like it?

[videoh]

I'm sure it's wonderful and great for noobs.