Vapoursynth

[stax76]

@Myrsloik

I found a solution that should be fine for me, moving the cleanup code past the message loop, apparently it has something to do with the message system. Sorry for not finding it earlier and thanks for trying to help.

[Myrsloik]

@Myrsloik

I found a solution that should be fine for me, moving the cleanup code past the message loop, apparently it has something to do with the message system. Sorry for not finding it earlier and thanks for trying to help.

Definitely looks like some kind of memory corruption but I can't explain why it happens. A true mystery. It crashes in vpy_clearEnvironment() which gets called twice in a row (impossible) and I can't explain that at all.

[stax76]

Definitely looks like some kind of memory corruption but I can't explain why it happens. A true mystery. It crashes in vpy_clearEnvironment() which gets called twice in a row (impossible) and I can't explain that at all.

I've set few breakpoints and stepped through the code, turned out that WM_DESTROY is not only sent to the main window but also to the child windows so the code runs more than once... :o

https://docs.microsoft.com/en-us/windows/win32/winmsg/wm-destroy

[Myrsloik]

I've set few breakpoints and stepped through the code, turned out that WM_DESTROY is not only sent to the main window but also to the child windows so the code runs more than once... :o

https://docs.microsoft.com/en-us/windows/win32/winmsg/wm-destroy

So it was your fault after all...

Your code sample was helpful to the development of audio support anyway since it exposed another bug.

[lansing]

I trying out the vapousynth sdk to build a filter, I want to start off by testing out the examples in the sdk folder, but I don't know how to compile them into dll. Can I get some instruction?

[Myrsloik]

I trying out the vapousynth sdk to build a filter, I want to start off by testing out the examples in the sdk folder, but I don't know how to compile them into dll. Can I get some instruction?

From the VS2019 startup screen select "create a new project".

Then "windows desktop wizard" and when asked select dll as project type and check "empty project". Add the sdk source file and then add the location of vapoursynth.h to the include dirs (or simply stuff it in the same dir as the source code if you're lazy).

Something like that.

[lansing]

From the VS2019 startup screen select "create a new project".

Then "windows desktop wizard" and when asked select dll as project type and check "empty project". Add the sdk source file and then add the location of vapoursynth.h to the include dirs (or simply stuff it in the same dir as the source code if you're lazy).

Something like that.

For some reason the header files were not detected, got the same problem if I put them in the header folder.

https://i.imgur.com/NHrNBAE.jpg

[stax76]

In the project properties there is a VC++ Directory section where you can set include and lib directories.

[lansing]

In the project properties there is a VC++ Directory section where you can set include and lib directories.

Thanks it works

[Myrsloik]

Go try the builds with audio support (https://forum.doom9.org/showthread.php?t=177623).

[feisty2]

update on the syntactic sugar: https://github.com/IFeelBloated/vaporsynth-syntactic-sugar/blob/master/sugar.hpp

new function: View(y, x)
creates an offset view of the src plane centered on (y, x), you can also create a second/third/... order view on a view to manipulate the relative coordinates of, well, relative coordinates. high order views are useful to algorithms with nested search windows (eg. NLMeans, you got a sliding similarity window(s) inside a sliding search window(a)).

the previous box blur example

auto srcp = MakePlane<const float>(srcp8, width, height, Repeat);
auto dstp = MakePlane<float>(dstp8, width, height, Zero);

for (auto y : Range{ height })
for (auto x : Range{ width })
dstp[y][x] = (srcp[y-1][x-1] + srcp[y-1][x] + srcp[y-1][x+1] + srcp[y][x-1] + srcp[y][x] + srcp[y][x+1] + srcp[y+1][x-1] + srcp[y+1][x] + srcp[y+1][x+1]) / 9;


could now be further simplified to

auto srcp = MakePlane<const float>(srcp8, width, height, Repeat);
auto dstp = MakePlane<float>(dstp8, width, height, Zero);

for (auto y : Range{ height })
for (auto x : Range{ width }) {
auto center = srcp.View(y, x);
dstp[y][x] = (center[-1][-1] + center[-1][0] + center[-1][1] + center[0][-1] + center[0][0] + center[0][1] + center[1][-1] + center[1][0] + center[1][1]) / 9;
}

[feisty2]

@Myrsloik
can you somehow merge these helper functions into VSAPI so it would be easier for filter developers to code or prototype new filters? I don't wanna keep it homegrown.

[lansing]

I got a crash from the audio test build while running benchmark in vsedit. It works fine when I switch back to R48.


core.avs.LoadPlugin(vd_filter)
core.avs.LoadVirtualdubPlugin(neatvideo_vdf, "NeatVideo", 2)

clip = core.lsmas.LWLibavSource(src, prefer_hw=1)
rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.COMPATBGR32)

denoise_clip = core.avs.NeatVideo_2(rgb_clip, profilePath, presetPath)

denoise_clip.set_output()

[Myrsloik]

@Myrsloik
can you somehow merge these helper functions into VSAPI so it would be easier for filter developers to code or prototype new filters? I don't wanna keep it homegrown.

No, this is definitely not something the API should handle. Not that it'd be possible anyway to expose a huge lump of C++ templates in a header through a C API anyway...

[Myrsloik]

I got a crash from the audio test build while running benchmark in vsedit. It works fine when I switch back to R48.


core.avs.LoadPlugin(vd_filter)
core.avs.LoadVirtualdubPlugin(neatvideo_vdf, "NeatVideo", 2)

clip = core.lsmas.LWLibavSource(src, prefer_hw=1)
rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.COMPATBGR32)

denoise_clip = core.avs.NeatVideo_2(rgb_clip, profilePath, presetPath)

denoise_clip.set_output()


Is it reproducible with blankclip?

[feisty2]

No, this is definitely not something the API should handle. Not that it'd be possible anyway to expose a huge lump of C++ templates in a header through a C API anyway...

well I could wrap everything into a C++ wrapper and enable a python scripting kind of filter writing experience with various syntactic sugar in c++20, but there's already vsxx so I don't know...

[lansing]

Is it reproducible with blankclip?

Yes, I tested with QTGMC, also crashed.


import vapoursynth as vs
import havsfunc as haf
core = vs.get_core()

clip = core.std.BlankClip(width=1920, height=1080, format=vs.YUV420P8, length=10000)
clip = haf.QTGMC(clip, TFF=True)

clip.set_output()

I previewed the script, do a couple of seeks and it crashed. Every seek took about 1G of memory.

[feisty2]

@Myrsloik
can I store vsapi, in and out as global variables and for each filter in the plugin, refresh them in xxxCreate()? will there be some unknown conflict? I assume this should work since you can only invoke one filter in the plugin with each function call?

I haven't finished but basically, it's https://github.com/IFeelBloated/vaporsynth-syntactic-sugar/blob/master/gaussblur(example).cxx#L70

[Myrsloik]

@Myrsloik
can I store vsapi, in and out as global variables and for each filter in the plugin, refresh them in xxxCreate()? will there be some unknown conflict? I assume this should work since you can only invoke one filter in the plugin with each function call?

I haven't finished but basically, it's https://github.com/IFeelBloated/vaporsynth-syntactic-sugar/blob/master/gaussblur(example).cxx#L70

That won't work for several reasons:
1. You can get different vsapi pointers to different filters. This is how different api versions are handled.
2. Multithreaded filter creation is actually a thing. So you'll have races when using globals for in and out.

[feisty2]

well, then I guess the simplest solution is to make the global variables templated, so each filter has the access to its own copy

[feisty2]

can I request a frame once and get the same frame multiple times? I know it sounds weird but it has something to do with temporal padding.

[Myrsloik]

can I request a frame once and get the same frame multiple times? I know it sounds weird but it has something to do with temporal padding.

Sure, you simply have to free all the different references you get to the frame.

[feisty2]

I just finished wrapping the video access process, the "zero" mode for temporal padding is quiet a nuisance: https://github.com/IFeelBloated/vsFilterScript/blob/master/TemporalPaddingPolicies.hxx#L8 since it requires creating a new frame, which then requires a "core".
the temporal padding functions are called here: https://github.com/IFeelBloated/vsFilterScript/blob/master/Clip.hxx#L72 and the global "Core<Filter>" is obtained from xxxCreate(), I've tested it and it seemed to work properly, but I'm not sure, is there a potential problem here with the core stuff for zero padding at the temporal dimension?

Also can I store "frameCtx" in the video clip struct when requesting the frame so I can get rid of this useless parameter in GetFrame()? Do all frames share the same frameCtx or do I need a map to store the Ctx for each frame?

[Jukus]

How can crop be an odd number? (but with resize to even numbers, ofс)
There is a technically interlaced DVD, but visually progressive, 2 pixels above are a black stripe, do crop top 2 and resize to 480, but a video doesn't encoded, what should I do?

[Myrsloik]

I just finished wrapping the video access process, the "zero" mode for temporal padding is quiet a nuisance: https://github.com/IFeelBloated/vsFilterScript/blob/master/TemporalPaddingPolicies.hxx#L8 since it requires creating a new frame, which then requires a "core".
the temporal padding functions are called here: https://github.com/IFeelBloated/vsFilterScript/blob/master/Clip.hxx#L72 and the global "Core<Filter>" is obtained from xxxCreate(), I've tested it and it seemed to work properly, but I'm not sure, is there a potential problem here with the core stuff for zero padding at the temporal dimension?

Also can I store "frameCtx" in the video clip struct when requesting the frame so I can get rid of this useless parameter in GetFrame()? Do all frames share the same frameCtx or do I need a map to store the Ctx for each frame?

The point of frameCtx is that it's unique to each frame being processed. I didn't review the rest of your code.

[StainlessS]

How can crop be an odd number? (but with resize to even numbers, ofс)
Well if YV12 (420), there are half the number of chroma samples as there are luma samples (both X and Y dimension),
and you cant chop a chroma sample in half.
If Interlaced YV12, then can only crop vertical in multiples of 4. [Otherwise Interlacing destroyed]

[_Al_]

in Vapoursynth you cannot crop "within" subsampling,
if you have YUV420P8 , it means you can crop only even numbers in x and y
say you have YUV411P8, then you can crop only multiples of 4 in x but odd lines in y
..etc,
you can get those multiples like this:
import vapoursynth as vs
clip = vs.core.std.BlankClip(clip, format=vs.YUV420P8)
print(1 << clip.format.subsampling_h)
print(1 << clip.format.subsampling_w)
you'd get:
>>>2
>>>2
for format=vs.YUV411P8
you'd get:
>>>1
>>>4

anyway, but do not cut some lines and resize back to the same number, that is nasty

[Jukus]

Thanks for answers.

anyway, but do not cut some lines and resize back to the same number, that is nasty
You mean it's a hoax?
Some people think that the resolution of the image should necessarily be multiples of 8 or even 16.
There are also hardware players that do not understand the non-standard resolution, anamorph.

[_Al_]

There is copping scripts, tutorials, calculators included, etc., on web since era of digital video started, especially at the beginning everyone was cropping like they'd get candy each time they did that. And those scripts and advises are still out there.
About mod, take a major HD resolution dimension 1080 that does not even qualify for mod 16.
If there is a filter that needs mod 8 or 16 you go with it otherwise there is no reason to change video to get to that mod. What I meant with those 2 pixels black strips, just leave it there (if it is an top or bottom). Maybe the best advice is going with basics for DVD, you see black stripes on sides crop them , total 16 pixels left and right, then resize to 16:9 to 854x480. At this point if those black stripes bother you, you can crop them, but that I'd do only if having letterbox, not some 2 pixels.

[Jukus]

@_AI_
Likely, people look at x264 log and see there "magic blocks of pixels"
...
[libx264 @ 0x55565ecbfd80] mb I I16..4: 10.1% 74.9% 14.9%
[libx264 @ 0x55565ecbfd80] mb P I16..4: 2.2% 9.2% 1.0% P16..4: 52.0% 17.5% 13.3% 0.5% 0.2% skip: 4.0%
[libx264 @ 0x55565ecbfd80] mb B I16..4: 0.1% 0.4% 0.0% B16..8: 42.9% 5.1% 1.0% direct: 2.7% skip:47.8% L0:36.5% L1:46.9% BI:16.6%
[libx264 @ 0x55565ecbfd80] 8x8 transform intra:73.7% inter:70.4%
...
[libx264 @ 0x55565ecbfd80] i16 v,h,dc,p: 11% 6% 3% 80%
[libx264 @ 0x55565ecbfd80] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 10% 8% 4% 9% 15% 14% 16% 13% 13%
[libx264 @ 0x55565ecbfd80] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 7% 3% 8% 16% 15% 15% 12% 12%
[libx264 @ 0x55565ecbfd80] i8c dc,h,v,p: 30% 25% 19% 26%

[Jukus]

How to write an analog of that code for VS?

interp = nnedi3(field=0, qual=2)
deint = YadifMod(order=0, edeint=interp)
TFM(order=0, mode=3, clip2=deint, slow=2).TDecimate(hybrid=1)

[ChaosKing]

How to write an analog of that code for VS?

interp = nnedi3(field=0, qual=2)
deint = YadifMod(order=0, edeint=interp)
TFM(order=0, mode=3, clip2=deint, slow=2).TDecimate(hybrid=1)



clip = your source
interp = core.nnedi3.nnedi3(clip, field=0, qual=2)
deint = core.yadifmod.Yadifmod(clip, order=0, edeint=interp)
clip = clip.vivtc.VFM(order=0, mode=3, clip2=deint).vivtc.VDecimate()
There's no hybrid=1 parameter in VDecimate().

[stax76]

Hello Myrsloik, I've some questions...

Is there some documentation or sample code for DLL discovery and loading?

I've looked at the nvenc code and it is using Run-Time Dynamic Linking, I don't know if it relies on VS being in Path or if it searches the VS location in the registry and modifies the Path env var of the process.

vspipe appears to do Load-Time Dynamic Linking, discovery and manual loading is not necessary because the DLLs are located in the same directory.

Do all setup variants add VS to the Path env var and apps can rely on that?

Do all setup variants register the VS location in the registry? If so which key(s)?

I believe Load-Time Dynamic Linking is possible and preferred, I'm mostly unsure about path discovery.

[ChaosKing]

Do all setup variants add VS to the Path env var and apps can rely on that?

Do all setup variants register the VS location in the registry? If so which key(s)?
Yes, see https://github.com/vapoursynth/vapoursynth/blob/master/installer/vsinstaller.iss#L142

Since VS was split in global and user installation variants we need to check 4 reg paths:
CurrentUser / LocalMachine => "SOFTWARE\VapourSynth-32" and "SOFTWARE\VapourSynth"

[stax76]

Thanks, might somebody have done this before with C++ and Win32? I'm very clumsy with that...

[Myrsloik]

Hello Myrsloik, I've some questions...

Is there some documentation or sample code for DLL discovery and loading?

Do all setup variants add VS to the Path env var and apps can rely on that?

Do all setup variants register the VS location in the registry? If so which key(s)?

I believe Load-Time Dynamic Linking is possible and preferred, I'm mostly unsure about path discovery.

Is there some documentation or sample code for DLL discovery and loading?
No. You read the location you need from the registry. HKLM/HKCU (depends on current user install mode) and then software\vapoursynth (or vapoursynth-32 for poor people) and the keys of interest are called VSScriptDLL and VapourSynthDLL. Simply use loadlibrary and call vsscript_getVSApi2() or getVapourSynthAPI() depending on which dll you use.

Do all setup variants add VS to the Path env var and apps can rely on that?
No. It's a user option and you can never depend on this.

Do all setup variants register the VS location in the registry? If so which key(s)?
Yes, see above. I guess portable doesn't but that's hardly a setup variant...

[stax76]

Made some simple wrapper library for AviSynth and VapourSynth, it was necessary because AviSynth has no C or COM interface...

https://github.com/staxrip/staxrip/tree/master/FrameServer

https://github.com/staxrip/staxrip/blob/master/General/FrameServer.vb

https://github.com/staxrip/staxrip/blob/master/General/VideoRenderer.vb

[amichaelt]

it was necessary because AviSynth has no C or COM interface...

What about avisynth_c.h?

[stax76]

I don't know, maybe I should try it, sample code would help. When I started the library I didn't know a lot about C/C++ (last project more than 10 years ago) and AviSynth, after completing it I know a little more. With the AviSynth part I had few issues, no problems with VapourSynth.

[DJATOM]

The way you're using frame fetching in VS is inefficient. In general you have to care about async frame requests on your end, otherwise it works like a single-threaded frameserver.

[Myrsloik]

What about avisynth_c.h?

Using the avisynth C api is really iffy. If all you want is to evaluate a script and get frames out then writing your own C++ wrapper dll will be so much more reliable and also faster. Believe me, I've tried both ways long ago when developing yatta.

VapourSynth has a very straightforward C-api in comparison.

[_Al_]

If trying to modify props and clip resolution is ridiculously small, it crashes, where no recovery is possible, it locks a PC.

rgb = core.std.BlankClip(width=30, height=16, format=vs.RGB24)
rgb2 = rgb.std.PlaneStats(prop='PlaneStats')
PROP_NAME = f'New_string_name_here'
def copy_prop(n,f):
f_out = f[0].copy()
f_out.props[PROP_NAME] = '{:.1f}'.format(f[1].props['PlaneStatsAverage']*100)
return f_out
rgb = core.std.ModifyFrame(rgb, [rgb, rgb2], copy_prop)

[stax76]

The way you're using frame fetching in VS is inefficient. In general you have to care about async frame requests on your end, otherwise it works like a single-threaded frameserver.

For a GUI like staxrip it's totally fine, it's not used for encoding but only to access parameters and for a basic crop and preview dialog that doesn't support playback and HDR color handling but has mpv/mpc integration to take care of this. The GDI based WinForms rendering was ridiculously slow but it was addressed with a hardware accelerated Direct2D replacement. It does a vertical flip and converts to RGB32 in software via avs/vs. For best possible performance vertical flip and YV12 could be handled by Direct3D and like you told frames could be accessed asynchronously.

[Myrsloik]

If trying to modify props and clip resolution is ridiculously small, it crashes, where no recovery is possible, it locks a PC.

rgb = core.std.BlankClip(width=30, height=16, format=vs.RGB24)
rgb2 = rgb.std.PlaneStats(prop='PlaneStats')
PROP_NAME = f'New_string_name_here'
def copy_prop(n,f):
f_out = f[0].copy()
f_out.props[PROP_NAME] = '{:.1f}'.format(f[1].props['PlaneStatsAverage']*100)
return f_out
rgb = core.std.ModifyFrame(rgb, [rgb, rgb2], copy_prop)

I can't reproduce this and need more information. CPU? OS? Number of threads? Does it happen every time?

[Myrsloik]

R49-RC1 is now available (https://github.com/vapoursynth/vapoursynth/releases/tag/R49-RC1). All maintenance this time since I'm slowly working on the separate audio build.

Changes:
r49:
updated to python 3.8 on windows
updated visual studio 2019 runtime version
updated zimg and added support for spline64 resize method
fixed a savestring bug in avscompat (sekrit-twc)
interleave, selectevery and separate fields now have a modify_duration argument to determine if they modify frame durations and fps
addborders and crop now update the _fieldbased attribute properly when an odd number of lines are cut from the top
fixed add to path not working for single user installs
fixed compilation on non-x86 systems
fixed an infinite loop in the expr filter optimizer that was introduced in r48 (sekrit-twc)

[lansing]

I don't know if this is a bug from vapoursynth or KNLMeansCL. KNLMeansCL requires input clip to be YUV444P10, but it still complains about it when I converted my clip to YUV444P10.


clip = core.resize.Bicubic(clip, format=vs.YUV444P10)
clip = core.knlm.KNLMeansCL(clip)


But then it works when I convert it to YUV444P8

[Selur]

@lansing: read https://github.com/Khanattila/KNLMeansCL/issues/42

[lansing]

@lansing: read https://github.com/Khanattila/KNLMeansCL/issues/42

Thanks, I'll follow up over there.

[Myrsloik]

R49-RC2 (https://github.com/vapoursynth/vapoursynth/releases/tag/R49-RC2) is out. Should be the final RC.

r49:
updated to python 3.8 on windows
updated visual studio 2019 runtime version
updated zimg and added support for spline64 resize method
fixed transfer characteristics not being applied to gray format clips (sekrit-twc)
fixed vdecimate bugs when compiled on systems where char is unsigned by default
fixed a regression introduced in r48 in that could sometimes cause corrupt output from expr on cpus without sse4.1 (sekrit-twc)
fixed a savestring bug in avscompat (sekrit-twc)
interleave, selectevery and separate fields now have a modify_duration argument to determine if they modify frame durations and fps
addborders and crop now update the _fieldbased attribute properly when an odd number of lines are cut from the top
fixed add to path not working for single user installs
fixed compilation on non-x86 systems
fixed an infinite loop in the expr filter optimizer that was introduced in r48 (sekrit-twc)

[poisondeathray]

Can 2 python versions 3.x coexist ? 3.7.x and 3.8.x ? I ask because I need 3.7 for some other projects but would like to test new version of vapoursynth, and I'm assuming 3.8 is a requirement for r49-rc2 ?