Vapoursynth

[Myrsloik]

It uses std.Lut2 inside. With two 16 bit clips, that means there will be a table of 65536 * 65536 elements, which is over 4 billion. With 2 bytes per element, that requires 8 GiB of RAM and probably takes a while to initialise.

Actually that should be rejected and the memory allocation would fail first. But there's probably something weird going on in general.
Definitely sounds like a 16bit bug in one filter.

[Boulder]

But basically it doesn't make sense to try contrasharpening 16-bit clips?

[Are_]

Actually that should be rejected and the memory allocation would fail first. But there's probably something weird going on in general.
Definitely sounds like a 16bit bug in one filter.

Not actually, because that function tries to allocate the full table in memory before sending it to std.Lut2 (I think).

lut = []
for y in lut_range:
for x in lut_range:
lut.append(clamp(0, expr(x, y), vmax))
return self.core.std.Lut2(c1, c2, lut=lut, planes=planes)


@Boulder: That module is like super unmaintained, it's surprising it even loads. For contrasharpening try with the one in havsfunc, it uses std.Expr so it should work OK with 16 bit clips.

[feisty2]

Replace lut2 with expr
Edit: too slow :)

Edit2: lut stuff needs to initialize before running, and that would burn your poor machine up if the bit depth is too high, basically it works fine on low bit depth clips
Expr doesn't need to initialize, it executes the RPN on each pixel at runtime, it's the right tool to work around high bit depth pixel manipulations, it works even at 96 bits input (32 bits float input x3)

[Boulder]

@Boulder: That module is like super unmaintained, it's surprising it even loads. For contrasharpening try with the one in havsfunc, it uses std.Expr so it should work OK with 16 bit clips.Yes, it seems to work fine. Funny that there's at least three versions of contrasharpening out and I end up picking the wrong one :D

[feisty2]

That one's not wrong, just gotta take some more time...

[Myrsloik]

R28 test4 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r28-test4.exe). Now works with (and prefers) per user python installations. Expr has also had several big fixes (ternary operator, crashes and so on) so give it a try again. Log, pow and maybe exp is probably not working properly. Everything else should be working though so try it out.

On the positive size Expr can now have up to 26 inputs, x-z, a-w.

Expr correctness and speed comparisons welcome.

[nu774]

The following script works via VSScipt, but fails when directly executed from python3.5 command.
Using Python3.5 on windows, vapoursynth git latest. Both of 32bit/64bit result in the same.

Error:

Traceback (most recent call last):
File "Baloon.issue.vpy", line 5, in <module>
last = core.std.FrameEval(last, lambda n, c=last: c)
File "src\cython\vapoursynth.pyx", line 1360, in vapoursynth.Function.__call__ (build\temp.win32-3.5\Release\pyrex\vapoursynth.c:24919)
vapoursynth.Error: FrameEval: Internal environment id not set. Report this function wrapper creation error.

Script:

import vapoursynth as vs

core = vs.get_core()
last = core.lsmas.LibavSMASHSource('baloon-pops.mp4')
last = core.std.FrameEval(last, lambda n, c=last: c)
last.set_output()

[Are_]

Is current git supposed to compile with gcc? I was about to give new expr a try but no luck.

Loads of:
/var/tmp/portage/media-libs/vapoursynth-9999/work/vapoursynth-9999/src/core/jitasm.h:1935:11: error: expected ‘)’ before ‘const’
/var/tmp/portage/media-libs/vapoursynth-9999/work/vapoursynth-9999/src/core/jitasm.h:1935:10: error: expected ‘;’ at end of member declaration
void and(const Reg8& dst, const Imm8& imm) {AppendInstr(I_AND, 0x80, E_SPECIAL, Imm8(4), RW(dst), imm);}
^
/var/tmp/portage/media-libs/vapoursynth-9999/work/vapoursynth-9999/src/core/jitasm.h:1935:28: error: expected unqualified-id before ‘const’
void and(const Reg8& dst, const Imm8& imm) {AppendInstr(I_AND, 0x80, E_SPECIAL, Imm8(4), RW(dst), imm);}
^

EDIT: I didn't say anything, jackoneill fixed it while I was typing this.

[Myrsloik]

Is current git supposed to compile with gcc? I was about to give new expr a try but no luck.

Loads of:
/var/tmp/portage/media-libs/vapoursynth-9999/work/vapoursynth-9999/src/core/jitasm.h:1935:11: error: expected ‘)’ before ‘const’
/var/tmp/portage/media-libs/vapoursynth-9999/work/vapoursynth-9999/src/core/jitasm.h:1935:10: error: expected ‘;’ at end of member declaration
void and(const Reg8& dst, const Imm8& imm) {AppendInstr(I_AND, 0x80, E_SPECIAL, Imm8(4), RW(dst), imm);}
^
/var/tmp/portage/media-libs/vapoursynth-9999/work/vapoursynth-9999/src/core/jitasm.h:1935:28: error: expected unqualified-id before ‘const’
void and(const Reg8& dst, const Imm8& imm) {AppendInstr(I_AND, 0x80, E_SPECIAL, Imm8(4), RW(dst), imm);}
^

No, still working on the not windows part.

[Myrsloik]

Another test version. This one should have a fully working Expr with no bugs at all. It also fixes the issue nu774 reported that was introduced in the test versions.

This version is also compiled with some extra checks to see if any plugins trigger them. Report any that do.

R28 test5 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r28-test5.exe)

[Are_]

Some speed comparasions:

r27
finesharp-lut
Output 2001 frames in 29.26 seconds (68.38 fps)
Output 2001 frames in 30.64 seconds (65.31 fps)
Output 2001 frames in 32.25 seconds (62.05 fps)
Average fps for finesharp-lut: 65.24

finesharp
Output 2001 frames in 56.01 seconds (35.73 fps)
Output 2001 frames in 55.35 seconds (36.15 fps)
Output 2001 frames in 54.97 seconds (36.40 fps)
Average fps for finesharp: 36.09

psharpen
Output 2001 frames in 40.66 seconds (49.21 fps)
Output 2001 frames in 41.05 seconds (48.74 fps)
Output 2001 frames in 41.24 seconds (48.52 fps)
Average fps for psharpen: 48.82

git
finesharp
Output 2001 frames in 47.69 seconds (41.95 fps)
Output 2001 frames in 46.86 seconds (42.70 fps)
Output 2001 frames in 47.46 seconds (42.16 fps)
Average fps for finesharp: 42.27

psharpen
Output 2001 frames in 28.09 seconds (71.24 fps)
Output 2001 frames in 27.21 seconds (73.53 fps)
Output 2001 frames in 26.59 seconds (75.25 fps)
Average fps for psharpen: 73.34

[Myrsloik]

Interesting comparison. But where can I find psharpen for vs?

I think I can make it even faster by combining several expr calls. At least finesharp is kinda inefficient in that eay.

[Reel.Deel]

Interesting comparison. But where can I find psharpen for vs?

I think I can make it even faster by combining several expr calls. At least finesharp is kinda inefficient in that eay.

I'm not aware of psharpen for vs but here's the avs version: http://forum.doom9.org/showthread.php?t=172422

[Are_]

Sorry, I somehow f..... up the test, speed is a little better, results updated. psharpen.py (https://gist.github.com/4re/2545a281e3f17ba6ef82)

[foxyshadis]

I can'y get assvapour to do anything at all, and as far as I can tell I'm using it right:

import vapoursynth as vs
core = vs.get_core()
std = core.std

vid = core.lsmas.LWLibavSource(r'C:\train\karaoke\work\Rilo Kiley - Portions for Foxes.mkv')
subs = core.assvapour.AssRender(vid,r'Rilo Kiley - Portions for Foxes.ass (https://www.dropbox.com/s/ock9pb1ukz9v1qn/Rilo%20Kiley%20-%20Portions%20for%20Foxes.ass?dl=0)')
subs[0] = core.resize.Bicubic(subs[0], format=vid.format.id)
vid = std.MaskedMerge(vid, subs[0], subs[1])
vid.set_output()


I get pure black video. What it should look like (https://www.dropbox.com/s/kj9utyui52tlck5/Rilo%20Kiley%20-%20Portions%20for%20Foxes.mkv?dl=0), you can use this as input above. The subs are muxed and work fine in MPC and MPDN.

Edit: Also attempted
vsfilter = core.avs.LoadPlugin(r'vsfilter_avs.dll')
subs = vsfilter.TextSub(vid,r'Rilo Kiley - Portions for Foxes.ass')
and that was also a no-go, with error "Python exception: 'NoneType' object has no attribute 'TextSub'", although it worked perfectly from Avisynth.

[splinter98]

Edit: Also attempted

vsfilter = core.avs.LoadPlugin(r'vsfilter_avs.dll')
subs = vsfilter.TextSub(vid,r'Rilo Kiley - Portions for Foxes.ass')

and that was also a no-go, with error "Python exception: 'NoneType' object has no attribute 'TextSub'", although it worked perfectly from Avisynth.

That's because you're using avs Loadplugin incorrectly. Loadedplugins go into the core.avs namespace

Try: (untested)

core.avs.LoadPlugin(r'vsfilter_avs.dll')
subs = core.avs.vsfilter.TextSub(vid,r'Rilo Kiley - Portions for Foxes.ass')

Regarding AssRender not working, I get the same output on the latest build, however on an older build it works as expected.

[Myrsloik]

I can'y get assvapour to do anything at all, and as far as I can tell I'm using it right:

import vapoursynth as vs
core = vs.get_core()
std = core.std

vid = core.lsmas.LWLibavSource(r'C:\train\karaoke\work\Rilo Kiley - Portions for Foxes.mkv')
subs = core.assvapour.AssRender(vid,r'Rilo Kiley - Portions for Foxes.ass (https://www.dropbox.com/s/ock9pb1ukz9v1qn/Rilo%20Kiley%20-%20Portions%20for%20Foxes.ass?dl=0)')
subs[0] = core.resize.Bicubic(subs[0], format=vid.format.id)
vid = std.MaskedMerge(vid, subs[0], subs[1])
vid.set_output()


I get pure black video. What it should look like (https://www.dropbox.com/s/kj9utyui52tlck5/Rilo%20Kiley%20-%20Portions%20for%20Foxes.mkv?dl=0), you can use this as input above. The subs are muxed and work fine in MPC and MPDN.

Edit: Also attempted

and that was also a no-go, with error "Python exception: 'NoneType' object has no attribute 'TextSub'", although it worked perfectly from Avisynth.

Your assvapour script works perfectly for me with r28 test5.

[jackoneill]

Look at the output of Assvapour directly, both clips. How do they compare to the last working version?

[foxyshadis]

That's because you're using avs Loadplugin incorrectly. Loadedplugins go into the core.avs namespace

Thanks! I'm sure I knew this before, but I've been VS-only so long now that I screwed up the syntax. The VS docs are good, but this is one of the areas that isn't written up.

Your assvapour script works perfectly for me with r28 test5.

I'm cranky now because I tested with both r27 and r28t5 and neither showed. Now they do and I don't know why. Sigh. Computers, man.

[splinter98]

Look at the output of Assvapour directly, both clips. How do they compare to the last working version?

Hmm I can get it to work using the R27 assvapour. I use a custom build (which has minor changes to Subtitle which I keep meaning to create a pull request with) which don't work. Those custom builds use the latest version of libass (0.12.3) (exact library used here (http://repo.msys2.org/mingw/x86_64/mingw-w64-x86_64-libass-0.12.3-1-any.pkg.tar.xz)).

[feisty2]

so "^(pow)" is working in Expr now?

[splinter98]

Hmm I can get it to work using the R27 assvapour. I use a custom build (which has minor changes to Subtitle which I keep meaning to create a pull request with) which don't work. Those custom builds use the latest version of libass (0.12.3) (exact library used here (http://repo.msys2.org/mingw/x86_64/mingw-w64-x86_64-libass-0.12.3-1-any.pkg.tar.xz)).

Must have been my build of libass, built against the latest git and now working again :)

[jose1711]

fwiw even with the latest (today's) ffms2 i still get the same crash as here: http://forum.doom9.org/showthread.php?p=1733283#post1733283

6 of such videos in my small home video library. here: http://stackoverflow.com/questions/11507675/memcpy-ssse3-segmentation-fault a few guys suggest it may be due to misaligned memory, could it be this problem? just wondering.

[Khanattila]

It is probably a stupid question but, while compiling VapourSynth, which features can be disabled?
Besides '--disable-ocr'.

[jackoneill]

It is probably a stupid question but, while compiling VapourSynth, which features can be disabled?
Besides '--disable-ocr'.

The plugins are all optional. The average user will need all the other components.

If you have an application that only uses VapourSynth through the C API, you won't need VSScript, vspipe, or the Python module. If you want to use VapourSynth only through Python.exe or an application like vsedit, you won't need vspipe. That's about it.

[videoh]

@Myrsloik

It would be sweet if at your convenience you could add DGDecNV to your list of native plugins. Thank you.

[Myrsloik]

@Myrsloik

It would be sweet if at your convenience you could add DGDecNV to your list of native plugins. Thank you.

I will be there the next time I regenerate the docs. Which I usually only do when I release a new version.

Maybe I should have a small wiki as a complement to the static docs.

[videoh]

That's great, thanks.

"Maybe I should have a small wiki"

Something else to maintain! It might be easier to just update the lists somewhat more frequently, IMHO.

[Sparktank]

(+1 on the wiki. at the very least, a consolidated pile of examples)

[Myrsloik]

Here's R28 test6 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r28-test6.exe). The biggest change is the installer which will now automatically download and install the VS2013 and VS2015 runtimes if they're not present. It also has minor fixes here and there.

[vcmohan]

Is the imwri plugin in the autoload folder of included plugins correct version? When I make a call to imwri , it does not recognise. and an error message
Failed to evaluate the script:
Python exception: No attribute with the name imwri exists. Did you mistype a plugin namespace?
It works if I load plugin imwri from another folder where I had a another version of imwri. However it is about 20+ mb while the included is only 44kb. I remember once I downloaded a version of that size(44kb), but it had a additional folder having a number of dlls.

[vcmohan]

Where did you see the imwri plugin is included? IIRC the imwri plugin is never included in the official installer.

Here in the documentation (http://www.vapoursynth.com/doc/includedplugins.html)

[sneaker_ger]

Like HolyWu said it's not in the installer.
That should be read as "included in the source tree"
It's not included and never will be due to its insane size.

[vcmohan]

Like HolyWu said it's not in the installer.
All other plugins listed on that page are included. So it gives impression that imwri also is included. Documentation needs to be corrected then.

[Boulder]

Is there an equivalent in the core to Spline36Resize(2560,1440,src_left=0.25, src_top=0.25)?

[mawen1250]

For top-left aligned resizing:
fmtc.resample(clip, 2560, 1440, center=False)

[jackoneill]

All other plugins listed on that page are included. So it gives impression that imwri also is included. Documentation needs to be corrected then.

It has been corrected in git.

[Reel.Deel]

Maybe I should have a small wiki as a complement to the static docs.

A GitHub wiki would be nice, very easy to set up set up also.

edit: didn't notice you had already created one (https://github.com/vapoursynth/vapoursynth/wiki).

[stax76]

I'm also using the github wiki for StaxRip but can't really tell how it compares to other solutions. What could also be interesting is readthedocs.org, most people here know it from the excellent x265 docs, it looks like it's built with Python so a natural fit for VapourSynth.

https://readthedocs.org

https://x265.readthedocs.org

[Myrsloik]

I've now enabled the github wiki. Anyone with a github account can freely edit it. It's empty right now so suggest a good structure.

https://github.com/vapoursynth/vapoursynth/wiki

[stax76]

I suggests a Tools page like so:

Editors

VapourSynth Editor

GUIs

StaxRip
Simple x264/x265 Launcher

Encoders

QSVEncC
NVEncC
VCEEncC

Misc

vspipe
VirtualDub

[splinter98]

I'm also using the github wiki for StaxRip but can't really tell how it compares to other solutions. What could also be interesting is readthedocs.org, most people here know it from the excellent x265 docs, it looks like it's built with Python so a natural fit for VapourSynth.

https://readthedocs.org

https://x265.readthedocs.org

+1 for read the docs it should be a simple case of sign up, link to the github and enable the webhook and it will build on pull updates (and should support tag based version history which is useful for people using older versions for whatever reason). (it could also be set up as a cname for docs.vapoursynth.com if desired).

The wiki still useful for user contributed items, such as example snippets.

[foxyshadis]

Myrsloik, would you consider extending clip.format with min and max (or minval/maxval)? 2**clip.format.bits_per_sample-1 works for integer formats, though ugly, but that should be 1.0 for floats. Making it generic would be even uglier, like:
maxval=2**clip.format.bits_per_sample-1 if clip.format.sampletype==vs.INTEGER else 1.0

[foxyshadis]

I've made a new page for the community to list AviSynth equivalents to ease the transition to VS: https://github.com/vapoursynth/vapoursynth/wiki/Avisynth-Equivalents

Hopefully this will keep others from banging their head as much as I did on the path to enlightenment. Also helps me, since I'm so forgetful.

[Myrsloik]

I guess I could add a function for it but it's actually a bit more complicated. For floating point it also depends on the plane and format. I'll think about it a bit because it definitely could be useful to have in many scripts.

And when you write about function equivalents, don't always assume Lut is faster than expr. The jit compilation in r28 gives a huge speedup.

[foxyshadis]

I guess I could add a function for it but it's actually a bit more complicated. For floating point it also depends on the plane and format. I'll think about it a bit because it definitely could be useful to have in many scripts.

And when you write about function equivalents, don't always assume Lut is faster than expr. The jit compilation in r28 gives a huge speedup.

Didn't know that, nice. Even then I was going to edit it to say that it'd only be useful for 8-bit, since memory bandwidth on most luts is likely to be a much bigger speed-killer, unless it's a very expensive expression.

[DarkSpace]

For the integer formats, will your min/max values handle limited/full/custom ranges, too?

[Myrsloik]

For the integer formats, will your min/max values handle limited/full/custom ranges, too?

Don't go there. Limited/full range is a frame property so that's per frame. There's a reason I didn't add a simple function. It's not simple at all. Ever.

[Myrsloik]

Behold R28 RC1 (https://dl.dropboxusercontent.com/u/73468194/vapoursynth-r28-rc1.exe). No more changes planned unless a serious bug is found. Most of the performance regression relative to R27 should be fixed.

All R28 changes:
r28:
the installer will now download and install the vs2013 and vs2015 runtimes if needed
now uses python 3.5
fixed vdecimate parallel mode
pow is now accepted in expr expressions
expr now accepts up to 26 inputs (x-z, a-w)
expr now has runtime code generation on x86 cpus using jitasm
now normalizes the framerate returned from avisynth filters
it is now a fatal error to set the videoinfo of a filter to a non-normalized framerate
added a few more checks for proper api usage
get_core() can now be used in callbacks and other external functions in python
now returns an error message saying that windows needs to be updated in certain cases when plugin loading fails
fixed loop filter (nodame)
lut and lut2 can now output float as well
now accepts scripts that start with a BOM as well
fixed an image corruption bug with 9-16 bit input to rgvs when the c++ code is used
fixed division by zero issues in muldivrational in vshelper.h
blankclip can now create 0 (unknown/variable) fps clips
added float support to planedifference and planeaverage
added half support to addborders
relevant compile time options are now in the version string