Vapoursynth

[Myrsloik]

I would like to request a setup option (enabled by default for user setup) to add vspipe to the PATH environment variable on Windows if it's not already available. As far as I know modifying user PATH does not require elevated privileges, nor does it require a reboot.

Create an issue for it and maybe I'll do it.

[ChaosKing]

Hmm so installed R46 32+64bit in a vm, only per user installation. 32 first then 64. But only the 32bit version seems to work. (I also restarted the pc)
See here: https://i.imgur.com/QFUOxBu.png

What can I do to "debug" this further?

EDIT
vs 32 has no vsvfw.dll?
https://i.imgur.com/2m1WwyT.png

Re-install of VS64 didn't help.

[gonca]

I uninstalled Vapoursynth and Python completely.
Then installed Python 3.7.1 followed by Vapoursynth 64-R46
Worked fine
Then I upgraded Python to 3.7.3
This way you get to select per user or all users mode

[Myrsloik]

Hmm so installed R46 32+64bit in a vm, only per user installation. 32 first then 64. But only the 32bit version seems to work. (I also restarted the pc)
See here: https://i.imgur.com/QFUOxBu.png

What can I do to "debug" this further?

EDIT
vs 32 has no vsvfw.dll?
https://i.imgur.com/2m1WwyT.png

Re-install of VS64 didn't help.

I tested this in a clean VM and vsvfw.dll is installed properly.

Loading the module also worked fine in both cases. I have no idea what you're doing wrong.

[Myrsloik]

Import notice regarding installs:

PYTHON FROM THE MS STORE DOES NOT WORK

Installing the Python module with the "--user" option in pip ALSO DOES NOT WORK (for example if you want to have VapourSynth in multiple Python environments)

I think I'll finally look into the possibility of (optionally) installing python.

[Boulder]

Because of the "not installed for all users" problem, I uninstalled Python, reinstalled 3.7.3 for all users and then updated VS to R46 (64-bit). Now when I try to import a function, I just get an error "Python exception: No module named 'resamplehq'". In registry, the Python path seems ok and also all my .py files are in their usual location, in C:\Program Files\Python37\Lib\site-packages\vapoursynth.

EDIT: Looks like it's missing the __init__.py file from that Python directory?

[ChaosKing]

I tested this in a clean VM and vsvfw.dll is installed properly.

Loading the module also worked fine in both cases. I have no idea what you're doing wrong.

I tested it again. Fresh installation in a VM with windows Pro 1903 x64 (created by the MediaCreationTool1903.exe tool from ms)

Vapoursynth 32 didn't work after installing it as a per user installation.

See video here: https://www.dropbox.com/s/a2humc9d4lb7h3q/vs32err.mp4?dl=0

(watch with 1.5x speed ;-))

[Myrsloik]

I tested it again. Fresh installation in a VM with windows Pro 1903 x64 (created by the MediaCreationTool1903.exe tool from ms)

Vapoursynth 32 didn't work after installing it as a per user installation.

See video here: https://www.dropbox.com/s/a2humc9d4lb7h3q/vs32err.mp4?dl=0

(watch with 1.5x speed ;-))

I tried exactly the same thing on win10 1903 x64 home and it worked. I did install all updates first but that's the only thing that's possibly different.

Can you verify that you have the vs2019 (or possibly 2017) installed? Not having it could make it fail to load I guess.

[ChaosKing]

Yep that was it. After installing vcredist x86 2019 (14.21.xxx) it worked.

EDIT0:
And the runtimes are not shown in this installer bcs they can't be installed with user, rights!?



EDIT1
And I think there is a python installer bug or maybe I'm understanding it wrong. Vapoursynth detects it as a "per user installation" and not "for everyone".
https://i.imgur.com/IS3xRjF.png

EDIT2:
OK I can click on "Install for all users" in customize installation. So python launcher and python are two seperate things I guess!?
The thing is, on my PC it shows always the "for all users" option by default.

https://i.imgur.com/Hn4VZac.png

[Boulder]

Because of the "not installed for all users" problem, I uninstalled Python, reinstalled 3.7.3 for all users and then updated VS to R46 (64-bit). Now when I try to import a function, I just get an error "Python exception: No module named 'resamplehq'". In registry, the Python path seems ok and also all my .py files are in their usual location, in C:\Program Files\Python37\Lib\site-packages\vapoursynth.

EDIT: Looks like it's missing the __init__.py file from that Python directory?

Is there any way to make the 64-bit R46 installation work properly? R45 also seems to create a symlink to that Python directory and installs a .pyd file there as well. Installation of R46 didn't do either of those things.

[ChaosKing]

It seems that r46 just copies the dll instead of linking it. A reinstall does not help? You could try the diagnose function in vsrepogui to see which plugin folder is used and if it shows any problems.

[Myrsloik]

It seems that r46 just copies the dll instead of linking it. A reinstall does not help? You could try the diagnose function in vsrepogui to see which plugin folder is used and if it shows any problems.

I stopped linking because:
1. You need elevated privileges to create links
2. The Python bits are now installed as a normal package using pip and it doesn't really do links either

[Myrsloik]

Is there any way to make the 64-bit R46 installation work properly? R45 also seems to create a symlink to that Python directory and installs a .pyd file there as well. Installation of R46 didn't do either of those things.

You should never have put things in the vapoursynth subdir. That was reserved for VS and nothing else. Make your own directory if you want to do things that way or simply let VSRepo stuff everything into the right place...

[Boulder]

You should never have put things in the vapoursynth subdir. That was reserved for VS and nothing else. Make your own directory if you want to do things that way or simply let VSRepo stuff everything into the right place...

I think that once it was required to put your .py files containing your own VS functions under site-packages to get them to autoload so you could just import them in your script. It has worked perfectly fine until R46, hence the confusion.

[Selur]

Anyone maintaining some build script for all the plugins for Linux (Debian based systems)?

[ChaosKing]

There was a ppa by djcj but it seem to be gone. See here (and sublinks) https://github.com/vapoursynth/vapoursynth/issues/455

EDIT his git repo: https://github.com/darealshinji/vapoursynth-plugins

[Selur]

@ChaosKing: I know about djcjs repository problem is it doesn't get updated any more (even the repository is 'archived'), I spoke with him and he's trying to make a 'build' script for all the filters, but I was wondering if somebody already did that and may be shared it with others. :)

---
For those interested in the issue: https://github.com/darealshinji/scripts/tree/master/Hybrid (currently not 'finished' or 'well tested', but probably helpful)

[Selur]

Got an issue on Linux, regarding:

VSFilter https://github.com/HomeOfVapourSynthEvolution/VSFilter
VsFilterMod: https://github.com/sorayuki/VSFilterMod
xy-VSFilter: https://github.com/HomeOfVapourSynthEvolution/

all of them seem to be Windows only. Does anyone know how and if theses can be compiled on Linux?

Cu Selur

[Myrsloik]

Got an issue on Linux, regarding:

VSFilter https://github.com/HomeOfVapourSynthEvolution/VSFilter
VsFilterMod: https://github.com/sorayuki/VSFilterMod
xy-VSFilter: https://github.com/HomeOfVapourSynthEvolution/

all of them seem to be Windows only. Does anyone know how and if theses can be compiled on Linux?

Cu Selur

VSFilter will always be windows only due to how it's coded. Use Subtext which is included or any other libass based alternative (if there is one).

[Selur]

okay, thanks for the info.

[lansing]

Is there a max speed limit set in vapoursynth? I'm testing out my new cpu with source filter like ffms2 on my dvd source, and the max speed I got is 1800 fps, while the same ffms2 (https://forum.doom9.org/showthread.php?p=1879001#post1879001) in avisynth reaches 5200 fps.

[Myrsloik]

Is there a max speed limit set in vapoursynth? I'm testing out my new cpu with source filter like ffms2 on my dvd source, and the max speed I got is 1800 fps, while the same ffms2 (https://forum.doom9.org/showthread.php?p=1879001#post1879001) in avisynth reaches 5200 fps.

How did you test the speed?

[lansing]

How did you test the speed?

I open the script in vs editor and run the benchmark function, for avs+ I use avsmeter.

[Myrsloik]

I open the script in vs editor and run the benchmark function, for avs+ I use avsmeter.

"vspipe script.vpy ." <- try that

If it's still slow lower the number of threads. You're effectively creating a worst possible scenario if you throw a lot of threads at only one source filter with no processing.

[lansing]

"vspipe script.vpy ." <- try that

If it's still slow lower the number of threads. You're effectively creating a worst possible scenario if you throw a lot of threads at only one source filter with no processing.

Using vspipe gives 4200 fps, though it is still 25% slower than avs+.

Then I ran another test on a longer sd video (2 hours), avs+ got 4700 fps and vs got 4400 fps, the difference is much closer, while both took less than 25% cpu usage.

[ChaosKing]

Using different tools (avsmeter and vspipe) for performance comparison won't give you very precise result, since the mechanism for measuring the speed in each application is not the same. You can't really conclude your result unless the mechanism in each application is identical.

I can confirm it. It just tested via vspipe, vsedit, avsmeter and vdub2 with a 1h h264 DV res video.

vsedit is the "slowest" (always ~220fps slower). avsmeter always wins by a couple of sec (3-5sec) or 1586fps vs 1404fps in vspipe. This is also confirmed with a stopwatch in powershell.

In vdub2 on the other hand, avsiynth takes 58sec while vapoursynth is done in 54sec.

Conclusion: ffms2 is more or less equally fast in VS and AVS+ and the tools to measure the speed can have a significant impact on the speed and/or the results.

[Myrsloik]

Using vspipe gives 4200 fps, though it is still 25% slower than avs+.

Then I ran another test on a longer sd video (2 hours), avs+ got 4700 fps and vs got 4400 fps, the difference is much closer, while both took less than 25% cpu usage.

Go back a few years and you'll see similar questions to this. Anyway...

The 25% cpu usage is because you reach the multithreading limit of ffmpeg. This is a very artificial experiment.

The remaining speed difference is due to differences in how memory allocation works. You're basically benchmarking the respective memory pools in the most artificial way possible. This is an extremely artificial experiment.

Actually I think the ffmpeg binary would win this competition if you just find the right options. I'm generally not interested in benchmarks like these since people rarely even get the options right. See my first answer.

Go benchmark something meaningful next time.

[lansing]

Something weird I just noticed after more testings, in the beginning of my benchmarks, task manager shows all drives at 0% read/write activity while the cpu was running. But some 30 seconds in, the drive that contains the source video began to have some read activities for no reason. And this had effectively bottlenecked my speed. In my script here, it was running at 100+ fps with 100% cpu load for the first 30 seconds, then it had dropped to 70 fps when the drive started to have some 40% reading.


clip = core.ffms2.Source(file)
clip = core.dfttest.DFTTest(clip)

I have tested on both vs and avs+, same behavior.

Can anyone reproduce it?

[ChaosKing]

Something weird I just noticed after more testings, in the beginning of my benchmarks, task manager shows all drives at 0% read/write activity while the cpu was running. But some 30 seconds in, the drive that contains the source video began to have some read activities for no reason. And this had effectively bottlenecked my speed. In my script here, it was running at 100+ fps with 100% cpu load for the first 30 seconds, then it had dropped to 70 fps when the drive started to have some 40% reading.


clip = core.ffms2.Source(file)
clip = core.dfttest.DFTTest(clip)

I have tested on both vs and avs+, same behavior.

Can anyone reproduce it?

Sound like (os) caching...
Need more info: ram size, file size, ssd or hdd?

[lansing]

Sound like (os) caching...
Need more info: ram size, file size, ssd or hdd?

ram is 32GB, source video is 6.5G and I tested on both 5400rpm and 7200rpm hdd.

[littlepox]

Encoders typically encode the video in a percentage of that 4000 fps speed. If you are talking about difference between avs/vs to the actual encoding, it is becoming a basis point.

For a 1 hour encode you save not more than 10 seconds.

It might be the way avs/vs fetching and caching the data.
It might be the advantage that avs is C++ compiled while vs is dependent on Python, which in turn depends on the explainator.
It might be ... wait I'm wasting too much time than that 10 seconds.

VapourSynth is powerful for many reasons, but raw speed on absolutely simplest script doesn't fall into the list.

[Myrsloik]

R47 RC1

Test it so it's stable. Especially the makediff/mergediff/merge/maskedmerge filters since they were converted to intrinsics from asm.

64bits (https://www.dropbox.com/s/40yed0njm05lhvr/VapourSynth64-R47-RC1.exe?dl=1)
32bits (https://www.dropbox.com/s/gbs3t5kh4bfz8u9/VapourSynth32-R47-RC1.exe?dl=1)

Changes:
fixed a crash in vdecimate when both dryrun and clip2 is set (no1d)
updated zimg to 2.9.2 to fix a crash that would happen on certain invalid input combinations
improved message handler api and core info api
removed dependency on nasm
various installer improvements including a warning if the vs2019 runtimes aren't installed

[Myrsloik]

R47 has been released. The usual blog post here (http://www.vapoursynth.com/2019/08/r47-fixing-small-issues/).

[Boulder]

You should never have put things in the vapoursynth subdir. That was reserved for VS and nothing else. Make your own directory if you want to do things that way or simply let VSRepo stuff everything into the right place...
I think that once it was required to put your .py files containing your own VS functions under site-packages to get them to autoload so you could just import them in your script. It has worked perfectly fine until R46, hence the confusion.

So what is the proper way of doing things in R47?

[Myrsloik]

So what is the proper way of doing things in R47?

Use vsrepo to install things. Then it's always in the right place.

[_Al_]

I think that once it was required to put your .py files containing your own VS functions under site-packages to get them to autoload so you could just import them in your script. It has worked perfectly fine until R46, hence the confusion.
It should still work, it is python thing. Your .py file within site-packages of the same Python version you are running. Then it could be imported as module.
With new Python version installed , you move those files as well. So organize it well, always gather them in one directory, like site-packages/vapoursynth/.

I do not know where vsrepo puts those .py files, maybe somewhere else, into AppData.

[Boulder]

It should still work, it is python thing. Your .py file within site-packages of the same Python version you are running. Then it could be imported as module.
With new Python version installed , you move those files as well. So organize it well, always gather them in one directory, like site-packages/vapoursynth/.

I do not know where vsrepo puts those .py files, maybe somewhere else, into AppData.

I thought so as well, but it doesn't work. The exact same Python version installed (3.7.3) and Vapoursynth R45 works but R47 doesn't.

Using VSRepo is fine, but my own custom .py files need to be taken care of as well and the problem lies there.

[_Al_]

For example put just that line into your script:
import os
That would mean you can import standard library modules but not yours? Seems odd. Or it still pulls from somewhere else than you think.
Maybe domain conflict? For example you cannot name your module as subprocess.py, because there is already standard library modul named subprocess, I did it once and error message was not particularly clear about that.

[poisondeathray]

I thought so as well, but it doesn't work. The exact same Python version installed (3.7.3) and Vapoursynth R45 works but R47 doesn't.

Using VSRepo is fine, but my own custom .py files need to be taken care of as well and the problem lies there.

I just upgraded, and it complained something about python not being installed for all users

Short version - I reinstalled python for all users now, but it's in a different location now; so that's where you have to copy over the scripts to the new site-packages location . When you install for "all users" it's Program Files => Python37 => Lib => site-packages (at least on Windows , for x64)

[Boulder]

The only easy way to make it work was to use the folder which VSRepo uses (%appdata%\Roaming\Python\Python37\site-packages). I've had Python installed in that mentioned directory under Program Files for a long time now but I don't know why it stopped working now.

[poisondeathray]

The only easy way to make it work was to use the folder which VSRepo uses (%appdata%\Roaming\Python\Python37\site-packages). I've had Python installed in that mentioned directory under Program Files for a long time now but I don't know why it stopped working now.

At least you have a workaround . The site-packages in the Program Files seems works for me, but my old Vapoursynth Plugins64 directory changed to the AppData\Roaming\VapourSynth\plugins64 directory . Not sure why all the changes.

[Myrsloik]

At least you have a workaround . The site-packages in the Program Files seems works for me, but my old Vapoursynth Plugins64 directory changed to the AppData\Roaming\VapourSynth\plugins64 directory . Not sure why all the changes.

1. It never changed. There's always been a per user directory for plugins but people on doom9 hate the idea of not manually stuffing things into program files. That's still doable if you want to though.

2. Unprivileged installs. Try reading the changelog. Program files isn't writable for everyone.

[Txico]

Sorry to ask this here, I haven't seen it anywhere else. So ... HELP!

Where can I get Vapoursynth packages? Ubuntu PPAs is not active any more and doesn't seem to be any viable alternative...
I already tried to compile it myself, but after compiling from sources zimg and vapoursynth R47.1, after solving the Python environmental variable (In Ubuntu Python 3.7 is not in the same place), now I can't load the ffsm2 plug-in. I tried to download and compile it myself, but I don't know where to copy the plug-in, or which file it is ... ffmsindex is working from that sources after "make install", but don't know where the plug-in is or goes.
Something as simple as:

import vapoursynth as vs
core = vs.get_core()
video = core.ffms2.Source(source="Lol.mp4")
video.set_output()

returns me using vspipe: "AttributeError: No attribute with the name ffms2 exists. Did you mistype a plugin namespace?"

Something I forget in the vapoursynth compilation?

[Selur]

Have you tried explicitly loading the plugin?
core.std.LoadPlugin(path="path to ffms library")

[Txico]

Yep! That solved it!

So, Ubuntu is installing everything in the wrong places or Vapoursynth is looking always where it shouldn't. Grrrr!
And what about having a working Ubuntu PPA? That will had been so much easy, as it was before! Now I don't have a working vsedit ...

[Myrsloik]

Yep! That solved it!

So, Ubuntu is installing everything in the wrong places or Vapoursynth is looking always where it shouldn't. Grrrr!
And what about having a working Ubuntu PPA? That will had been so much easy, as it was before! Now I don't have a working vsedit ...

We're looking for people who will actually maintain linux packages and not just disappear. Applications welcome.

[george84]

Downloaded VapourSynth64-R47.exe double click and get Message Setup, Python 3.7 (64-bit) is installed for the current user only. ... . After clicking OK installation exits.

Uninstalled Python and Python Launcher and reinstalled it. Tried reinstall for All users as well as not. But still get same message and cannot install.

Running Windows10 with newest updates.

[poisondeathray]

Downloaded double click and get Message . After clicking OK installation exits.

Uninstalled Python and Python Launcher and reinstalled it. Tried reinstall for as well as not. But still get same message and cannot install.

Running Windows10 with newest updates.


Where did you get Python ? MS STORE ?

According to this


PYTHON FROM THE MS STORE DOES NOT WORK


https://forum.doom9.org/showthread.php?p=1878822#post1878822

[LoRd_MuldeR]

Downloaded double click and get Message . After clicking OK installation exits.

Uninstalled Python and Python Launcher and reinstalled it. Tried reinstall for as well as not. But still get same message and cannot install.

Running Windows10 with newest updates.

When installing Python via "official" installer, it is very important to select "Customize installation", not "Install now" ;)

Then, skip over the "Optional Features" page that comes next (you can keep defaults here) and, on the "Advanced Options" page, be sure to enable the "Install for all users" option.

If you installed Python before and missed to do this, then you'll have to uninstall first and then install again.

(Also be sure to download and use the "Windows x86-64 executable installer" of Python, if you intend to use VapourSynth64. The Python website tries to trick you into downloading the "x86" version)

[george84]

When installing Python via "official" installer, it is very important to select "Customize installation", not "Install now" ;)

Thank you. This worked.