VSRepo - the official easy plugin manager

[Myrsloik]

I have created a simple repository which can now basically install havsfunc and all its dependencies with one simple command. And a pile of other plugins and some scripts too now that I've gotten help creating more packages.

Latest version is now always bundled with VapourSynth. If you want to try out development versions or report bugs you can do it at Gihub (https://github.com/vapoursynth/vsrepo)


Usage:

vsrepo.py update
vsrepo.py install havsfunc ffms2 d2v
vsrepo.py upgrade-all
vsrepo.py installed
vsrepo.py available

You can refer to plugins/scripts in several ways. The id which is long and complicated and the namespace for plugins. For scripts it's generally the lowercase module name.

You can also add the -p switch to run it in portable mode and -f to force unknown versions of plugins to be upgraded.

Files will be installed to %APPDATA%\VapourSynth\PluginsXX and %APPDATA%\Python\X.Y\site-packages

(also known as the per user autoload directory and python's per user site directory)

All installed files will by default end up in the correct paths and instantly be usable. Make sure that the portable python directory is also the working directory when running in portable mode.

HELP CREATING MORE PLUGIN DEFINITIONS WELCOME. See the files in the local folder for examples. Report any issues.

[ChaosKing]

I see you changed the RainbowSmooth url from https://raw.githubusercontent.com/dubhater/vapoursynth-smoothuv/master/RainbowSmooth.py to https://github.com/dubhater/vapoursynth-smoothuv/archive/v2.zip. I tested it with the first one and it worked well. Any reason to prefer the bigger zip file?

[Myrsloik]

I see you changed the RainbowSmooth url from https://raw.githubusercontent.com/dubhater/vapoursynth-smoothuv/master/RainbowSmooth.py to https://github.com/dubhater/vapoursynth-smoothuv/archive/v2.zip. I tested it with the first one and it worked well. Any reason to prefer the bigger zip file?

Mostly that it points to a specific release/version. Pointing at master is a horrible idea that will make the hash fail as soon as a change is made to RainbowSmooth.py. If there's no release you have to at least pick a git revision and stick to that.

[ChaosKing]

Makes sense ... But I think it would be a good idea to offer both, a stable and master (latest/beta) version, since for scripts there aren't many official releases like in plugin repos. I think less then half of the scripts on github have official releases and even then, in 99% of the time, you just pick the master. At least this is my experience.

[Myrsloik]

Maybe I'll provide master/unstable labels at some point later. This is just the first version to get things started for the most common use cases. Even debugging a script with QTGMC caused greater buttpains for me just to go on the plugin/script treasure hunt for havsfunc.

[lansing]

Will it install both 32 bit and 64 bit of the plugin to each python directory respectively?

[Myrsloik]

Will it install both 32 bit and 64 bit of the plugin to each python directory respectively?

No, by default it installs the same bitness as the python you're running it with. You can control which one is installed with "-t win32" or "-t win64"

[lansing]

Shouldn't the Vapoursynth plugin path be inside "program files (x86)/vapoursynth"? Why is it pointing to %appdata% instead?

[Are_]

Just a wild guess
Files will be installed to %APPDATA%\VapourSynth\PluginsXX and %APPDATA%\Python\X.Y\site-packages

(also known as the per user autoload directory and python's per user site directory)

[foxyshadis]

Shouldn't the Vapoursynth plugin path be inside "program files (x86)/vapoursynth"? Why is it pointing to %appdata% instead?

In default use, it won't have permissions. I have mine in a custom folder with liberal permissions, but I'm pretty sure Myrsloik's intention is having a well-known folder with plugins and scripts for better support. Customize away if you support yourself!

[Myrsloik]

In default use, it won't have permissions. I have mine in a custom folder with liberal permissions, but I'm pretty sure Myrsloik's intention is having a well-known folder with plugins and scripts for better support. Customize away if you support yourself!

1. Permissions. Default global directories need administrative rights and that's a problem.
2. Why do you feel this need to stuff everything into program files where it shouldn't be?
3. It's a horrible idea to change state of plugins for all users in a true multi-user environment. Just saying.

[Myrsloik]

Updated version posted, now has 46 plugins and scripts in total. Feel free to request what should be added next or create the definitions yourself.

[Myrsloik]

Updated to test5 in the first post, it now has various small bugfixes and simple caching to make it work faster when many things are being installed at once.

About 10 more packages since yesterday too. Keep the submissions coming and we'll have a complete package system created in under one week!

(it's already quite complete so start using havsfunc and vsTAAmbk without having to go on a treasure hunt)

[Myrsloik]

The daily update. Now we're up to 67 plugins and scripts. I'm surprised there hasn't been more feedback yet. Is it too sunny outisde?

[lansing]

There can be a command to list all available plugins and their installation status, and there can also be a command to just install all plugins.

[Myrsloik]

There can be a command to list all available plugins and their installation status, and there can also be a command to just install all plugins.

Listing all available ones, sure. Installing all plugins is just ridiculous though.

[ChaosKing]

About 10 more packages since yesterday too. Keep the submissions coming and we'll have a complete package system created in under one week!


One Problem ist that not every git repo has also a release aka binaries, only source code...

I hope this motivates people to provide some binaries as well. :)

[Myrsloik]

One Problem ist that not every git repo has also a release aka binaries, only source code...

I hope this motivates people to provide some binaries as well. :)

Yes, it really annoys me when people can't be bothered to make releases and put binaries on github.

[lansing]

Installing all plugins is just ridiculous though.

Why is that? It gives an option to people who are new or people who don't really care about customization a hassle-free installation.

[Myrsloik]

Why is that? It gives an option to people who are new or people who don't really care about customization a hassle-free installation.

It's a waste of bandwidth and a recipe for potential problems. There's a reason "apt-get install-all" is not a thing in linux...

[ChaosKing]

Install havsfunc and you have downloaded 50% of all plugins :D

[Myrsloik]

There's no reason to install more or less obsolete plugins (nnedi3 replaced by znedi3, depan replaced by mvtools, and so on). Or cuda using plugins if you have intel/amd graphics.

[DJATOM]

Is znedi3 really safe? I mean on some sources results looks not that good as on legacy nnedi3, so I think it's fine to have both. Or that was fixed with the latest version?

[Myrsloik]

Is znedi3 really safe? I mean on some sources results looks not that good as on legacy nnedi3, so I think it's fine to have both. Or that was fixed with the latest version?

All bugs should be fixed, if not report one.

[Myrsloik]

Test7 linked in the first post. I've added the "available" command to list all known package names. This version can also identify a huge number of older releases since vsrupdate.py can now grab release information from the github api AND generate new entries.

The idea is that vsrepo.py will download the latest sources.json from someplace that's regularly updated. But that's coming in a later version. Keep contributing packages (if there's anything meaningful left not already there).

Does anyone actually care about uninstalling? I mean it'd be trivial to delete a dll or two but honestly, would anyone use it?

[DJATOM]

Pretty useless unless you need uninstalling feature for resolving conflicts or so.

[lansing]

I think uninstall still deserve a spot in there. For example if I'm to uninstall havsfunc, I would like to remove all its dependencies as well. I don't want to hunt them down one by one manually.

Playing around with the install command, the success message can be little more informative like stating how many dependencies have been install from the command and stating the location of the installation paths.

[Myrsloik]

I think uninstall still deserve a spot in there. For example if I'm to uninstall havsfunc, I would like to remove all its dependencies as well. I don't want to hunt them down one by one manually.

Playing around with the install command, the success message can be little more informative like stating how many dependencies have been install from the command and stating the location of the installation paths.

The problem with uninstalling all dependencies is that then I need to keep track how things happened, not just what they are now. One of the advantages of my backwards approach is that you can simply drop all your existing things into the per user autoload directory and they'll be managed.

Uninstall would only be for individual packages if added but it's easy to do so it'll be in the next test version. Maybe an info command to show the used paths would make sense too.

So the only change you want is showing the number of installed/updated packages?

[lansing]

The problem with uninstalling all dependencies is that then I need to keep track how things happened, not just what they are now. One of the advantages of my backwards approach is that you can simply drop all your existing things into the per user autoload directory and they'll be managed.

I'm thinking of some logic like when uninstalling a package, if a plugin was a dependency of another package, then don't uninstall it. There can also be a "force" option to uninstall it anyway. Then if anything had been removed accidentally, just run the "update" command and everything would be okay again.


So the only change you want is showing the number of installed/updated packages?
Yes, message like "1 package installed, 10 dependencies installed, 1 dependencies updated" at the end.

[lansing]

I tested installing a 32 bit znedi3 plugin and got this:


$ python vsrepo.py install znedi3 -t win32
Fetching: https://github.com/sekrit-twc/znedi3/releases/download/r1/znedi3_r1.7z
Successfully installed NNEDI3 Weights r1
No binaries available for win32 in package ZNEDI3, skipping installation
Package(s) installed

The nnedi3 weights bin shouldn't be installed since znedi3 doesn't have a 32 bit version.

[Myrsloik]

I tested installing a 32 bit znedi3 plugin and got this:


$ python vsrepo.py install znedi3 -t win32
Fetching: https://github.com/sekrit-twc/znedi3/releases/download/r1/znedi3_r1.7z
Successfully installed NNEDI3 Weights r1
No binaries available for win32 in package ZNEDI3, skipping installation
Package(s) installed

The nnedi3 weights bin shouldn't be installed since znedi3 doesn't have a 32 bit version.

Will be fixed in the next version

[Myrsloik]

Test8 posted. I guess I'll start experimenting with the final critical part next, an update command that fetches a recent package list.

Notable changes:
no longer installs dependencies for packages that themselves can't be installed because there are no relevant binaries
nicer summary of what was done at the end of an operation
list command renamed to available

[lansing]

There're still problem on the checking of dependencies compatibility:

$ python vsrepo.py -t win32 install oyster
Fetching: https://github.com/sekrit-twc/znedi3/releases/download/r1/znedi3_r1.7z
Successfully installed NNEDI3 Weights r1
Fetching: https://github.com/dubhater/vapoursynth-nnedi3/releases/download/v11/vapoursynth-nnedi3-v11-win32.7z
Successfully installed NNEDI3 v11
Fetching: https://github.com/Khanattila/KNLMeansCL/releases/download/v1.1.1/KNLMeansCL-v1.1.1.zip
Successfully installed KNLMeansCL v1.1.1
Fetching: ftp://ftp.fftw.org/pub/fftw/fftw-3.3.5-dll32.zip
Successfully installed FFTW3 Library 3.3.5
Fetching: https://github.com/HomeOfVapourSynthEvolution/VapourSynth-BM3D/releases/download/r7/BM3D-r7.7z
Successfully installed BM3D r7
Fetching: https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DFTTest/releases/download/r4/DFTTest-r4.7z
Successfully installed DFTTest r4
Fetching: https://github.com/EleonoreMizo/fmtconv/releases/download/r20/fmtconv-r20.zip
Successfully installed fmtconv r20
No binaries available for win32 in package MVTools (float), skipping installation
7 missing dependencies installed


This package shouldn't be install either because one of its dependencies (mvtools float) doesn't have a 32 bit version. Also it doesn't catch znedi3's incompatibility.

[Myrsloik]

There're still problem on the checking of dependencies compatibility:

$ python vsrepo.py -t win32 install oyster
Fetching: https://github.com/sekrit-twc/znedi3/releases/download/r1/znedi3_r1.7z
Successfully installed NNEDI3 Weights r1
Fetching: https://github.com/dubhater/vapoursynth-nnedi3/releases/download/v11/vapoursynth-nnedi3-v11-win32.7z
Successfully installed NNEDI3 v11
Fetching: https://github.com/Khanattila/KNLMeansCL/releases/download/v1.1.1/KNLMeansCL-v1.1.1.zip
Successfully installed KNLMeansCL v1.1.1
Fetching: ftp://ftp.fftw.org/pub/fftw/fftw-3.3.5-dll32.zip
Successfully installed FFTW3 Library 3.3.5
Fetching: https://github.com/HomeOfVapourSynthEvolution/VapourSynth-BM3D/releases/download/r7/BM3D-r7.7z
Successfully installed BM3D r7
Fetching: https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DFTTest/releases/download/r4/DFTTest-r4.7z
Successfully installed DFTTest r4
Fetching: https://github.com/EleonoreMizo/fmtconv/releases/download/r20/fmtconv-r20.zip
Successfully installed fmtconv r20
No binaries available for win32 in package MVTools (float), skipping installation
7 missing dependencies installed


This package shouldn't be install either because one of its dependencies (mvtools float) doesn't have a 32 bit version. Also it doesn't catch znedi3's incompatibility.

Actually it does work. I fetch nnedi3_weights.bin from the znedi3 release archive and nothing else so no problem there.

Oyster being installable is the result of dependencies not being treated as required when no binaries exist. Not really a big deal. Ff it bothers you a lot simply annoy whoever didn't realease 32bit binaries into doing so.

[lansing]

Actually it does work. I fetch nnedi3_weights.bin from the znedi3 release archive and nothing else so no problem there.

Oyster being installable is the result of dependencies not being treated as required when no binaries exist. Not really a big deal. Ff it bothers you a lot simply annoy whoever didn't realease 32bit binaries into doing so.

I don't use 32 bit anymore myself but I just feel like the error checkings should be more robust on different scenarios.

[Myrsloik]

I don't use 32 bit anymore myself but I just feel like the error checkings should be more robust on different scenarios.

I'll just leave that case for now as it'd require a huge number of changes and be of little use. I did fix another related edge case though when binaries exist but in an older release.

[Myrsloik]

Test9 posted. Now it's basically feature complete. I guess. Have fun.

Changes:
Added update command which fetches the latest package definitions
Added uninstall command
Mark upgradable packages in a clearer way
Install binaries from an older version if the most recent version doesn't have it

[lansing]

After I installed a bunch of packages for testing, I got an error about missing "libfftw3-3.dll" when opening vs editor. I think it has to do with the dfttest package. But I do have that file in my global plugin64 folder in program files.

[Myrsloik]

After I installed a bunch of packages for testing, I got an error about missing "libfftw3-3.dll" when opening vs editor. I think it has to do with the dfttest package. But I do have that file in my global plugin64 folder in program files.

Doh, I didn't add it to the fftw3 package since I thought everything uses the si gle precision version. Will fix.

[Myrsloik]

Test10 available. Even closer to done now.

Changes:
On upgrade the currently installed version is first uninstalled to avoid leftover files
upgrade all is now upgrade-all
Nice progress bars if tqdm (https://pypi.org/project/tqdm/) is installed
Use sha256 instead of sha1

Go test it some more, as usual there are a few more packages.

[ChaosKing]

Works great so far.
I think an info msg where plugins and scripts are being installed would be nice. (and to avoid questions like "where are my plugins" in the future)
Maybe even add an parameter for that? like vsrepo paths or vsrepo locations

It would be nice if I could specify a target folder in portable mode (-p). If possible for scripts and plugins separately. Currently scripts are downloaded in the root dir of vsrepo which is a bit annoying.

[Myrsloik]

Sure, I could add an override for the locations easily. Maybe listing the default paths would be good too.

I don't think there's anotjer guaranteed search path for portable python. Correct me if I'm wrong.

[ChaosKing]

You can specify search paths in python36._pth in portable python (default is only root folder & python3X.zip). But I actually want it also for my installed version since I use a custom scripts folder.

[Myrsloik]

Test11 posted

Changes:
Plugin and script install paths can be specified
There's now a paths command to show where things will be installed

[ChaosKing]

SangNom2 Plugin Ported for VapourSynth. Beat by James1201 on https://bitbucket.org/James1201/vapoursynth-sangnom/
I guess it should be changed to this git repo then. Currently this one is used https://github.com/HomeOfVapourSynthEvolution/VapourSynth-SangNomMod

[ChaosKing]

I just re-checked and noticed that "identifier": "com.mio.sangnom" is actually from SangNom from https://bitbucket.org/James1201/vapoursynth-sangnom and not from the github version (it has a different identifier!)
What would be the prefered way now? Replace/correct sangnom.json or create a new json file? what about the filename? You knew this day will come ;)

Here's the corrected file
{
"name": "SangNom",
"type": "Plugin",
"description": "SangNom is a single field deinterlacer using edge-directed interpolation but nowadays it's mainly used in anti-aliasing scripts.",
"website": "https://bitbucket.org/James1201/vapoursynth-sangnom",
"doom9": "https://forum.doom9.org/showthread.php?t=173752",
"category": "Deinterlacing",
"identifier": "com.mio.sangnom",
"namespace": "sangnom",
"releases": [
{
"version": "r40",
"win64": {
"url": "https://bitbucket.org/James1201/vapoursynth-sangnom/downloads/libsangnom%20r40%20win64.7z",
"files": [
"libsangnom r40 win64/gcc/libsangnom.dll"
],
"hash": {
"libsangnom.dll": "cd62d7e54f086e5752965072b07cc9e90b168517768472c76e754007763de9ae"
}
}
}
]
}

[Myrsloik]

Correct it with the most up to date version I guess.

[Myrsloik]

Test12 posted. Fixes some downloads failing if the total size isn't known.

Adds some more packages as usual.

[Myrsloik]

Don't forget to test the update functionality. I've updates the package list with all the recent additions.

We're also getting close to a final release. All I want to do is add the possibility to install files into subdirectories (a very stupid limitation of the current code).

[ChaosKing]

Just for fun... In avisynth world there is no identifier or namespace equivalent, correct? So if one would transform vsrepo to an avisynth package management (avsrepo), the only "unique" thing would be the filter name or filter filename!?

[Myrsloik]

Just for fun... In avisynth world there is no identifier or namespace equivalent, correct? So if one would transform vsrepo to an avisynth package management (avsrepo), the only "unique" thing would be the filter name or filter filename!?

Exactly, there's no unique identifier or namespace there. That's why I added it to avoid most of the mess. FILENAME IS NOT UNIQUE.

Btw, if your transformations are minor consider not forking the codebase. I mean all you have to do is basically change how the plugin and script paths are read from the registry...

[ChaosKing]

Btw, if your transformations are minor consider not forking the codebase. I mean all you have to do is basically change how the plugin and script paths are read from the registry...

I just tested it with masktools2 and it works very well. Also with custom paths -b -s no changes are necessary to the vsrepo codebase, like you said.

Theoretically Avisynth could be also updated via vsrepo easily.

[Myrsloik]

I just tested it with masktools2 and it works very well. Also with custom paths -b -s no changes are necessary to the vsrepo codebase, like you said.

Theoretically Avisynth could be also updated via vsrepo easily.

Sure, in theory it could work. However the other nice key part vsrupdate generally can't. Most Avisynth developers haven't heard of github or even version control. It'll be a lot messier and in the process you'll most likely end up having to host a pile of stuff yourself.

Anyway, not my problem anymore....

[Myrsloik]

RC1 released. See the first post.

Changes:
Files can now be installed into subdirectories, waifu2x added as a result of this
vsrupdate.py now does a lot more verification to ensure no typos are in the package files

Getting close to done. Update and test as usual. Maybe we can reach 100 packages before the final release... is there still stuff left to add anyway?

The only remaining enhancements are to fix a few corner cases and convenience things in vsrupdate so for all other purposes consider this version final.

[ChaosKing]

The models are "very old" in releases. These are up to date https://github.com/HomeOfVapourSynthEvolution/VapourSynth-Waifu2x-w2xc/tree/master/Waifu2x-w2xc/models

[Myrsloik]

The models are "very old" in releases. These are up to date https://github.com/HomeOfVapourSynthEvolution/VapourSynth-Waifu2x-w2xc/tree/master/Waifu2x-w2xc/models

Someone will have to package them and release. I don't care enough about that detail. Actually someone who stores so many floating point numbers as json text deserves "old" models.

[ChaosKing]

The vspackages.zip file on vapoursynth.com is broken

[Myrsloik]

The vspackages.zip file on vapoursynth.com is broken

Fixed now

[~ VEGETA ~]

can you list the available packages in a txt file so we can choose?

Also, make something like "install common" to get most used packages like mvtools, masktools, lsmash, ffms2, fmt,..etc all in one line.

[Myrsloik]

can you list the available packages in a txt file so we can choose?

Also, make something like "install common" to get most used packages like mvtools, masktools, lsmash, ffms2, fmt,..etc all in one line.

Read the help and use the "available" command to list them all. Common means different things but at this point simply installing havsfunc is pretty much that.

Or if you have another base package suggestion simply post the command to install it all at once.

[~ VEGETA ~]

i try to install a certain package like "mv" but it doesn't do a thing... cli doesn't change after my command "python vsrepo install mv" for so long. what is the problem?

I have python 3.6 and x64-bit and I am using your latest github script.

[Myrsloik]

i try to install a certain package like "mv" but it doesn't do a thing... cli doesn't change after my command "python vsrepo install mv" for so long. what is the problem?

I have python 3.6 and x64-bit and I am using your latest github script.

You probably need to male it vsrepo.PY for ot to work

[~ VEGETA ~]

Actually I put it .py but didn't write it properly here

[~ VEGETA ~]

Now works good but I couldn't find lsmas here. My python is 64 bit thus plugins32 in VS directory probably won't be used? so I need lsmas for x64?

[ChaosKing]

Now works good but I couldn't find lsmas here. My python is 64 bit thus plugins32 in VS directory probably won't be used? so I need lsmas for x64?

Yes, everything (except scripts) needs to be 64bit to work together.
lsmas is not available in vsrepo yet.

[Myrsloik]

Yes, everything (except scripts) needs to be 64bit to work together.
lsmas is not available in vsrepo yet.

Speaking of that. I wish someone would adopt lsmas and the orphaned scripts without proper github repos. I really can't go around adding random dropbox links to stuff.

This is especially easy to do for scripts and would only require minimal effort in case someone submits a pull request.

hint hint wink wink

[ChaosKing]

Are there many orphaned scripts? I could only find 1-2 ...

//Edit
But I guess every script would need its own git repo!? A collection wouldn't work with your git update logic, correct?

[Myrsloik]

Are there many orphaned scripts? I could only find 1-2 ...

Maybe only 1-2. That's still 1-2 too many.

[Myrsloik]

Are there many orphaned scripts? I could only find 1-2 ...

//Edit
But I guess every script would need its own git repo!? A collection wouldn't work with your git update logic, correct?

Yes, they'd need a separate repo. You could just lump all the orphaned scripts into one repo too. Basically it needs to show up on the github releases tab or it won't be picked up automatically. (I also think pastebin-like things are bad places to store things that should be permanent)

[lansing]

I got a "HTTP Error 404: Not Found" trying to install hnwvsfunc

[Wolfberry]

You should run the update script first (if it works). Since I only keep the latest release, it will fail if you try to get the older release.
Sorry for the inconvenience.

[Myrsloik]

I got a "HTTP Error 404: Not Found" trying to install hnwvsfunc

It's because wolfberry deletes old releases just to annoy people. Didn't have time to properly fix vsrupdate to handle it yet.

[Myrsloik]

I've updated the packages and fixed vsupdate so it doesn't barf when releases are removed.

Now only people who completely rename files between releases need to die:)

[lansing]

Does the update file have logic to update itself once vapoursynth got updated to use newer python version like 3.7?

[Myrsloik]

Does the update file have logic to update itself once vapoursynth got updated to use newer python version like 3.7?

The python version doesn't matter. The only quirk is that you need to install all scripts in the python site packages dir again or move them between the major python updates. No logic needed there.

[Yrosma]

I was trying to setup portable edition of vapoursynth and run into a missing dependecy after installing havsfunc using vsrepo.

In my test script I'm using Deblock_QED from havsfunc and got the error it was missing the namespace dctf.

I found it here:
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DCTFilter/releases

After adding this one it was working again.
Is there a reason why this one was left out and shouldn't I be using this one?
Or was this one just forgotten?

[Myrsloik]

I was trying to setup portable edition of vapoursynth and run into a missing dependecy after installing havsfunc using vsrepo.

In my test script I'm using Deblock_QED from havsfunc and got the error it was missing the namespace dctf.

I found it here:
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DCTFilter/releases

After adding this one it was working again.
Is there a reason why this one was left out and shouldn't I be using this one?
Or was this one just forgotten?

I think nobody got around to creating a package for it yet. Somebody *hint* *hint* will probably do it soon though since you pointed it out. I actually thought havsfunc had all its dependencies done already.

[Myrsloik]

And it's added.

[Myrsloik]

I've now updated the packages again. It took a little longer than usual since I had to add a published date to every release as well (in order to always properly preserve the release order when releases are deleted).

[Myrsloik]

I've released RC2. It performs slightly more input validation before starting operations and has a new package definitions.

The unsharpmask and xsharpen functions are also added as WarpSharpSupport (wss) for those of you who want to install it conveniently.

[hydra3333]

I must be doing something wrong,
C:\SOFTWARE\Vapoursynth-x64>.\python.exe .\vsrepo.py -p -t win64 update -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts
Local definitions already up to date: Sun, 09 Sep 2018 22:02:52 GMT

C:\SOFTWARE\Vapoursynth-x64>.\python.exe .\vsrepo.py -p -t win64 installed -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts
Name Namespace Installed Latest Identifier
+Bifrost bifrost Unknown v2.2 com.nodame.bifrost
Bilateral bilateral r3 r3 com.invalid.identifier.bilateral
BM3D bm3d r7 r7 com.vapoursynth.bm3d
+Cnr2 cnr2 Unknown v1 com.nodame.cnr2
CTMF ctfm r4 r4 com.holywu.ctmf
+D2V Source d2v Unknown v1.1 com.sources.d2vsource
DCTFilter dctf r2 r2 com.holywu.dctfilter
Deblock deblock r6 r6 com.holywu.deblock
DeLogo delogo v0.4 v0.4 in.7086.delogo
DFTTest dfttest r4 r4 com.holywu.dfttest
EEDI2 eedi2 r7 r7 com.holywu.eedi2
+FFmpegSource2 ffms2 Unknown 2.23 com.vapoursynth.ffms2
+FFTW3 Library fftw3_library Unknown 3.3.5 org.fftw.fftw3
+fmtconv fmtc Unknown r20 fmtconv
*AddGrain grain r5 r6 com.holywu.addgrain
*KNLMeansCL knlm v1.1.0 v1.1.1 com.Khanattila.KNLMeansCL
MVTools mv v20 v20 com.nodame.mvtools
MVTools (float) mvsf r9 r9 com.nodame.mvsf
+NNEDI3 nnedi3 Unknown v11 com.deinterlace.nnedi3
NNEDI3CL nnedi3cl r7 r7 com.holywu.nnedi3cl
NNEDI3 Weights nnedi3_weights r1 r1 com.deinterlace.nnedi3.weights
Retinex retinex r3 r3 com.invalid.identifier.retinex
TCanny tcanny r10 r10 com.holywu.tcanny
TDeintMod tdm r10 r10 com.holywu.tdeintmod
Tonemap tonemap R2 R2 com.ifb.tonemap
+vcfreq vcfreq Unknown 2016-05-12 in.vcmohan.freq
VagueDenoiser vd r2 r2 com.holywu.vaguedenoiser
VSFilter vsf r2 r2 com.holywu.vsfilter
+Waifu2x w2xc Unknown r7 com.holywu.waifu2x-w2xc
W3FDIF w3fdif r1 r1 com.holywu.w3fdif
+AWarpSharp2 warp Unknown v4 com.nodame.awarpsharp2
Yadifmod yadifmod r10 r10 com.holywu.yadifmod
ZNEDI3 znedi3 r1 r1 xxx.abc.znedi3

C:\SOFTWARE\Vapoursynth-x64>.\python.exe .\vsrepo.py -p -t win64 upgrade-all -f -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts
Fetching: https://github.com/dubhater/vapoursynth-bifrost/releases/download/v2.2/vapoursynth-bifrost-v2.2-win64.7z
Traceback (most recent call last):
File ".\vsrepo.py", line 455, in <module>
inst = upgrade_all_packages(args.force)
File ".\vsrepo.py", line 368, in upgrade_all_packages
res = upgrade_files(get_package_from_id(id, True))
File ".\vsrepo.py", line 346, in upgrade_files
install_files(p)
File ".\vsrepo.py", line 310, in install_files
result = subprocess.run([cmd7zip_path, "e", "-so", tfpath, fn_props[0]], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
File "subprocess.py", line 403, in run
File "subprocess.py", line 709, in __init__
File "subprocess.py", line 997, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified
When I extract portable 7zip to the folder C:\SOFTWARE\Vapoursynth-x64 I see a 7za.exe rather than a 7z.exe

Suggestions welcome.

[Myrsloik]

Sigh, just rename it to 7z.exe and see if it works. I didn't even consider 7zip having a portable version too...

[hydra3333]

thank you, that works ...

[hydra3333]

tried it today and got this message ...
.\python.exe .\vsrepo.py -p -t win64 -f -b vapoursynth64\plugins\dll-to-choose-from -s vapoursynth64\scripts upgrade-all
Fetching: http://rationalqm.us/hdr/DGHDRtoSDR_1.9.rar
Traceback (most recent call last):
File ".\vsrepo.py", line 455, in <module>
inst = upgrade_all_packages(args.force)
File ".\vsrepo.py", line 368, in upgrade_all_packages
res = upgrade_files(get_package_from_id(id, True))
File ".\vsrepo.py", line 346, in upgrade_files
install_files(p)
File ".\vsrepo.py", line 311, in install_files
result.check_returncode()
File "subprocess.py", line 369, in check_returncode
subprocess.CalledProcessError: Command '['7z.exe', 'e', '-so', 'C:\\Users\\u\\AppData\\Local\\Temp\\vsmez18y1v_', 'x64/DGHDRtoSDR.dll']' returned non-zero exit status 2.

Tried a 7z manually
7z.exe l DGHDRtoSDR_1.9.rar
7-Zip (a) 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30
Scanning the drive for archives:
1 file, 145886 bytes (143 KiB)

Listing archive: DGHDRtoSDR_1.9.rar

ERROR: DGHDRtoSDR_1.9.rar : Can not open the file as archive

The .rar opens OK with winzip.

Advice welcomed.

[ChaosKing]

Maybe you have an older 7z version? I tested this package and had no problems extracting it.

//edit:
hmm ok, 18.05 is the newest one. But why does it work for me...
I can also open and extract rar files with the 7z gui -> 7zFM.exe

//edit2
on the 7zip page rar is listed as supported
Try to reinstall 7zip I guess...

[hydra3333]

OK, I'll re-download try again. And the 32bit version too I suppose.
Out of interest, did you try the commandline version ?
7za.exe l DGHDRtoSDR_1.9.rar

edit: freshly downloaded the x32 and x64 standalone versions from https://www.7-zip.org/download.html (I never install it) and ...
E:\SOFTWARE-Win10\ZIP\7zip\x>7za.exe l DGHDRtoSDR_1.9.rar

7-Zip (a) 18.05 (x86) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30

Scanning the drive for archives:
1 file, 145886 bytes (143 KiB)

Listing archive: DGHDRtoSDR_1.9.rar

ERROR: DGHDRtoSDR_1.9.rar : Can not open the file as archive

Errors: 1

E:\SOFTWARE-Win10\ZIP\7zip\x>x64\7za.exe l DGHDRtoSDR_1.9.rar

7-Zip (a) 18.05 (x64) : Copyright (c) 1999-2018 Igor Pavlov : 2018-04-30

Scanning the drive for archives:
1 file, 145886 bytes (143 KiB)

Listing archive: DGHDRtoSDR_1.9.rar

ERROR: DGHDRtoSDR_1.9.rar : Can not open the file as archive

Errors: 1

Winzip definitely opens it OK. How strange.

[ChaosKing]

It seems that there is a difference between 7z.exe and 7za.exe. Listing also works with 7z.exe.
Use this https://www.7-zip.org/a/7z1805-x64.exe


https://superuser.com/questions/1105516/comparing-7z-exe-and-7za-exe

[hydra3333]

Ah, thanks for that and the superuser link.

edit: DG is considering .zip or something instead of .rar

[Myrsloik]

I fixed the two biggest issues but as usual there may be a typo or something so help me test it. Just grab vsrepo.py from git because I'm lazy.

The first change is that it no longer uninstalls a previous version of a package before the new version is downloaded AND has the hashes verified. So much harder to get your plugins and scripts into a broken state by accident.

The second change is that packages where the download fails are skipped and reported as failed at the end, that way scripts with huge dependency lists won't end up broken as often. Note that multiple failures to fetch the same file may happen if multiple other packages depend on it.

Btw, I think DG secretly updates things after release to fix minor bugs => hash mismatches

I don't think there's a good way to handle that and it still barfs an exception. Discuss this one and think of a better solution if you can.

[ChaosKing]

In my initial idea for a database / plugin manager I had planned to create a bot which checks gist, github, bitbucket and http links periodically for new releases.
Maybe a http last-modified header check could be implemented, so one can keep up with "secret" updates, deletes or new releases ... I don't think there is an easy solution for this but only if the author would follow certain steps.

I can add a cronjob in my vsdb databse to check for non github links and report it if something changes like a hash mismatch or delete.

//Edit:
Auto generating an updated json could be also implemented. => just check if ok and push to github, less work


OR

This was discussed in the github issue: A myplugin.json could be maintained by the author itself and vsrepo just grabs it.

[Myrsloik]

In my initial idea for a database / plugin manager I had planned to create a bot which checks gist, github, bitbucket and http links periodically for new releases.
Maybe a http last-modified header check could be implemented, so one can keep up with "secret" updates, deletes or new releases ... I don't think there is an easy solution for this but only if the author would follow certain steps.

I can add a cronjob in my vsdb databse to check for non github links and report it if something changes like a hash mismatch or delete.

//Edit:
Auto generating an updated json could be also implemented. => just check if ok and push to github, less work

I guess your idea to regularly scan for secretly changed packages is required. Don't feel like implementing it due to how ugly it is though. I'll think about it...

Automating the current thing is trivial. The problem is if files are added or removed in a package because then it needs manual help so now and then it'd push completely broken updates. Minimal human review once per week is probably a must.

[gonca]

Btw, I think DG secretly updates things after release to fix minor bugs => hash mismatches
Actually, I believe he slip streams minor fixes, and appends the slip stream number to the version number, but the number on the download from his web site does not change.
No secret, all very public and announced to the users by e-mail and in the binaries update thread

[ChaosKing]

I guess your idea to regularly scan for secretly changed packages is required. Don't feel like implementing it due to how ugly it is though. I'll think about it...

Automating the current thing is trivial. The problem is if files are added or removed in a package because then it needs manual help so now and then it'd push completely broken updates. Minimal human review once per week is probably a must.

I don't think there will be many plugins that add or remove files. In 99% it stays the same. I thing this would be the easiest way:

Lets say we have a http://asd.xd/superplugin.zip url and a superplugin.json
- implement a scraper / bot function that checks if a) all urls of superplugin are still valid and b) if one of the "versions" have a different "last-modified" date. (maybe the whole zip could be hashed?)
- if a zip/rar has changed, download it, re-hash all known files that are stored in superplugin.json AND list also all contents of the zip/rar AND print a warning if a file is missing. Maybe it is a good idea to store the previous files list somewhere for comparison so that any new files could also be detected.
=> result: human review is reduced to a minimum.

@gonca yes, but it is only obvious if one checks the forum. VSRepo can't do that :)

[Myrsloik]

I don't think there will be many plugins that add or remove files. In 99% it stays the same. I thing this would be the easiest way:

Lets say we have a http://asd.xd/superplugin.zip url and a superplugin.json
- implement a scraper / bot function that checks if a) all urls of superplugin are still valid and b) if one of the "versions" have a different "last-modified" date. (maybe the whole zip could be hashed?)
- if a zip/rar has changed, download it, re-hash all known files that are stored in superplugin.json AND list also all contents of the zip/rar AND print a warning if a file is missing. Maybe it is a good idea to store the previous files list somewhere for comparison so that any new files could also be detected.
=> result: human review is reduced to a minimum.

@gonca yes, but it is only obvious if one checks the forum. VSRepo can't do that :)

I'd say it's simply beyond the scope. Let's just go shit on people who don't use github. It's so much faster and easier. Shaming works.

[ChaosKing]

.\vsrepo.py -p install adjust
adjust already exists. Output text is "Nothing done". I think it showed something like "Already installed" before...

[Myrsloik]

.\vsrepo.py -p install adjust
adjust already exists. Output text is "Nothing done". I think it showed something like "Already installed" before...

I'm going to improve that text later.

[Myrsloik]

Here's an issue someone with far too much time can help with:
https://github.com/vapoursynth/vsrepo/issues/80

Too lazy to do it myself and it's not that important.

[ChaosKing]

Can't you just check for invalid (empty) releases with vsrupdate and then just add it to the ignore list? I mean isn't vsrupdate checking for all releases anyway, empty ones included?

[Myrsloik]

Can't you just check for invalid (empty) releases with vsrupdate and then just add it to the ignore list? I mean isn't vsrupdate checking for all releases anyway, empty ones included?

No. Empty entries can either mean that the release has no relevant binaries (for example see HolyWu's linux build system updates that do nothing for windows users) OR that the filename guessing algorithm failed to create a new entry. It needs to be manually checked to determine which one it is.

This issue about editing existing entries and doing it manually is probably faster than writing a script. Or about the same speed since it's a one time thing.

[ChaosKing]

There are only 2 plugins with empty release tags left.
https://github.com/IFeelBloated/RGSF/releases <- see my commit
https://github.com/HomeOfVapourSynthEvolution/VapourSynth-Waifu2x-w2xc/releases <- older models have some different filenames/file count. No one will install the older releses anyway... not worth the work.

[ChaosKing]

Is this a good time to bring up the namespace issue? :D

I want to add https://github.com/sekrit-twc/EdgeFixer to vsrepo but we already have https://github.com/MonoS/VS-ContinuityFixer with the same namespace edgefixer.json

[Myrsloik]

Is this a good time to bring up the namespace issue? :D

I want to add https://github.com/sekrit-twc/EdgeFixer to vsrepo but we already have https://github.com/MonoS/VS-ContinuityFixer with the same namespace edgefixer.json

We need to have a fight to the death!

Actually the MonoS version should go since it's an obsolete fork. So make a replacement package and it's all good.

[ChaosKing]

ContinuityFixer is dead, long live EdgeFixer!

[kypec]

Is this tool supposed to work only on Windows platform? Because trying to run it on Linux like this python3 vsrepo.py install hnwvsfunc ffms2 gives me only :confused: Traceback (most recent call last):
File "vsrepo.py", line 35, in <module>
import winreg
ModuleNotFoundError: No module named 'winreg'

[Myrsloik]

Is this tool supposed to work only on Windows platform? Because trying to run it on Linux like this python3 vsrepo.py install hnwvsfunc ffms2 gives me only :confused: Traceback (most recent call last):
File "vsrepo.py", line 35, in <module>
import winreg
ModuleNotFoundError: No module named 'winreg'


Yes, it's windows only.

[kypec]

Yes, it's windows only.

OK, and how is one supposed to install VS plugins on Linux then? ffms2 for instance, I couldn't find installation instructions neither on your VapourSynth page nor on FFMS github...

[StainlessS]

AutoLoading:- http://www.vapoursynth.com/doc/autoloading.html

More stuff:- Google "vapoursynth install plugins linux" (EDIT: Excluding the double quotes)
https://www.google.com/search?source=hp&ei=GpwOXJ2xOtCXsAf5ua7YCA&q=vapoursynth+install+plugins+linux&btnK=Google+Search&oq=vapoursynth+install+plugins+linux&gs_l=psy-ab.3...46.6438..6743...0.0..0.774.6279.1j12j5j1j2j2j1....2..0....1..gws-wiz.......0j0i131j0i10j0i30j0i22i30j0i22i10i30j33i22i29i30.GZB1zOcGccA

EDIT: And from D9 forum only:- Google "vapoursynth install plugins linux site:forum.doom9.org"
https://www.google.com/search?ei=c54OXN35CdGX1fAPn_Og4Ao&q=vapoursynth+install+plugins+linux+site%3Aforum.doom9.org&oq=vapoursynth+install+plugins+linux+site%3Aforum.doom9.org&gs_l=psy-ab.3..0i71l2.0.0..39740...0.0..0.0.0.......0......gws-wiz.qWv89ZQjA_c

EDIT: And from Vapoursynth.com only:- Google "vapoursynth install plugins linux site:vapoursynth.com"
https://www.google.com/search?ei=t50OXNiMNbuc1fAPqtaX6As&q=vapoursynth+install+plugins+linux+site%3Avapoursynth.com&oq=vapoursynth+install+plugins+linux+site%3Avapoursynth.com&gs_l=psy-ab.3...40613.40613..41226...0.0..0.98.98.1......0....1j2..gws-wiz.......0i71.whXPltYrUqY

EDIT: Enclose linux in above Google search in double quotes if eg post MUST HAVE linux in results posts.

[Are_]

Most probably somebody have them packaged for your distro, I know Arch, Ubuntu (and anybody that can use their packages) and Gentoo have them. If not you will have to compile and install them yourself.

[kypec]

Most probably somebody have them packaged for your distro, I know Arch, Ubuntu (and anybody that can use their packages) and Gentoo have them. If not you will have to compile and install them yourself.

Could you please post a link to Ubuntu PPA with such packages? :thanks:
I' mostly interested in recent build of ffms2 VapourSynth plugin. I am desperately trying to compile it myself from sources but can't get past configure step, even though I have built fairly recent ffmpeg following this thorough guide (https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu). The farthest I have been able to get was after specifying my local paths like this: kypec@acer:~/ffms2$ ./configure FFMPEG_CFLAGS="/home/kypec/ffmpeg_build/include" FFMPEG_LIBS="/home/kypec/ffmpeg_build/lib"
It seems that it fails with libswscale recognition, though the files reported as missing (https://pastebin.com/ZXsucmuV) are there as expected. I'll be extremely grateful for any help with building this plugin!

[VS_Fan]

Could you please post a link to Ubuntu PPA with such packages?

From the VS documentation (http://www.vapoursynth.com/doc/installation.html#linux-installation-from-packages): Ubuntu – PPA (https://launchpad.net/~djcj/+archive/ubuntu/vapoursynth)

vapoursynth-extra-plugins - 20180501-1~bionic package includes ffms2 2.23.1

darealshinji (a.k.a. djcj) has a repo (https://github.com/darealshinji/vapoursynth-plugins)that contains all the sources used for the package

[ChaosKing]

I think it would be nice if vsrepo or vsupdate would have a "search for dependencies for filter/script XY in repo" function. Because I actually forgot to look for dependencies in vs-scripts while replacing edgefixer. Luckily there weren't any, so nothing broke. With this one can quickly check if replacing a filter would break stuff.

[Myrsloik]

Can someone help with VMAF r3 models? That one just has so many files to list and I don't have that much time to do it mysef.

[ChaosKing]

I still should have my crappy models hash script somewhere that I used vor v1. Wait for my PR :)
I was thinking if a "hash this folder" or "hash everything inside this zip" (and create a release entry ofc) could be implemented in vsrupdate.
Something like vsrupdate -lazypack path/url -version 1.2.3 -remove-path "plugins64" -target vmaf_model

[Myrsloik]

I still should have my crappy models hash script somewhere that I used vor v1. Wait for my PR :)
I was thinking if a "hash this folder" or "hash everything inside this zip" (and create a release entry ofc) could be implemented in vsrupdate.
Something like vsrupdate -lazypack path/url -version 1.2.3 -remove-path "plugins64" -target vmaf_model

Yes, I also thought about implementing it. Then I saw the pile of actually paid work with deadlines I've got to do.

[ChaosKing]

Idea for some automation: compile it, zip it and upload to ftp via travis-ci.

[Pat357]

Upon upgrade-all using vsrepo, it places the 64bit TCanny in the 32 bit plugin directory.
Because there is no 32bit version for TCanny r12, I'd expect vsrepo not to touch the plugins32 directory.


{
"name": "TCanny",
"type": "VSPlugin",
"description": "Canny edge detection",
"website": "https://github.com/HomeOfVapourSynthEvolution/VapourSynth-TCanny",
"doom9": "https://forum.doom9.org/showthread.php?t=172158",
"category": "Other",
"identifier": "com.holywu.tcanny",
"namespace": "tcanny",
"github": "https://github.com/HomeOfVapourSynthEvolution/VapourSynth-TCanny",
"releases": [
{
"version": "r12",
"published": "2019-02-05T10:01:28Z",
"win32": {
"url": "https://github.com/HomeOfVapourSynthEvolution/VapourSynth-TCanny/releases/download/r12/TCanny-r12.7z",
"files": {
"TCanny.dll": [
"plugins64/TCanny.dll",
"ce9bb45071b326c6c8e626da2b94ea2b86fe2ee39a1be5bd14a065ac42547121"
]
}
},
"win64": {
"url": "https://github.com/HomeOfVapourSynthEvolution/VapourSynth-TCanny/releases/download/r12/TCanny-r12.7z",
"files": {
"TCanny.dll": [
"plugins64/TCanny.dll",
"ce9bb45071b326c6c8e626da2b94ea2b86fe2ee39a1be5bd14a065ac42547121"
]
}
}
},

[Myrsloik]

TCanny fixed. The script gets a bit confused when one type of binaries disappears sometimes.

[lansing]

vcmove is missing from the repo:

https://forum.doom9.org/showthread.php?t=171414

[ChaosKing]

It's there https://github.com/vapoursynth/vsrepo/blob/master/local/vcmove.json

[lansing]

It's there https://github.com/vapoursynth/vsrepo/blob/master/local/vcmove.json

Oh I saw it now, I must be looking at the "w" section instead of "v" on the listing

[Myrsloik]

I'd really appreciate it if someone could take the time to manually update the https://github.com/HomeOfVapourSynthEvolution/VapourSynth-Waifu2x-w2xc packages since it's a complicated one.

The best strategy is probably to keep both w2xc.json and w2xc_models.json and simply make them both point to the same file.

[ChaosKing]

done, see PR.

[Myrsloik]

done, see PR.

Nice. I guess the other small tasks I've got lying around would be to go through this rejected pull request and do things correctly. After that we're up to date again.

https://github.com/vapoursynth/vsrepo/pull/98

[MeteorRain]

https://github.com/HomeOfVapourSynthEvolution/VapourSynth-DeLogo
is deprecated in favor of
https://github.com/HomeOfAviSynthPlusEvolution/DelogoHD

DelogoHD supports both AviSynth+ and VapourSynth interfaces, and has SIMD optimizations and native HBD support from YUV420P8 up to YUV444P16.

[ChaosKing]

PR is ready. Should vsrepo show a warning if a package is deprecated? I mean should we add a new field like "deprecated": "deleogohd" to vsrepo. Would then show "Delogo is deprecated, better install/use DelogoHD". But it won't happen very often...

[Myrsloik]

I'll add some kind of deprecation warning then now that we actually have something to deprecate...

[Boulder]

Muvsfunc fetched by VSRepo is a very old version. Is there any way to make VSRepo use the one from github?

[Myrsloik]

Muvsfunc fetched by VSRepo is a very old version. Is there any way to make VSRepo use the one from github?

Just tell the maintainer to make an actual github release and it'll be picked up automatically.

[lansing]

There is a more up to date descale plugin from one of its fork

https://github.com/BluBb-mADe/vapoursynth-descale/releases

[Myrsloik]

There is a more up to date descale plugin from one of its fork

https://github.com/BluBb-mADe/vapoursynth-descale/releases

The original author is still around so it has to be renamed or I won't include it.

[Myrsloik]

We're missing l-smash source I think. No surprise considering how elusive the binaries and releases are but lots of people use it so it'd be nice to have.

[ChaosKing]

The only github repo with "vsrepo auto update compatibility" is this one https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works
I could make a PR, a package exists already https://github.com/theChaosCoder/avsrepo/blob/master/local/lsmash.json

p.s. yes, the binary supports both avs and vs

[Myrsloik]

The only github repo with "vsrepo auto update compatibility" is this one https://github.com/HomeOfAviSynthPlusEvolution/L-SMASH-Works
I could make a PR, a package exists already https://github.com/theChaosCoder/avsrepo/blob/master/local/lsmash.json

p.s. yes, the binary supports both avs and vs

That'd be nice. It's not ancient either so better than nothing.

[ChaosKing]

That'd be nice. It's not ancient either so better than nothing.

Arghh it was a different one. This one only supports avisynth :o

[ChaosKing]

I made a handy script to quickly check for new dependencies in VS-scripts. Maybe it can also be added to vsrupdate.
It downloads the latest script und searches for all known namespaces.

EDIT: maybe it would be also a good idea to track removed plugins (namespaces)?

print('Checking dependencies')

with open('vspackages.json', 'r', encoding='utf-8') as vsp:
packages = json.load(vsp)['packages']

for plugin in packages:
if (plugin['type'] == 'PyScript'):
os.system("vsrepo.py -p -d install " + plugin.get('modulename'))
print("\nDependencies for: ", plugin.get('modulename'))
print("------------------")

filename = list(plugin['releases'][0]['script']['files'].keys())[0]
with open(filename, 'r', encoding='utf-8') as module:
current_script = module.read()
deps = []
deps_new = []
for p in packages:
if (p['type'] == 'VSPlugin'):
search = "." + p['namespace'] + "." # search for core .dfttest. DFTest() and clip .dfttest. DFTest(). Seems to be reliable enough...
if search in current_script:
deps.append(p['identifier'])
if not p['identifier'] in plugin['dependencies']:
deps_new.append(p['identifier'])
print(p['identifier'])
print("NEW: ", deps_new)

[Myrsloik]

What do you even mean by "track removed plugins"? Either they're there or not...

[ChaosKing]

What do you even mean by "track removed plugins"? Either they're there or not...

If a script gets updated and does not use plugin xy anymore. There is no need to keep it as a dependency then? (it won't happen very often)

[Myrsloik]

If a script gets updated and does not use plugin xy anymore. There is no need to keep it as a dependency then? (it won't happen very often)

That's so rare doing it manually is fine. Just like adding additional dependencies manually generally is fine too since they're rarely added.

Since VSRepo uses unique strategies (keeps no state and tries to figure out the mess you've got) there's only so much you can with regards to tracking these no longer required plugins for removal.

[Myrsloik]

Package requests:
https://github.com/Irrational-Encoding-Wizardry/Vapoursynth-RemapFrames
and f3kdb

[ChaosKing]

Pushed remapframes.
There is no f3kdb repo with releases. But we have neo_f3kdb which is basically f3kdb + two more modes

[Myrsloik]

Pushed remapframes.
There is no f3kdb repo with releases. But we have neo_f3kdb which is basically f3kdb + two more modes

Right, I completely forgot that. Then it's pointless.

[ChaosKing]

What is the correct/preferred way again for package file names?

1# MyMoDulE.json <=> MyMoDulE.py
2# mymodule.json <=> MyMoDulE.py

Most script packages follows 1# but some also 2#.
I would like to correct the script packages to 1#

[ChaosKing]

What is the correct/preferred way again for package file names?

1# MyMoDulE.json <=> MyMoDulE.py
2# mymodule.json <=> MyMoDulE.py

Most script packages follows 1# but some also 2#.
I would like to correct the script packages to 1#

I noticed it because VSRepoGUI shows a link to every git package and some are not working bcs it is assumed that Module Name = File Name. Otherwise I don't know how to direct link as the url is case sensitive.

[Myrsloik]

What is the correct/preferred way again for package file names?

1# MyMoDulE.json <=> MyMoDulE.py
2# mymodule.json <=> MyMoDulE.py

Most script packages follows 1# but some also 2#.
I would like to correct the script packages to 1#

I noticed it because VSRepoGUI shows a link to every git package and some are not working bcs it is assumed that Module Name = File Name. Otherwise I don't know how to direct link as the url is case sensitive.

Simple rules:
For plugins it's always the namespace which is lowercase.
For scripts it's generally the lowercased main script filename. In case of collisions with a plugin name you append _script.

Maybe I accidentally merged something that doesn't follow this pattern in which case you're welcome to fix it.

[shader]

I successfully installed neo_f3kdb in VS 64bit portable on Win10. neo-f3kdb.dll is now in pluginsfolder.
But when I run "vsrepo.py installed" all installed plugins are listed except that one.

[ChaosKing]

Did you run your command with -p for portable?

vsrepo installed looks in %appdata%\VapourSynth\plugins64
vsrepo -p installed looks in plugins64

[shader]

@ChaosKing You got it!!!

In %appdata%\VapourSynth\plugins64 there was an old vspackages.json without neo f3kdb inside.
I just used -p for install and update, but forgot it when running installed.

[ChaosKing]

@Myrsloik You missed some packages in your last packages update (descale + neo_f3kdb). Has the updater not detected it?

[Myrsloik]

@Myrsloik You missed some packages in your last packages update (descale + neo_f3kdb). Has the updater not detected it?

They weren't properly detected and I didn't have time to manually poke around. I think there's some updater bug causing the descale failure too.

[ChaosKing]

Strange, because on my machine it was detected (and updated) properly, descale plugin and script.

[ChaosKing]

BTW I added support for auto updating scripts (without releases) in (a)vsrepo. It works, but not fully tested yet.
https://github.com/theChaosCoder/avsrepo/commit/191e2ae2984d53b53aec93a05fc817e8a1893b93

It adds only the newest commit of a script. Should I make a PR when I'm finished or is this too evil :devil:
The last commit on vsrepo was generated with this function.

[Myrsloik]

BTW I added support for auto updating scripts (without releases) in (a)vsrepo. It works, but not fully tested yet.
https://github.com/theChaosCoder/avsrepo/commit/191e2ae2984d53b53aec93a05fc817e8a1893b93

It adds only the newest commit of a script. Should I make a PR when I'm finished or is this too evil :devil:
The last commit on vsrepo was generated with this function.

It's too evil. Better to force script writers to make releases.

[shader]

I'm using VS portable for a long time now.

I wanted to run R49//Python 3.8 (python-3.8.2-embed-amd64.zip) on my Win10 64bit machine with installed Python 3.7 64bit.

When I run "python vsrepo.py -p available" in the R49 folder I get the error:
Traceback (most recent call last):
File "vsrepo.py", line 100, in <module>
os.makedirs(py_script_path, exist_ok=True)
File "os.py", line 223, in makedirs
FileNotFoundError: [WinError 3] Das System kann den angegebenen Pfad nicht finden: ''


The same call in my old R48 folder works fine.

Seems that the portable version needs some packages from outside?!? At least on Win64 machine.

Can anybody help me?

[ChaosKing]

Can confirm. The py_script_path var is empty.
It works if you use the full path for python like C:\Python38\python.exe vsrepo.py -p available

[Myrsloik]

I'm using VS portable for a long time now.

I wanted to run R49//Python 3.8 (python-3.8.2-embed-amd64.zip) on my Win10 64bit machine with installed Python 3.7 64bit.

When I run "python vsrepo.py -p available" in the R49 folder I get the error:
Traceback (most recent call last):
File "vsrepo.py", line 100, in <module>
os.makedirs(py_script_path, exist_ok=True)
File "os.py", line 223, in makedirs
FileNotFoundError: [WinError 3] Das System kann den angegebenen Pfad nicht finden: ''


The same call in my old R48 folder works fine.

Seems that the portable version needs some packages from outside?!? At least on Win64 machine.

Can anybody help me?

What about R50-RC1?

[ChaosKing]

I tested it with R50. But vsrepo did not change in R50 so it's the same err.

[Myrsloik]

I tested it with R50. But vsrepo did not change in R50 so it's the same err.

Will give it a try then.

[Myrsloik]

Download the latest https://github.com/vapoursynth/vsrepo/blob/master/vsrepo.py and give it a try. There were some minor path handling issues in portable mode.

[shader]

Works, thank you!

[shader]

Not serious, but anyway...


if (args.operation in ['install', 'upgrade', 'uninstall']) == ((args.package is None) or len(args.package) == 0):
print('Package argument only required for install, upgrade and uninstall operations')
exit(1)


leads to


Traceback (most recent call last):
File "vsrepo.py", line 74, in <module>
exit(1)
NameError: name 'exit' is not defined

[Myrsloik]

Not serious, but anyway...



leads to

Doh, fixed.

[ChaosKing]

Small issue in vsrepo ... I pushed some updates for descale_script https://github.com/vapoursynth/vsrepo/blob/master/local/descale_script.json
Because the script itself does not have changed (only the descale plugin) in r4, r5, r6, the hash is the same as r3.


vsrepo recognices it as r3 and not r6 and an update does not fix it (hash stays the same)

.\vsrepo.py installed
*Descale descale r3 r6

Adding r4-6 to the ignore list would be one option. But I think vsrepo should just read the releases from bottom to top to avoid this.
What do you think?

EDIT
No wait its a totally different issue

EDIT2
script
"name": "Descale Script",
"identifier": "descale",
"modulename": "descale",

plugin
"name": "Descale",
"identifier": "tegaf.asi.xe",
"namespace": "descale",

.\vsrepo.py upgrade descale
Package Descale Script not upgraded, latest version installed
Nothing done

Both names or identifier are "descale", that seems to be the problem.

[Myrsloik]

I'm a bit sleepy but the identifier should probably be changed to descale_script. I think that's what causes the problem. I'll investigate it further tomorrow and will probably add a better check for it too.

[Myrsloik]

Things are now adjusted. The identifier is changed to descale_script which makes more sense for duplicate names.

And for "duplicate" releases simply ignore the older duplicates. That's the easiest way. They're fortunately quite rare and mostly harmless anyway.

[ChaosKing]

Does someone get an error with vsrepo.py update or is it just me?
The url works in my browser... :/


vsrepo.py update
Traceback (most recent call last):
File "D:\Apps\VapourSynth\vsrepo\vsrepo.py", line 821, in <module>
update_package_definition('http://www.vapoursynth.com/vsrepo/vspackages.zip')
File "D:\Apps\VapourSynth\vsrepo\vsrepo.py", line 627, in update_package_definition
with urllib.request.urlopen(req_obj) as urlreq:
File "C:\Python38\lib\urllib\request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "C:\Python38\lib\urllib\request.py", line 531, in open
response = meth(req, response)
File "C:\Python38\lib\urllib\request.py", line 640, in http_response
response = self.parent.error(
File "C:\Python38\lib\urllib\request.py", line 569, in error
return self._call_chain(*args)
File "C:\Python38\lib\urllib\request.py", line 502, in _call_chain
result = func(*args)
File "C:\Python38\lib\urllib\request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

EDIT
Seems my browser used the cached version. In Edge I get a 404 wordpress page

EDIT2
Should be working again

[Cary Knoop]

Not sure if I should report this here but VSRepo refers to an old version ffm2.dll
(from 2016).

[ChaosKing]

We have basically 2 options here:
- The ffms2 authors make a new release
- Someone make a github repo with ffms2 releases so vsrepo can use these releases. (+ auto update)

It would be too much work to track down individuall web links + keep them up to date. Of course if someone wants to do it, make a pull request on github ;-)

[Emulgator]

To not doublepost, I will link to the appropriate thread.
https://forum.doom9.org/showthread.php?p=1933141#post1933141

[MysteryX]

Does vsrepo work under Linux? Where do I find it, and any issues or limitations to be aware of?

[ChaosKing]

It's a python download script / manager for DLLs. There is no Linux support. This could change if someone would offer compiled .so files. Not see it to happen anytime soon.

See first page for link...

[MysteryX]

So although VapourSynth works in Linux, all packages need to be custom-compiled, I'd have no vsrepo support, so everything would have to be done manually?

What work would need to be done to get proper Linux support?

[ChaosKing]

So although VapourSynth works in Linux, all packages need to be custom-compiled, I'd have no vsrepo support, so everything would have to be done manually?

What work would need to be done to get proper Linux support?

Yes, vsrepo was made only for windows since only windows binaries of plugins are provided.
Welcome to the world of linux package management.

Depending on which Distro you use, many plugins may already be compiled and packaged. See http://www.vapoursynth.com/doc/installation.html#linux-installation

I saw several repos on Github with ready to use compile/install scripts for most of the plugins.


Maybe if someone does the work and provides Linux binaries (and keeps them up tp date), then I'm sure linux support could be add to vsrepo.

Relevant github issue https://github.com/vapoursynth/vsrepo/issues/33

[Are_]

So although VapourSynth works in Linux, all packages need to be custom-compiled, I'd have no vsrepo support, so everything would have to be done manually?

What work would need to be done to get proper Linux support?

vsrepo is useless in Linux, most distros have the libs and plugins already packaged ready to consume with their package managers.

Also, are you going to provide a different build that will properly dynamically link with the system libraries of every distro? Or are you going to compile them statically?

If you are new to Linux my recommendation is to get an Arch-based distro and use the already packaged plugins.

[ChaosKing]

But the problem remains that not every distro has all plugins packaged. For the average joe compiling a plugin is very challenging task. I mean people already complain that clip.std.Crop() is too compilicated instead of just crop() :devil:

What would be so bad about statically linked plugins? It's not like these are some system libs and are always in use by the system.

[Are_]

No real problem for me, but packaging that is huge. And keep in mind that not all the plugins support static linking out of the box, so you are going to have to dirty your hands with it.

I was only suggesting that using rpm / deb distros is a bad idea if you want to use vapoursynth painlessly.

[MysteryX]

I have an Arch-based distro. So how do I install those plugins?

By the way, with all the excitement around Windows 11, I expect Linux to grow a lot in the coming 1 to 3 years.

Also with Wine/Proton allowing to play most Windows games on Linux, and with the Steam Deck. That's another huge push for Linux adoption. Vapoursynth is one of those few areas that remain lacky on Linux.

[quietvoid]

I have an Arch-based distro. So how do I install those plugins?

There are a lot of plugins/scripts on the AUR. Most of them are prefixed by vapoursynth-plugin.

[MysteryX]

Cool! That's actually pretty good Linux support; for Arch with AUR at least.

Linux and MacOS should be better documented on the main website IMO.

[MysteryX]

I've installed all the packages I needed from the AUR. Only BM3D_CUDA fails to compile, and... Akarin plugin (https://github.com/AkarinVS/vapoursynth-plugin) is the only one missing from the AUR.

BM3D_CUDA gives this


==> Starting build()...
-- The CXX compiler identification is GNU 11.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake/Modules/CMakeDetermineCUDACompiler.cmake:179 (message):
Failed to find nvcc.

Compiler requires the CUDA toolkit. Please set the CUDAToolkit_ROOT
variable.
Call Stack (most recent call first):
source/CMakeLists.txt:3 (project)

[MysteryX]

vsakarin got added, that was quick! Less than 12h
https://aur.archlinux.org/packages/vapoursynth-plugin-vsakarin-git/

[sl1pkn07]

I've installed all the packages I needed from the AUR. Only BM3D_CUDA fails to compile, and... Akarin plugin (https://github.com/AkarinVS/vapoursynth-plugin) is the only one missing from the AUR.

BM3D_CUDA gives this


==> Starting build()...
-- The CXX compiler identification is GNU 11.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /usr/share/cmake/Modules/CMakeDetermineCUDACompiler.cmake:179 (message):
Failed to find nvcc.

Compiler requires the CUDA toolkit. Please set the CUDAToolkit_ROOT
variable.
Call Stack (most recent call first):
source/CMakeLists.txt:3 (project)



try now

[MysteryX]

For BM3D_CUDA, it worked after rebooting. It needs a reboot for cuda package to be detected.

One annoying thing, however, is that it requires a 3GB dependency to compile, and that dependency remains there afterwards.

[sl1pkn07]

you not need reboot. just relogin your user (or run by hand `source /etc/ld.so.conf.d/cuda.conf`). the plugin needs cuda to run. so need keep for runing (for that is setted in depends=() ). arch not separe the package in modules like ubuntu/debian. package all cuda as only one package

and please use the package comments in [AUR] if you have a problem with the packages

greetings

[Tohno_Neil]

I usually manually install VapourSynth plugins and scripts,

and last time get bugs that I can't fix (haf.QTGMC),

until I use the vsrepo, :helpful: :thanks:


vsrepo.py install havsfunc ffms2 d2v -t win64 -b "C:\VapourSynth\plugins" -s "C:\Users\Administrator\AppData\Local\Programs\Python\Python39\Lib\site-packages"

vsrepo.py install Bifrost Bilateral "CAS - Contrast Adaptive Sharpening" Cnr2 Curve Damb DCTFilter Deblock DeLogo DFMDerainbow DFTTest degrainmedian DotKill fillborders FieldHint Histogram IT LSMASHSource MCDenoise "Miscfilters (obsolete)" ReadMpls MSmoosh muvsfunc MVTools "MVTools (float)" mvsfunc NRDB Oyster DeblockPP7 Retinex RemoveGrain Scxvid SmoothUV SSIQ TCanny TComb TDeintMod TIVTC TempLinearApproximate TNLMeans vctrans VagueDenoiser VFRToCFR VSFilter VSFilterMod vsTAAmbk AWarpSharp2 xvs Yadifmod ZNEDI3 -t win64 -b "C:\VapourSynth\plugins" -s "C:\Users\Administrator\AppData\Local\Programs\Python\Python39\Lib\site-packages"

[Selur]

Please add a comment to vsrepo on github that it's only for Windows. Thanks