Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th February 2013, 02:18   #1  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 462
Avisynth / Vapoursynth - Package Management System

Hi,

I've been thinking lately about how cool it would be, if Avisynth/Vapoursynth would have its own package management system.
Or more precisely a downloader which also downloads all necessary dependencies for you. Like apt-get etc, but far more simple like the downloader in megui, only better

I already done some testing and my current "solution" would look like this:
- A package.json file (structure would be similar to package.json files from https://npmjs.org/). This file holds all information about the author, maintaner, dependencies...
Code:
An example {
    "filtername": "flash3kyuu_deband",  // I like your filter :)
    "version": "1.5",
    "category": "Restoration Filters",
    "sub-category": "Debanding",
    "filetype": "dll",
    "platform": "x86",
    "usesGPU": "false",
    "description": "This avisynth plugin debands the video by replacing banded pixels with average value of referenced pixels, and optionally add grain (random dithering) to them.",
    "mediawiki": "http://avisynth.org/mediawiki/flash3kyuu_deband",
    "homepage": "http://forum.doom9.org/showthread.php?t=161411",
    "doomX-link": "http://forum.doom9.org/showthread.php?t=161411",
    "keywords": [],
    "plugin-author": {
        "name": "SAPikachu",
        "email": "idk@somefantasymail.com",
        "url": "http://www.google.com"
    },
    "package-maintainer": {
        "name": "ChaosKing",
        "email": "thechaoscoder@gmail.com",
        "url": "http://chaosking.de/avisynth-filter-db"
    },
    "dependencies": {  // I know flash3kyuu_deband doesn't have any dependencies :P
        "Fastlinedarkenmod": "1.3", 
        "sangnom": "1.0"
    }
}
- The filter itself + doc ( + maybe source code too) will be compressed with zip/7zip to keep the file size small.
Code:
/bin/filter.dll
/doc/{filtername}/filter.html
/src/{filtername}/[sourcecode]
/some other folder/
- The package will be uploaded to a website, where the package information will be stored in a mysql database. (but the package.json will stay in the package). You can search online ...
- An update file will be generated which holds the download url, filter name and version of every uploaded package, so the desktop app knows that to download.
- The desktop app downloads all necessary packages, extracts them and save them in a folder of your choice (my filters are not stored in the avisynth folder)
Code:
Structure:
/[filters]
/doc/{filtername}/doc.html, txt ... package.json
/src/{filtername}/[source code]
Basically it consists of two parts. The desktop app (downloads/extracts/updates your files) and a web server + db with all packages + an generated update file.

So, that do you think of this idea. Is it a good idea or waste of time?
I'm open to all suggestions.



Problems:
- Not every filter has a version number. Some uses date, some nothing :O
- How can I trust a package or verify its creator? (signing files with gpg + list of trust?)
- much work xD


// Interesting Sites
coapp.org (THE apt-get for windows, but still alpha/beta)
chocolatey.org
npmjs.org <-- seems to be a good solution, but needs a CouchDB and other stuff
__________________
Search and denoise

Last edited by ChaosKing; 19th February 2013 at 02:20.
ChaosKing is offline   Reply With Quote
Old 19th February 2013, 05:53   #2  |  Link
Adub
Fighting spam with a fish
 
Adub's Avatar
 
Join Date: Sep 2005
Posts: 2,685
I have to admit, this certainly is a cool idea.

Writing the client is going to be the interesting part, especially because you want to support multiple OSes (especially now with Vapoursynth). It will be very important for the user to be able to specify a custom install location, as a lot of people install them in weird locations.

Still, something that runs across systems and has support for Avisynth/Vapoursynth/Avisynth for Linux/Future Scripting Language is a very cool idea.
__________________
FAQs:Bond's AVC/H.264 FAQ
Site:Adubvideo
Adub is offline   Reply With Quote
Old 19th February 2013, 08:29   #3  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 971
In the interests of being cross-platform, I'd suggest using actual tarballs, not zip or 7zip archives. There's a much longer history of package management dealing with either tarballs (usually for source, sometimes for standalone binary distro) and distinct packaging formats (for binaries; deb, rpm, etc.). If LZMA2 is desired, it's not like tar.xz is at all exotic anymore (and tar.gz is nigh-ubiquitous); or even tar.bz2, for that matter.

There's going to be an issue either way: Windows doesn't natively have an untar program, and OSX and Linux usually only go as far as zip (and that can be kind of iffy, depending on which distro you're talking about here). But due to the typical conventions of Windows software distribution, it's probably a lot simpler/less of a hassle to just include a copy of tar itself with the package manager than having to resolve the necessary dependencies in the *nixen first.

Not to mention that for OSX/Linux you're going to need to deal with filesystem permissions (or even the possibility of symlinks), and so tarballs are more well-suited to the task.

Last edited by qyot27; 19th February 2013 at 08:36.
qyot27 is offline   Reply With Quote
Old 19th February 2013, 19:20   #4  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,870
Note that I'm not against the idea I just think it's kinda complicated. A big compressed file could accomplish most of it.

Personally I intend to bundle most open source plugins in the vapoursynth installer in the future. It just seems like the most reasonable way to do things. Every not too buggy/insanely big filter is welcome. Scripts too I guess once that gets going.

As for not windows, binaries are such a mess to distribute you'll just have to do it the source code way. The only exception being osx.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is online now   Reply With Quote
Old 21st February 2013, 13:38   #5  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 462
I don't think the client would be much of a problem. I thought about to code it in Qt5 with QML or c# (mono)... every external software will be bundled, like in megui -> portable
Quote:
It will be very important for the user to be able to specify a custom install location, as a lot of people install them in weird locations.
like me

@Myrsloik, but the user still would need to download newer filters himself. Except you offer regular updates.

My goal is that a filter developer could/should create a package himself. Once he has created a package, all he needs to do is change the version string in his next release and repack it, done! ... One minute effort. Every one is happy ^^


The idea of a big compressed file is also interesting. Then you could also offer incremental file updates to keep the download size small.
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Old 21st February 2013, 22:21   #6  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 462
Ideally a filter developer would do this himself and set the package config accordingly. Since he knows his filter best. (What would be the case for newer filters)
For older ones, well I guess the most popular plugins will be added quick (by me)
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Old 22nd February 2013, 22:30   #7  |  Link
Adub
Fighting spam with a fish
 
Adub's Avatar
 
Join Date: Sep 2005
Posts: 2,685
Indeed, for Windows users this will definitely require a GUI of some kind, simply because of historical reasons.

Linux users (like myself) should certainly be able to get by without a GUI, but I'm sure one would be appreciated nonetheless.

OSX, it's probably 50/50. As a Mac Book Pro owner, I use Homebrew enough for package management that I could easily do without the GUI. But again, considering the history of graphical applications in OS X and Windows, a GUI would likely be appreciated.

I believe that the Qt QML approach is a strong bet, especially considering that a number of systems have these libraries installed already, versus something like Mono. In addition, Vapoursynth already uses some of the Qt libraries, so there is a nice parallel there.
__________________
FAQs:Bond's AVC/H.264 FAQ
Site:Adubvideo
Adub is offline   Reply With Quote
Old 24th February 2013, 01:26   #8  |  Link
Caroliano
Registered User
 
Join Date: Feb 2005
Location: São Paulo, Brazil
Posts: 392
I wonder if something like ZeroInstall could be adapted for a plugin system...

The idea of big compressed file is simpler, but more difficult to maintain up to date. And it is already kinda being done in avisynth with filters like QTGMC that have lots of dependencies.
Caroliano is offline   Reply With Quote
Old 24th February 2013, 12:56   #9  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 462
Looks promising, will look into it. thx for the link
__________________
Search and denoise
ChaosKing is offline   Reply With Quote
Reply

Tags
avisynth, filter, management, package, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:39.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.