Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th August 2007, 17:17   #1  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,692
Centralised avisynth plugin repository

Hi all, just wondering if there are any opinions on an idea for an avisynth utility I've been mulling over for the past few weeks. I've also actually built a functioning prototype in python too, it's linked at the end of this message.

In a nutshell, the program will be a specialised repository tool for avisynth, along the lines of programs such as apt-get and yum on linux.

So far, this is what my prototype can do. It provides five functions:
o update: download a fresh copy of the repository metadata file
Code:
C:\>avup.py update
Downloading latest repository file now...
Successfully downloaded an updated repository file!
o info: print info (ie, all the metadata) on a given plugin
Code:
C:\>avup.py info limitedsharpenfaster
Name: LimitedSharpenFaster
Version: 29 Oct 2005
Author: didee
Homepage: forum.doom9.org/showthread.php?t=84196
Description: None
Functions implemented: limitedsharpen
Requires Plugins: masktools2, removegrain, warpsharp
o list: glob match for package names (this is currently folded into the info command, if no exact match of the given name is found, it assumes the input was a glob pattern and filters it against the list of packages).
Code:
C:\>avup.py info removegrain*
No exact matches, checking for glob matches now...
Matches: removegrain, removegrain-SSE3, removegrain-SSE2
o whatprovides: given an avisynth function name (ie 'xsharpen'), search the repository for a package that provides that function.
Code:
C:\>avup.py whatprovides *sharp*
Matches:
  removegrain: denoisesharpen, rsharpen
  limitedsharpenfaster: limitedsharpen
  warpsharp: warpsharp, xsharpen, unsharpmask
  removegrain-SSE3: denoisesharpen, rsharpen
  removegrain-SSE2: denoisesharpen, rsharpen
o download: download the given package.
Code:
C:\>avup.py download limitedsharpenfaster
Downloading limitedsharpenfaster now...
Successfully downloaded the plugin.
Successfully downloaded limitedsharpenfaster!

I was originally driven to write this program to help for two cases in particular:
o know a function, don't know the plugin it's part of. Useful when searching back through the doom9 archives and looking at ancient scripts in particular, but would be also useful for the many who won't remember that warpsharp contains xsharpen.
o know a plugin, don't know the latest version. Some plugins, like limitedsharpen, mvbob, removegrain are perenially asked after for the latest version, and different people have different versions. Some sort of central version control / download system would make much of this problem dissappear.

I am less interested in creating a high level package management system, which handles installation, package tracking and dependancies. Installing avisynth plugins is easy enough to do by hand, I just want a tool to make it easier to *get* these files. The true strength of a program like this would be in the above two cases.

Some issues that this program will run into:
o needs maintainers! Package management tools need people to create the packages. This is almost certainly the biggest potential problem, so I've tried to create a system which is extremely easy to use. The YAML repository metadata file is dead simple to manually edit, and the zip files just need a standardised name and that's it. The only hard part is needing to keep the repository in sync with multiple people changing it, but a basic system like svn + patches would be fine.
o needs servers & bandwidth! Probably not a real problem. I couldn't see this utility using more than 50gb/month for a long time, and I can happily donate that myself. A mirror system would be easy to implement though, and I'm sure volunteers could be found.
o People need python installed! Lies, you can pack up the script and the interpreter w/ required modules into a single executable as per AvsP. The only catch is that the interpreter + modules add up to around 2mb, which is a lot compared to the 2kb script.
o People need a GUI! That's not a problem to create at all, it would be trivial to adapt the current prototype to run with a GUI.
o ???




Sorry for the longpost, Here is the prototype, written in pure python. It works on windows and unix (whyever you'd want to run it on that..., but that's what I developed it on), and requires python 2.5 to be installed (should run with earlier versions, just with peculiar exception handling (such as keyboard interrupts not working)) along with PyYAML. Note that the repository and packages currently in there (all 5ish of them) are samples, but should be complete (apart from fft3dfilter, which is missing the library it requires).

Feedback welcome and requested
check is offline   Reply With Quote
Old 6th August 2007, 18:16   #2  |  Link
Dr.Khron
Registered User
 
Dr.Khron's Avatar
 
Join Date: Oct 2006
Location: Gotham City, USA
Posts: 389
Very cool idea! Thanks for the work!
As much as I love the various sites that archive plugins, sometimes its easy to get the different versions of a particular plugin mixed up.

Hopefully you'll get some support from the people who work on plugins...

I'll download and fool around with it.
Dr.Khron is offline   Reply With Quote
Old 6th August 2007, 19:39   #3  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
Could you create an AvsP version. That way people wouldn't have to download python if they have AvsP installed
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 7th August 2007, 01:09   #4  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,692
New mode planned for addition, will probably be named 'requires'. This will scan an input avs and return the packages needed.
Code:
C:\>avup.py requires test.avs
This script requires the following packages:
  removegrain: removegrain()
  masktools2: mt_lutxy(), mt_edge()
There are two tricky parts to it:
o detecting import()ed scripts, and those called which are autoloaded. This would just need every package found to be needed for the original file to be checked recursively for dependencies too.
o detecting plugins referenced with the plugin name prepended, ie dgdecode_mpeg2source(). Getting around this should be easy: if the full text of the filter doesn't match, try everything after the first underscore.

Quote:
Hopefully you'll get some support from the people who work on plugins...
Well, my wildest dreams involve new plugins being released along with the data required for addition into the avup repository Thankfully, it's pretty easy to actually write the metadata even if you aren't the author, so this isn't vital, just important.

Quote:
Could you create an AvsP version. That way people wouldn't have to download python if they have AvsP installed
There are two possible ideas from this
o Build a version that runs from avsp as a macro. Could be interesting, but I think it would be more flexible to build the backend as a standalone executable. You could always build a frontend into AvsP easily, which is probably the more flexible solution.
o Build a version that can use the avsp python libraries. I could, but there are a few problems that probably prevent this being useful. The first is that you need to hardcode the path to the libraries at packing time. This means everyone will need their own build.. and to build, you need python, which means you don't need a build. >_>
check is offline   Reply With Quote
Old 7th August 2007, 16:57   #5  |  Link
honai
Guest
 
Posts: n/a
You could use py2exe to make a standalone version that doesn't require local installation of the Python runtime.
  Reply With Quote
Old 9th August 2007, 15:44   #6  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,692
Quote:
Originally Posted by honai View Post
You could use py2exe to make a standalone version that doesn't require local installation of the Python runtime.
As I said I would in the first post, I have!

I also added the first test of the new mode 'requires'. Behaviour is similar to what I originally thought of:
Code:
>avup.py requires "Restore24.avs"
Required functions: removegrain-sse, warpsharp, avisynth-internal

Functions which no package could be found for: countnewpattern, yv12lutxy, leakkernelbob, soothe,...<etc>
This mode seems to work pretty well, but a few times it commits a bit of an oops, for example, with limitedsharpenfaster.avs it doesn't seem to be able to work out what supplies 'limitedsharpenfaster' ^^;. I will probably also add a 'verbose' mode for an output like my original idea later down the track too.

For now, I will do up a quick GUI frontend tonight, and then add a few more packages into the repository.


Requirements:
.py (3kb): Python 2.5, pyYAML
.exe (2mb): Windows 2k+ (possibly XP)

EDIT: one sort of bug that you will need to watch out for. The repository is looked for in the current directory. If the program is in C:\ but the command window is in C:\WINDOWS, your repo needs to be in the latter.
check is offline   Reply With Quote
Old 9th August 2007, 18:19   #7  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,692
Newer versions are up, same address as the previous post. What's important enough to warrant a double post? A GUI! The wxPython libraries have blown the exe version out to 4mb though
I am now planning on writing a few utilities to facilitate creating repository entries for plugins, mostly because I want to convince a few other people to help too Any volunteers?
check is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.