Avisynth virtual file system

[joe48]

Update
-------
AVFS 1.0.0.6 released, 2015.03.25

The latest release of Avisynth Virtual File System is available at:

http://www.turtlewar.org/avfs/

The package includes a readme.txt file with installation instructions and release notes.

For users of prior version of AVFS, things have changed. AVFS is now an executable, not a DLL. The installation is now simpler, but some explorer integration is lost. Also, the required Pismo File Mount (PFM) build 174 now implements mounting differently, through a c:\volumes folder instead of a virtual mount point placed over the AVS script.

End Update
-----------



I would like to help build an open source or freeware Avisynth virtual file system. This would allow a user to mount an Avisynth script, exposing a virtual uncompressed AVI file through the file system. This would provide alternative solutions for:
- chaining non VFWAPI enabled video processing tools.
- chaining tools that are running on different systems.
- chaining between 32 bit and 64 bit tools.

I have the Windows system extension work completed for this, see http://www.pismotechnic.com/ . If someone is willing to help who has Avisynth/VFWAPI/AVI code experience and resources, I believe things would come together pretty quickly.

[Gerard V]

FWIW this is an excellent concept - provides a means for a Linux port by mounting an avs script as a file system which can be accessed by other apps, bypassing the VFW issues entirely.

[IanB]

This sounds like an automagic MakeAvis implementation, could be very interesting.

This could be a very good way of mapping audio only scripts to a *.WAV file. This has always been a pain as currently they end up appearing as an Audio only *.AVI file, which of course most apps turn there noses up at.

Question :- if you AVFS *.avs files thru to there resulting AV streams how do you propose to edit the underlying script text?

[joe48]

I'm not familiar with AVFS.

With the Pismo File Mount system extension on Windows, files are mounted/unmounted manually by the user, not automatically based on name or type. This is done through mount/unmount commands in explorer context menus, or from the command line through the pfm.exe command. While mounted, the file appears as a folder, the contents of the folder provided by a user mode file system.

A user would edit the .avs file, mount it, access the resulting .avi (or .wav), unmount, edit the .avs file some more, etc..

[IanB]

AVFS="Avisynth virtual file system"

While mounted, the file appears as a folder, the contents of the folder provided by a user mode file system.

So you "mount" fred.avs, then access fred.avs\fred.avi or fred.avs\fred.wav

Would it then be possible to access fred.avs\fred.avs so you could edit the script without having to unmount/remount fred.avs (assuming you "compile" the script at each open of fred.avs\fred.avi).

All up this sounds pretty easy. The application opens the fred.avs\fred.avi, internally you have a "map" of the file's .avi structure. As the application read bytes from the file, you PClip->GetFrame() and/or PClip->GetAudio() to populate the user buffer with the appropriate data.

What do you need? Is the Pismo source available?

[joe48]

Ian,

I did a quick search for AVFS after your first post. It hit on a Linux project that allows transparent mount of archive file contents. I guess that wasn't what you were referring to. I like the acronym AVFS. It's not like it is really possible to avoid acronym collisions.

I think we're on the same page. The file system could be written a number of different ways. Like you mention, the source for the avs script could be echoed into a virtual file to allow it to be edited without unmounting. Could also dynamically change portions of the script based on the name of the output file that is opened, so opening the out.avi would run a slightly different script than opening out.wav.

fred.avs
+- script.avs
+- out.avi
+- out.wav
(or fred.avi, fred.wav, like your example)

To get started, a basic mode would be a single output avi file with Avisynth initialized only at the initial mount. This would avoid multiple instances of the filters running in a single process. Maybe that's not an issue.

[IanB]

So where to from here?

[joe48]

What I need to get started is another developer who can do the things you mentioned earlier.

Most of what I'll provide is already included in the Pismo File Mount Audit Package. This package includes a developer document, headers files, and a few C/C++ sample formatters (user mode file systems). Still, I can speed up the PFM specific parts of the development, do troubleshooting, etc..

To start with I can put together the basic AVFS formatter source, minus the actual AVI/Avisynth bits. I need to know the general parameters first:

C or C++?
Allow edit of AVS file while mounted?
Virtual file naming conventions?
Source code repository?
AVS file type recognition strategy?
Tools? (I prefer the SDK compilers myself)

The Pismo File Mount Audit Package can be downloaded from http://www.pismotechnic.com/download/ . It is a freeware utility that allows mounting if various archive file formats, and includes the PFM development kit. The audit package exists and is licensed for free use by individuals, business users, developers, and to be used as an enabling system extension for other freeware and open source projects.

If you are willing to work with me on this, send me a PM and we can continue offline.

[hanfrunz]

i came up with similar ideas years ago, hopefully now someone is coding it :-) here are the links to older threads:

http://forum.doom9.org/showthread.php?t=44028
http://forum.doom9.org/showthread.php?t=49383
http://forum.doom9.org/showthread.php?t=103426


i'm not a filessystem/driver programmer, but i'll help whereever i can.

regards,
hanfrunz

[joe48]

Coding is in progress. The driver bits were already done. AVFS is pretty simple as file systems go. Things should come together pretty quickly.

I expect some longer term work will be needed to add output file format options, to allow compatibility with more video processing programs.

[sh0dan]

Feel free to contact me, if you need additional, or help coding some of it.

I can help with various audio formats, and avisynth internals.

[Fizick]

Probably it was related discusion?
http://forum.doom9.org/showthread.php?t=108653

[joe48]

I hadn't read that thread. It is related.


I have finished the file system bits for AVFS. The code is on sourceforge in the module avfs at the root of the avisynth2 CVS repository. It builds with MSVC6 and has been through some developer testing.

http://avisynth2.cvs.sourceforge.net/avisynth2/avfs/

What is finished is a virtual file system that allows you to "mount" an avs script, turning it into a folder that contains an empty virtual media file and an editable copy of the script file.

Not finished:
1. Determine the initial names and sizes of the virtual media file(s).
- use stream info from avisynth and options from avs file to determine appropriate file format for virtual media file.
- support spanned media files, for the varios 32 bit file size limitations.
- support different media file formats, wav, avi1, avi2.
2. Handle virtual file read requests.
- figure out which frames and audio are needed for a range of bytes in the virtual file.
- read frames and audio from avisynth and fill the read buffer.
- fabricate any file or frame headers needed for portions of the read buffer.

Anyone interested in working on these remaining bits, send me a PM.

[hanfrunz]

two wishes:

-support for uncompressed bitmap sequences (TIFF, TGA, YUV, ...)
-support for uncompressed Quicktime-MOV

regards
hanfrunz

[IanB]

@Hanfrunz,

It would help if you did the google homework and find the references needed to write the file formats.

[Zarxrax]

Hmmm, would it be possible for this to support multiple files?
For instance, if I have a folder of 100 avisynth scripts, could they all be mounted?

[joe48]

Mounting and unmounting files is a manual action, but you can simultaneously mount as many files as you want. Each mount is handled by an instance of AVFS+Avisynth running in a dedicated pfmhost.exe process. You will be able to do things like feed the output of 2 scripts into a 3rd script.

[Mug Funky]

on quicktime support:

http://forum.doom9.org/showthread.php?t=104293

there should be some useful stuff there about stuffing yuy2 into a mov file, although i think it uses quicktime itself to handle it, so it may not work for virtual filesystem stuff.

one question:

would this work over a network? could i for example create an AVFS folder on the SAN here from one computer (an 8-core PC with dual fibre-channel connection to the SAN), and load it on another machine (a mac edit suite)?

it'd be truly awesome to be able to dump an avs straight into final cut pro, and render it there for realtime playback, especially considering the colour problems with quicktime (ie, if you use quicktime to export a mov, it'll bugger up the colours in a peculiar and unrecoverable way - something like YUV to RGB with made-up coefficients and some kind of gamma "correction").

i'm not sure if this would be possible though, bearing in mind the network share in question is a mac one, running who-knows-what filesystem.

[IanB]

@Mug Funky,

Ta! muchly (but the idea was to make Hanfrunz earn his keep )

Well we have the .WAV file virtualization working (this is a trivial file format) and as far as I can see W2K3 (at least) cannot tell there is anything different from real files in the same file tree. In fact for testing as this box has no sound card I shared the upper directory and accessed the virtual files remotely. Beeeuuudy!

[squid_80]

Sorry to be a spoilsport, but what happens if the .wav file is >4GB (5.1 channels etc.)?