Avisynth virtual file system

[joe48]

As Ian noted, AVFS is truncating .5s off the audio stream when the no-interleave option was set, creating an invalid AVI file. I put together a fix for this and Robert verified it resolved this issue. The fix will be in 1.0.0.5, which should be out pretty quickly.

The extra frame in the virtual AVI compared to the source file (491/490, 16.366s vs 16.333s) seems related to the DirectShowSource and ffdshow-tryout setup. Using FfmpegSource the frame count and duration are correct, verified by Robert. I have given up on reproducing this as I have been unable to get DirectShowSource and ffdshow-tryout to work on my system, a 2 yr old Vista 64 install.

The "open as audio only" issues are probably limitation of the specific application. I expect most applications will have trouble with non-interleaved audio, that was my experience during the initial AVFS development.

Thanks for the help Robert.

[Robert Martens]

You're welcome, I'm glad to be of service.

I think I've finally learned my lesson, and I'll stick to FFmpegSource and DGDecode for my non-AVI files from now on. Loading the clip as audio-only is likewise something I don't foresee a need for, interleaved or not, so if I need an external file as an audio source I can simply use the WAV files AVFS creates.

Thank you for the fix, I'll be sure to append that DV Info Net thread with the information once 1.0.0.5 is released!

[joe48]

I know a few pople have worked out how to setup scripts so AVFS will work with specific products, such as Quicktime and Final Cut Pro. In particular, being able to import without converting to RGB.

I would like to get this info in the AVFS readme to save some users having to figure this out, or giving up trying. Please post any tips you think will help.

Also, anyone who has had problems with things like explorer, shell extensions, anti-virus, content indexer, etc., causing grief by reading the virtual AVI files, get that info posted here as well so I can look into it.

[JoeH]

Hi Joe!

Thanks for this product. I use it together with DGIndexNV in order to import H264 / VC1 files directly into Sony Vegas Pro 8 for editing. I find that final rendering is about 25% faster than this way compared to using an intermediate codec, and obviously it's great for disk space.

The only difficulty I have found is that the thumbnails in Vegas are painted very slowly - it takes about 10 seconds usually to paint the whole timeline of thumbnails. In comparison, if I use an intermediate codec it takes around 1 second.

I have an NVidia 9500 GT which is reading the video using DGIndexNV. As Vegas can't import AVS scripts, I use your AVFS to create an AVI which I import. Any ideas for getting improved performance on the thumbnail creation?

[joe48]

Hi Joe!
The only difficulty I have found is that the thumbnails in Vegas are painted very slowly - it takes about 10 seconds usually to paint the whole timeline of thumbnails. In comparison, if I use an intermediate codec it takes around 1 second.
....
Any ideas for getting improved performance on the thumbnail creation?

I do not have any experience with Vegas (or any other pro editing tools for that matter), so do not know any specifics. Things are going to be slower with NLE tools since you have the script overhead for every frame that is touched by a read from the editing app.

You may be able to setup your script so that you can easily disable/enable the more time consuming processing steps. Use the script with the time consuming steps disabled when doing editing, then enable full processing and reopen your project before generating final output.

[Jim_Pansen]

I found the AVFS today & I'm very impressed! It matchs my needs for using AVS scripts in my workflow.
It speeds up my workflow immediately!
THX a lot for developing this!

Great tool!

Jim

[kolak]

Is there a way to implement V210 at the output, so in case of 10bit codecs (v210, Cineform, ProRes, DNxHD) we could keep this precision?
It's a great and simple piece of software :)

Thanks,
Andrew

[IanB]

@kolak,

Avisynth does not have any internal pixel formats other than 8 bit at present.

And the design here is to output uncompressed pixel data wrapped as an uncompressed virtual .AVI file.

This is an open source project, sure you could add a compression interface, but why would one want to burn cpu cycles generating pixel data. then compressing it into memory, only to then decompress it again immediately to use the raw pixel data. :confused:

[kolak]

@kolak,

Avisynth does not have any internal pixel formats other than 8 bit at present.

And the design here is to output uncompressed pixel data wrapped as an uncompressed virtual .AVI file.

This is an open source project, sure you could add a compression interface, but why would one want to burn cpu cycles generating pixel data. then compressing it into memory, only to then decompress it again immediately to use the raw pixel data. :confused:

You did not understand me correctly. I don't want to put any codec in between- these are exapmle of 10bit codecs only.

I just would like to keep 10bit precision. If my source file is 10bit (eg Cineform file) I would like to have 10bit uncompressed virtual file. It gets converter to YUY2 now. This is what I expcted, it's an avisynth limitation :(

Is there any trick to pass v210 output from eg. Cineform, ProRes?


Andrew

[StainlessS]

WARNING, Pismo File Mount Audit Package (pfmap-159-win.exe) is flagged
by COMODO AntiVirus as suspicious (Today with just updated AV DB).
Have submitted to VirusTotal.com where only COMODO out of 42 AV scanners
had any concern over it. Have submitted to COMODO as false +ve.

[joe48]

WARNING, Pismo File Mount Audit Package (pfmap-159-win.exe) is flagged
by COMODO AntiVirus as suspicious (Today with just updated AV

Anti virus products should be required by law to have a warning label attached to their packaging.
+------------------------------------------
| WARNING: Windows security products
| contain snake-oil, a substance known to
| the admin general to deteriorate the
| performance, reliability, and security of
| computer systems.
+------------------------------------------
(joke)

This looks like a heuristic (fuzzy logic) detection. Likely the anti-virus product noticed that the PFM installer loads a kernel driver. Possible the anti-virus product mistakenly thinks the PFM installer is a packed executable. I doubt PFM has specifically been classified as malware by the anti-virus product.

The PFM install package and every contained executable/DLL is signed with a code signing certificate. Most security products trust signed executables until they get a sample of some malicious code that was signed with the same certificate. I am surprised if Comodo is not doing the same.

Joe L.

[StainlessS]

Yes, just suspicious, as Heur.Suspicious@116614127.

Also, as an aside, I think Comodo is supposed to be one of the
biggest certificate issuers in the world.

[JoeH]

The only difficulty I have found is that the thumbnails in Vegas are painted very slowly - it takes about 10 seconds usually to paint the whole timeline of thumbnails. In comparison, if I use an intermediate codec it takes around 1 second.


I found a solution to this problem - index the file with FFMS2 and then mount with AVFS the AVS script that opens the ffindex file. The H264 / VC1 files fly... just like an intermediate codec. Very useful if you need to edit one in Sony Vegas, or any other video editing program.

[name123]

i like this concept but i am unable to test this out because when i run the installer for pismo film mount it crashes my win7 x64, has anyone experienced this?

[joe48]

i like this concept but i am unable to test this out because when i run the installer for pismo film mount it crashes my win7 x64, has anyone experienced this?

PFM runs fine on Win7 x64. A crash is most likely a compatibility issue between the PFM kernel extension and some other kernel extension or malware installed on the system. I am not aware of any specific compatibility issues at this time, but there have been a few cases over the last few years. If you are interested in helping to diagnose the issue, contact me via the support e-mail address listed on the Pismo Technic website.

[name123]

will do, thanks

[torwart]

You are a genious joe48!!!! Great program!!!!!!

[JoeH]

I use it on Win7 64 without any problems.

[Overdrive80]

Hi guys, i have a problem when i try reg dll with pfm command but windows says: "Not recognized as internal or external command, program or file batch executables".

Any idea?

[joe48]

Hi guys, i have a problem when i try reg dll with pfm command but windows says: "Not recognized as internal or external command, program or file batch executables".


Sounds like the PFM Audit Package is not installed.

[Overdrive80]

I try install "Pismo File Mount Audit Package build 159" but merge windows with this message:

http://www.imagengratis.org/images/capturadj7td.png

[joe48]

I try install "Pismo File Mount Audit Package build 159" but merge windows with this message:


PFM installs and functions on the vast majority of Windows configurations. If you are interested in helping to diagnose the issue with your configuration and PFM, contact the support e-mail address listed on the Pismo Technic website.

[verydoomed]

I am confused, I have the Pismo File Mount system extension and it works fine, why do I need this AVFS thingy? what is the advantage?

[IanB]

The Avisynth Virtual File System is a Pismo File Mount system extension to mount an Avisynth script as a virtual .AVI file. The .AVI file can then be opened in various programs that cannot normally open Avisynth scripts.

[Mug Funky]

quick question...

i've had some luck serving to crapple land by opening the yuy2 avi file, saving a reference mov of it, then using a text editor manually searchreplacing "yuy2" with "yuvs".

i just did some googling, and found this link:

http://developer.apple.com/quicktime/icefloe/dispatch020.html

this is telling:

kYUVSPixelFormat
'yuvs'

8-bit 4:2:2 Component Y’CbCr format. Identical to the k2vuyPixelFormat except each 16 bit word has been byte swapped. This results in a component ordering of; Y0, Cb, Y1, Cr. This is most prevalent yuv 4:2:2 format on both Mac and Win32 platforms. The equivalent Microsoft fourCC is ‘YUY2’.


now i might add that there seems to be some colour encoding problems - when i tried going SD to HD a couple of years ago, i found the colours shifted. but what may have happened is FCP assumed 601 coeffs even though i gave it 709 data.

anyway, if yuy2 is simply a byte-swap of 2vuy, it should be trivial for AVFS to just do a byte-swap and make the mounted avi file a "2vuy" (blackmagic or other SDI capture card, 8-bit 4:2:2) instead of a yuy2.

this would immediately remove a lot of manual text editing and possible colour mismatches from anyone trying to get avs content smoothly into quicktime land...

[IanB]

Setting the fourcc in the pseudo .avi header to "yuvs" is zero cost. Other fourcc's with the same byte order as 'YUY2' include 'yuyv', 'YUYV' & 'YUNV'.

Doing a word swap to implement '2vuy' would require additional processing. Currently VFB data is simply copied to the application buffer. Other fourcc's with word swap byte order of 'YUY2' include 'UYVY', 'UYNV', 'cyuv', 'IYUV', 'HDYC', 'Y422' & '2vuy'.

[Mug Funky]

that's good to know.

the reason i was looking at 2vuy is because there's a limited number of formats that final cut pro is willing to play out through an SDI interface without having to render anything.

though network bandwidth would most likely mean having to render anyway, i thought having AVFS do the grunt work would possibly allow a fast enough network to go to tape in realtime straight from an avs script.

my test last night showed no colour shifts, so i'm gonna go ahead and check HD now.

[k-c-ksum]

I try install "Pismo File Mount Audit Package build 159" but merge windows with this message:

http://www.imagengratis.org/images/capturadj7td.png

reboot then install is as "Run it as Administrator"

[tateu]

Is anyone else seeing unbearably huge memory leaks?

My system:
Windows 7 Pro x64
2 X5550 Intel Xeon Processors @ 2.79 GHz
16 GB Ram
AviSynth v2.58 tsp MT v5 2009/07/12 07:46:21
Pismo File Mount Audit Package build 159 (1.0.0.159.x64 2010/07/07)
AVFS 1.0.0.5 2010/05/23

After a clean reboot my System process is only using a few hundred KB of memory.

AviSynth Script used
ColorBars(928,768).Trim(0,1799)
or
AviSource("lagarith_no_audio_928x768.avi").Trim(0,1799)

pfm mount test.avs
Open mounted test.avi file in VirtualDub or the player/encoder of your choice.

In VirtualDub, press F5 to play the avi. By the time all 1800 frames have been played, my System process is using over 2GB of memory. If I close and reopen the file, it will push my System process memory usage up to a max of 4GB.

Closing and unmounting the file does not release the memory. The only way I can get it back is to reboot.

[tateu]

After more investigation, it appears that the memory leak might be with the Pismo File Mount Audit Package itself, not just the Avisynth virtual file system plugin.

I created an uncompressed avi, zipped it and mounted the zip file with the Pismo File Mount Audit Package zip handler and then opened the mounted avi in VirtualDub. I experienced the same memory leak in the System process with the mounted zip file as I did with the mounted avs file: within a short time of playing the avi file my System process had consumed 1/4 of my total installed memory and it doesn't get released until I reboot. I have tested this on several different machines and two different OSes.

Pismo Technic Inc. has a support email address so I will send them a note about it and see if they have anything to say.

[joe48]

Is anyone else seeing unbearably huge memory leaks?
...
After more investigation, it appears that the memory leak might be with the Pismo File Mount Audit Package itself, not just the Avisynth virtual file system plugin.


PFM does not have a memory leak. The number showing in task manager is just a number, there are no other symptoms to bear. If you had a huge memory leak then your system would get sluggish (I mean more than normal for Windows), and would eventually crash.

Most file systems on Windows use ram to cache data, true of PFM and NTFS. The amount of ram used by PFM to cache data is under complete control of the system (Windows), the same as for NTFS. For technical reasons, PFM maps its cached data differently than other file systems resulting in the current ram usage showing up in task manager in the system process. The cache ram usage of NTFS does not show up in the system process because it maps its cached data in the kernel, not because any more or less ram is being used.

The large number you are seeing in task manager is because you have plenty of ram installed in your machine and the system has not yet had any need to reallocate ram from the PFM cache to other tasks. As soon as you run some other memory hungry tasks, the system will reallocate the ram for other uses.

Joe L.

[ffffuuu...]

Hi there,
I've been aiming to use NTFS (under XP) to open a video from DVD structure in an app that only handles avi, so my goal was to save as much space (on mobile HD) as possible. I extracted audio tracks while indexing with DGMPEGDec, and then wrote a script using mpeg2source for the d2v index file, alternatively NicAC3source or FFAudioSource for ac3 file and AudioDub for joining the two. The resulting script played well in Media Player Classic, but the avi file created with AVFS rapidly slowed down to about 10fps and went out of sync. A video-only avi created with a script using only mpeg2source played at more less right framerate. What problem may there be?

[B.F.]

Latest version of PFM+latest version of AVFS.
win7x64
32bit programs work, 64bit doesn't. Any ideas about what can go wrong?

[joe48]

Latest version of PFM+latest version of AVFS.
win7x64
32bit programs work, 64bit doesn't. Any ideas about what can go wrong?

No idea off the top of my head. PFM does not give AVFS any information about the client applications.

Please send some instructions to reproduce to the support e-mail on the pismotechnic.com website. I will look into it.

Joe L.

[TakuSkan]

Latest version of PFM+latest version of AVFS.
win7x64
32bit programs work, 64bit doesn't. Any ideas about what can go wrong?
I too cannot get the latest version of PFM+latest version of AVFS to work in Windows 7 x64. It installs and creates virtual files fine in Windows 7 x32.

I've tried 3 older versions of Pismo File Mount Audit Package, and the problem persists. I'm trying to track down an older version of Avisynth Virtual File System, but am not having much luck. If anyone has a download link , could they please post it here?

[JoeH]

I have always used in on Win 7 64bit. For me it works fine with Vegas 32 bit, but not with Vegas 64 bit, but this isn't a problem for me for the moment.

[StainlessS]

It would seem to be relying on a 64bit ONLY file system/OS facility (EDIT: can't guess what that might be),
ie Windows 64 bit, so assumes the file system is a W64bit. (Vegas64)

[TakuSkan]

I have always used in on Win 7 64bit. For me it works fine with Vegas 32 bit, but not with Vegas 64 bit, but this isn't a problem for me for the moment.
Are you saying that you can successfully mount an avs file in Windows 7 x64? And then load the virtual file into a 32 bit version of Sony Vegas in Win7 x64?

I can get an avs file to mount in a 32 bit copy of Windows 7, but in Windows 7 x64 I get the error message:

"MOUNT FAILURE
File x\xxx\xxxx.avs cannot be mounted.
It is not of a supported file format."

[joe48]

"MOUNT FAILURE
File x\xxx\xxxx.avs cannot be mounted.
It is not of a supported file format."

This sounds like an installation issue. From a cmd prompt try:

pfm formatters

Output should include a line:
AVFS C:\...\avfs.dll

Try using some other tool to process a script directly using the avfs.dll from the same location.

Joe L.

[JoeH]

Are you saying that you can successfully mount an avs file in Windows 7 x64? And then load the virtual file into a 32 bit version of Sony Vegas in Win7 x64?

Yes, I've used it this way for a couple of years without problems.

[TakuSkan]

I found the problem. It started in the AVFS readme.txt file where it told me I could:

Copy avfs.dll to a folder of your choosing.

I tend to put all dll files in \windows\system32. That works for the avfs.dll in Win7 x32, but not Win7 x64 (probably should have been \windows\SysWOW64). I unregistered the file, moved it to \windows, registered it there, and files mount fine now.

Glad it was obvious looking at it from a fresh start. I know nothing about avisynth scripting, and had no idea how I'd do what you suggested, or what other tools I might try. I'm just glad to get a bit of a handle on MeGUI & Sony Vegas. ;)

Fantastic little tool! Thanks.

[kolak]

2 notes for developers:

-avfs needs ditributor() at the end of the script to run with setmt modes (otherwise script runs in 1 thread)
-if script is memory hungry (QTGMC, interframe) you have to patch pfmhost.exe with Large Address Aware flag to have much better stability. Maybe Pismo developer can add this flag by the default.

[JeanMarc]

Great tool, thanks to the developer!

A question though: my avfs seems to work great (I am still in the process of testing an SSIM script), but only from Windows 7 Administrative user. From the regular user, I get the error:
File "E:\path\to\my\file\sample.avs" cannot be mounted.
It is not a supported file format.

Is there something I missed?

[joe48]

A question though: my avfs seems to work great (I am still in the process of testing an SSIM script), but only from Windows 7 Administrative user.

You do not need to run as administrator to mount files with PFM. I suspect the avfs.dll is not registered or can not be loaded from your user account.

[joe48]

-if script is memory hungry (QTGMC, interframe) you have to patch pfmhost.exe with Large Address Aware flag to have much better stability. Maybe Pismo developer can add this flag by the default.

Good suggestion. I will have this set for the executables in a future PFM release. Not sure why MS has not made this the default yet.

[kolak]

Great tool, thanks to the developer!

A question though: my avfs seems to work great (I am still in the process of testing an SSIM script), but only from Windows 7 Administrative user. From the regular user, I get the error:


Is there something I missed?

No need for user to be only Administrator, but...

you have to register dll for each user separately!

It would be good to have an option: "Register for all users" during registration process. It there such a switch already?

@joe48:

Also- how much work is required to make MOV format?

[JeanMarc]

@Kolak and joe48. That didt it! I didn't realize I had to register the dll for each user! Thanks a lot.

[Alanick]

Hello all.
I've recently discovered this amazing way of mounting an video file using AVFS, AND I LOVE IT.
Thank you for this.
Now, if i may, i have a question.
What is the best .avs script option for an .m2ts, that has multiple audios, to select which ever you want and create the .avs then to be mounted.
for instance i use this option
DirectShowSource("TEST.m2ts", fps=23.976, audio=true, convertfps=true).AssumeFPS(24000,1001)
LanczosResize(720,480)
Audio being set to true, still does not see the audio in the mounted folder.
Can someone help me out with a better script on how to choose the audio if there is more than one, and

how to make it in 16bit for instance, 48.000Hz, and to preserve the channels 2 or 6 or 8
And when i mount the .avs to see inside the folder, the raw video with the wav at 16bit, 48.000Hz and all original channels(if possible).

Thank you in advance

[kolak]

What if you drag script to Vdub- does it show audio fine? AVFS preserves audio properly for me.
directshow source is not the best one- use some other, eg ffvideosource.

[tyee]

Does this system work in Win7 64bit and Sony Vegas 64 bit. I'm going to trial Vegas and have to decide 32 or 64 bit?

How can this be off topic. If it doesn't work with Vegas 64 bit, I can't use it.

JoeH said this -
I have always used it on Win 7 64bit. For me it works fine with Vegas 32 bit, but not with Vegas 64 bit, but this isn't a problem for me for the moment.

Just wanted to know if it still applies?