cache program / filter

[patrick_]

I'm searching for a program or filter to cache an AVS script to a lossless YV12 AVI file.
I would like it to either overwrite an existing AVI file (without deleting / setting size to 0) or allocate the entire size first on the disk (to prevent fragmentation).

AVS2AVI doesn't work, because when overwriting it resets the filesize to 0.

It would be nice if it could write unexclusively to the target file (so allowing other instances to write to the same file - it would be the best way to do multi-threading suposing the hard-disk is fast enough). If it can't do that, doesn't matter too much, because I'll just have to write to more than one file.

Does someone know about any CLI program or AVS-filter that can be of some use?

Thanks in advance.

[mikeytown2]

http://avisynth.org/mediawiki/External_filters#Export_Filters

Specifically
http://forum.doom9.org/showthread.php?p=1073371#post1073371

[patrick_]

Thanks, I was looking for that post because I remembered to have it seen before. Now re-reading it I realize that it doesn't solve my problem :( It does about the same as AVS2AVI.

What I'm looking for is this:
a filter/program that creates a single-fragment/low-fragmented file with null frames (or just pre-alocate without writing anything). Divide (using trim?) the AVS-video in parts (one for each core), and execute all of them at the same time writing to the same file. Windows (Vista) caches about 32MB of diskwrites to RAM so it should have any problems with such kind of load (suposing that the script is CPU dependent).

In case it would be to hard to write to the same file, each thread could write to a diferent file and later the could be joined with an easy avs-script - but still the AVI file must be allocated before writing to prevent fragmentation. Another solution would be to use diferent drives, but that's fine with 2 cores, but with 4 it's getting a lot harder.

[mikeytown2]

You might want to look into using MT
http://avisynth.org/mediawiki/MT

What types of functions are you trying to run?

[patrick_]

Right now I'm doing this:
open video, process interlaced filters (remove rainbows / ...) and deinterlace. Save to lossless AVI
open lossless AVI process all other filters (noise reduction / dehalo / anti-alias / resize / ...). Save to lossless AVI

The reason to "cache" output after deinterlacing is that is speeds up encode by a huge amount (to me it seems like avisynth doesn't use any caching, because I'm talking about a speed difference of 2000%)

I thought about using the MT library, but because of the use of Motion Compensation, I'ld have to use an overlap x pixels (don't know how much, I'ld have to find out), so the approach would be slower that completely independent threads.

[mikeytown2]

You can try using AVFS and mount the image instead of saving it to a lossless AVI... who knows, maybe it will work better.
http://www.turtlewar.org/avfs/

If you want to play around with a hard cache, the WarpSharp package has FrameCache
http://niiyan.net/?FrameCache
IIRC IanB doesn't recommend using it, but if you can improve the performance of your script by using this, please post your results and I hope you have a lot of ram!

Another Cache Filter
http://avisynth.org/mediawiki/External_filters#Support_filters
http://rapidshare.com/files/161446636/FrameCache_dll.zip

[squid_80]

There's also pipeline (http://forum.doom9.org/showthread.php?t=139629), although it absolutely has to be used with avisynthMT (which is based on 2.5.7) to prevent crashes.
If you've got plenty of ram you could also try using tcpsource/tcpdeliver for out-of-process caching.
It definitely sounds like you'd want some sort of prefetch method rather than running separate filtering passes with a lossless file generated in between.

[patrick_]

Thanks to both of you for your sugestions. I'll do some testing next week to see if any of those will help me out.