Quote:
Originally Posted by Dreadkillz
... but whenever I so much as select the avi file from my script, my folder explorer would just freeze. I checked that pfmhost also grows in memory usage by a lot. ...
|
Various tools and explorer plugins like to 'investigate' .AVI files. And then there are virus scanners and security products. It is crucial these never be allowed to read through the contents of the pseudo .AVI files. If they do then the pseudo contents has to be rendered. If it takes 2 hours to render the full Avisynth script, then it will take 2 hours to read through the pseudo .AVI file.
Joe, does PFM have knowledge of the "Application name" when a pseudo file access happens? Could various applications be black listed and the returned contents just be 1 big zero fill. Or maybe a white list so only say VirtualDub.exe, X264.exe and Vegas*.exe get to see the rendered pseudo contents.
Quote:
Originally Posted by joe48
... The problem is that the programs accessing the virtual AVI files often issue reads that require frames to be rendered that the application does not use.
... This is caused by sloppy reads of the AVI file headers and indexes inadvertently causing unneeded frames to be rendered. ...
|
In the original implementation, I went for up tight maximum packing in the pseudo AVI file. On reflection perhaps a sparse highly padded approach would have been better.
In an AVI structure 'JUNK' segments can be freely used to align/pad other data segments, this is normally done when capturing uncompressed video to match the clustering alignment of physical disks and maximise performance.
It would be trivial to have the frame data aligned on some nice round offset (say 64K or more) and add a 'JUNK' padding/guard block between each frame chunk. The data contents in the 'JUNK' blocks doesn't even matter, so no actual memory transfer/zero fill needs to occur, further boosting performance.
I assume the marshalling code is happy if no actual data is written to the application's buffer.
_____________________________________
Case study :- On mapped Linux systems the /dev/zero driver can apparently achieve throughputs of 100's of GB/s, way in excess of the actual memory bandwidth of the host CPU. This is achieved by zero filling 1 physical memory page and mapping it repeatedly onto the users buffer. E.g. App does a 1Mb _read of /dev/zero, on a modern CPU one might expect this to take 40 microseconds (25GB/s), profile the _read call and it only takes 5 microseconds (200Gb/s). Woooohoooo!