SegmentedAviSource with large number of files

swiftwr · 4th October 2005, 21:10

We are capturing video of the night sky as part of a video meteor survey. Some of the software we use requires files smaller than 1Gb so we capture in 8000 frame segaments. The problem is that this produces over 100 files in a night and SegmentedAviSource only seems to open the first 75 files or 600000 frames.

The question is, how can SegmentedAviSource be made to read either all of the files or be made to open the next 75 or so. The work arround of opening each file in turn is very anoying and combining the files into a single file is time consuming. This is a LOT of data, generally over 110Gb a night.

Any suggestions appreciated.

IanB · 5th October 2005, 15:00

Sorry, the current implementation runs out of internal VFW resources above about 70 files. Even without the resource restriction, the naming scheme will only work for 100 files (.00. to .99.). Depending on the codec in use SegmentedDirectShowSource may be able to open more files at once.

Also the code ignores missing files in a sequence so you could move groups of files into subdirectories, say 00 to 49 into dir1 and 50 to 99 into dir2 and process each group separately.

Another round about alternative might be to open all the segments in VirtualDub and use the frame server function to serve the resultant .vdf file back into your AVS script.

IanB

swiftwr · 5th October 2005, 17:50

IanB,

Thanks! Could you elaborate on frame serving with VirtualDub? Can it open a larger sequence than AviSynthe? How can I launch / use it in a script under Windows?

I am using AviSynthe as a frame server to IDL and process the images there. Maintaining a pointer to the origional frame number/time is very importiant but I might be able to build a work arround in IDL. Currently IDL is writing and launching an AVS file with a call to SegmentedAviSource to initiate the file server. I have no problems opening multiple file servers in this way so long as I can keep track of the frame number/time. Using DirectShowSource means we hand codec choice over to Microsoft, which has resulted in corrupted data in the past.

IanB · 6th October 2005, 16:19

VirtualDub will open the full 100 files of it's segmented format. It has it's own scripting language, sylia script.

The FrameClient support is installed from the auxsetup program shipped with VirtualDub.

To use the frame server, open all the files, select File>Start frame server ..., choose a share name (the default is usually okay), choose a name for the .vdr file, the frame server will start. Open the .vdr file as you would an .avi file. The video format is always RGB24.

The VirtualDub section in this forum can offer more detailed help.

IanB

4th October 2005, 21:10	#1 \| Link
swiftwr Registered User Join Date: May 2005 Location: Huntsville, AL, USA Posts: 2	SegmentedAviSource with large number of files We are capturing video of the night sky as part of a video meteor survey. Some of the software we use requires files smaller than 1Gb so we capture in 8000 frame segaments. The problem is that this produces over 100 files in a night and SegmentedAviSource only seems to open the first 75 files or 600000 frames. The question is, how can SegmentedAviSource be made to read either all of the files or be made to open the next 75 or so. The work arround of opening each file in turn is very anoying and combining the files into a single file is time consuming. This is a LOT of data, generally over 110Gb a night. Any suggestions appreciated. __________________ Wes

5th October 2005, 17:50	#3 \| Link
swiftwr Registered User Join Date: May 2005 Location: Huntsville, AL, USA Posts: 2	IanB, Thanks! Could you elaborate on frame serving with VirtualDub? Can it open a larger sequence than AviSynthe? How can I launch / use it in a script under Windows? I am using AviSynthe as a frame server to IDL and process the images there. Maintaining a pointer to the origional frame number/time is very importiant but I might be able to build a work arround in IDL. Currently IDL is writing and launching an AVS file with a call to SegmentedAviSource to initiate the file server. I have no problems opening multiple file servers in this way so long as I can keep track of the frame number/time. Using DirectShowSource means we hand codec choice over to Microsoft, which has resulted in corrupted data in the past. __________________ Wes

5th October 2005, 15:00	#2 \| Link
IanB Avisynth Developer Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,167	Sorry, the current implementation runs out of internal VFW resources above about 70 files. Even without the resource restriction, the naming scheme will only work for 100 files (.00. to .99.). Depending on the codec in use SegmentedDirectShowSource may be able to open more files at once. Also the code ignores missing files in a sequence so you could move groups of files into subdirectories, say 00 to 49 into dir1 and 50 to 99 into dir2 and process each group separately. Another round about alternative might be to open all the segments in VirtualDub and use the frame server function to serve the resultant .vdf file back into your AVS script. IanB

6th October 2005, 16:19	#4 \| Link
IanB Avisynth Developer Join Date: Jan 2003 Location: Melbourne, Australia Posts: 3,167	VirtualDub will open the full 100 files of it's segmented format. It has it's own scripting language, sylia script. The FrameClient support is installed from the auxsetup program shipped with VirtualDub. To use the frame server, open all the files, select File>Start frame server ..., choose a share name (the default is usually okay), choose a name for the .vdr file, the frame server will start. Open the .vdr file as you would an .avi file. The video format is always RGB24. The VirtualDub section in this forum can offer more detailed help. IanB