VideoInputSource - grab video frame from video capture card or webcam in real-time

[fieliapm]

Preface:

I think it is well refined now, so I decide to share this plugin here for every end-user who want to retrieve video from capture device or web camera.

It is developed for my work in the beginning. But I think somebody might need this to processing real-time captured video, so I post here and wish I could help anyone who use AviSynth to do some prototyping easily.

One day, I am asked to import and process real-time captured video and merge with video from files in AviSynth for making some prototype.
In the beginning, I grabbed video frame from DirectShowSource() with grf file as input. However, I found that DirectShowSource() won't duplicate or skip frame when some video capture device (ex. webcam) cannot produce next frame ontime. This phenomenon will cause AviSynth's frame feeding slow down while playing AviSynth script.
For this reason, I made a AviSynth source filter which capture video from video capture device. It is based on a simple video capture library named VideoInput.



Download:

binary (prebuilt with AviSynth SDK 2.5.7 and VapourSynth R54, API version 3.6):
VideoInputSource.x86.dll (https://github.com/fieliapm/himawari_avs_plugin/blob/master/VideoInputSource/VideoInputSource.x86.dll) built for x86 32bit AviSynth 2.5.7 & VapourSynth API version 3.6
VideoInputSource.x64.dll (https://github.com/fieliapm/himawari_avs_plugin/blob/master/VideoInputSource/VideoInputSource.x64.dll) built for x86 64bit VapourSynth API version 3.6

source:
VideoInputSource on github (https://github.com/fieliapm/himawari_avs_plugin/tree/master/VideoInputSource)



Usage:

The usage of this source filter is as below:
VideoInputSource(device_id,connection_type,width,height,"fps_numerator","fps_denominator","num_frames","frame_skip")



# device_id: the n-th number of video capture device in your computer

# connection_type: can be these string: "Composite","S_Video","Tuner","USB".
# I leave this parameter because videoInput library offer this option.

# width, height: the width and height of frames captured from video capture device.

# fps_numerator, fps_denominator: FPS numerator and denominator.
# Default is 30/1 (30.0fps)

# num_frames: How many frame will be captured from video capture device.
# This will tell AviSynth the length of this video source.
# Default it will be the value which makes video source length become 24 hours.

# frame_skip: enable/disable frame skip while next new frame from video capture device is not ready.
# Default is true.
# When encoding is faster than real-time playing speed, please set this to false.




For example:

VideoInputSource(0,"Composite",1920,1080,24,1)


Will grab video from video capture device #0, 1920x1080, 24.0 fps, 24 hours long.



Appendix:

videoInput always output BGR24 packed pixels. For efficiency, current version of VideoInputSource supports AviSynth RGB24 color format only, and cannot be imported into VapourSynth script yet. Since I sometimes use VapourSynth for video processing, I might focus on compatibility of VapourSynth in the future.

By the way, It can work with MP_Pipeline very well since I often test this plugin in separate process generated by MP_Pipeline. However this plugin exclusively accesses to video capture device, so don't create video source from the same video capture device at the same time.

[StainlessS]

I've done a re-compile of this plugin source filter, and versioned it at v1.02 (did not previously have a version).
Also Added version resource (and v2.60/+ x86 & x64 plugs).

Plugs for Avs v2.58, avs v2.60/+ x86 & x64. + source + Vs 2008 project files.

Available for 30 days on SendSpace below this post or in the HOSTED folder @ MediaFire below this post.
Also available from direct link here:- http://www.mediafire.com/file/4meyk8qcbx7pic8/VideoInputSource_a25_a26_x86%2526x64_v1.02_20200331.zip/file


# Req VS 2008 Runtimes

# device_id: the n-th number of video capture device in your computer

# connection_type: can be these string: "Composite","S_Video","Tuner","USB".
# I leave this parameter because videoInput library offer this option.

# width, height: the width and height of frames captured from video capture device.

# fps_numerator, fps_denominator: FPS numerator and denominator.
# Default is 30/1 (30.0fps)

# num_frames: How many frame will be captured from video capture device.
# This will tell AviSynth the length of this video source.
# Default it will be the value which makes video source length become 24 hours.

# frame_skip: enable/disable frame skip while next new frame from video capture device is not ready.
# Default is true.


>>>>>>>>>>>>>>>>>>>>>>> Example >>>>>>>>>>>>>

# VideoInputSource.avs

ID=0
CONN="USB"
W=640
H=480
FPS_N=30
FPS_D=1
FRMS=100000
SKIP=False

VideoInputSource(ID,CONN,W,H,FPS_N,FPS_D,FRMS,SKIP)

Return Last

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<


Using TcpDeliver Plugin :::

TcpServer.avs

ID=0
CONN="USB"
W=640
H=480
FPS_N=30
FPS_D=1
FRMS=100000
SKIP=False

VideoInputSource(ID,CONN,W,H,FPS_N,FPS_D,FRMS,SKIP)

PORT=53270
TCPServer(port=PORT)


TcpSource.avs

/*

env->AddFunction("TCPServer", "c[port]i", Create_TCPServer, 0);
env->AddFunction("TCPSource", "s[port]i[compression]s", Create_TCPClient, 0);

*/

PORT=53270
COMPRESSION="GZip"

TCPSource("127.0.0.1", port=PORT, compression=COMPRESSION) # "127.0.0.1" = localhost ie same machine or self.


Eg load first script (TcpServer.avs) into Vdub and play. Then load 2nd (TcpSource.avs) script into another instance of Vdub [on same machine] and play.

If server on a remote machine, IP address eg 192.168.1.21, then on local machine use eg TCPSource("192.168.1.21", port=PORT, compression=COMPRESSION)

Still only works in RGB24.

EDIT: TcpDeliver [ TcpServer & TcpSource ] plugin by DJATOM :- https://github.com/DJATOM/TCPDeliver/releases

EDIT: Have tried on single x64 machine, using x86 VideoInputSource and TcpServer, and x64 TcpSource, seems to be no problem between the x86 and x64 plugins.

EDIT: Another thread with a little bit about TcpServer, TcpSource:- https://forum.doom9.org/showthread.php?p=1802408

[Milardo]

Hi,

Do you know of a way to add the audio source of a capture device to this so one can hear audio of the stream not just see video?

[StainlessS]

Nope, and I dont think the OP poster (author) would know either (and he aint been on-line since Oct 2018).
[He used Public Domain code for the video input stuff, and added the AVS interfacing stuff]
All I did was re-compile for the different dll's. (I wanted x64 ver$)

EDIT: You could maybe contact author on github, maybe he/she is now able to do it, is some kind of pro developer but not specializing in windows devices.
EDIT: He/She was recently active on github.

EDIT: Initial posting in Avs Devs forum:- https://forum.doom9.org/showthread.php?t=170311

[fieliapm]

Sorry I didn't came here long time because last year I have no time to focus on AviSynth usage and plugin development.

Somebody ask me so I am back :)

To reply Milardo's question, I think I should totally rewrite an AVCaptureSource, to capture AV stream by Windows native API directly or based on another full function capture framework.
But it is not an easy job to ensure capture procedure compatible to all AV input device, so it won't happen too soon.

Also thanks to StainlessS for plugin recompilication.
BTW, if you build plugin for official avs, I suggest to compile using Visual C++ 6.0 because official build is based on msvcrt from VC6.
Multiple dll based on different instance of C runtime will make program image in memory refer to two different C runtime code and cause resource leak.
Ideal dynamic linking should be:

VirtualDub.exe -----
　　　　　　　　　　|
　　　　　　　　　　V
avisynth.dll -> msvcrt.dll <- dynamic linked C runtime should be the same
　　　　　　　　　　^
　　　　　　　　　　|　　　　　　　　　
plugin.dll ----------

[StainlessS]

fieliapm,
thanks for the tip, not something that would have occurred to me.

[Milardo]

Sorry I didn't came here long time because last year I have no time to focus on AviSynth usage and plugin development.

Somebody ask me so I am back :)

To reply Milardo's question, I think I should totally rewrite an AVCaptureSource, to capture AV stream by Windows native API directly or based on another full function capture framework.
But it is not an easy job to ensure capture procedure compatible to all AV input device, so it won't happen too soon.

Also thanks to StainlessS for plugin recompilication.
BTW, if you build plugin for official avs, I suggest to compile using Visual C++ 6.0 because official build is based on msvcrt from VC6.
Multiple dll based on different instance of C runtime will make program image in memory refer to two different C runtime code and cause resource leak.
Ideal dynamic linking should be:

VirtualDub.exe ----|

avisynth.dll -> msvcrt.dll. <- dynamic linked C runtime should be the same

plugin.dll ----------|

Thanks for replying, be great when you can implement a complete capture source (video/audio)

[fieliapm]

This summer I promoted doing video processing using AviSynth and VapourSynth and introduced how to write their plugins to my local open source community members.
So I refine VideoInputSource source code, upgrade its dependency videoInput (current latest stable) and recompile it using VS2019 to support VapourSynth x86/x64.

Everyone might try it out.

[ChaosKing]

Could you make a github release? zip files and upload them in release tab?

[fieliapm]

It would happen later, I think.
Actually this plugin is currently under some experiment, something like reasonable dependencies, runtime settings, dependencies management are ugly and not well handled.
If you are interesting in development of this plugin, suggestion is welcome there http://forum.doom9.net/showthread.php?t=170311

I just came here for revision updating notification.