Experimental GPU Support in VapourSynth

[Myrsloik]

THIS IS A PREVIEW!!!

Modern GPU probably required at least in windows. By modern I mean something like AMD RDNA or Nvidia 2000+ series. Maybe. Do try and report your success and failures when using up to date drivers.

Longer blog post about it here (https://www.vapoursynth.com/2026/08/19/experimental-gpu-support/).

There's now a testing branch with experimental GPU support. The API of choice is Vulkan 1.4 but through the magic of mapping memory between most modern graphics APIs plugins can be written in CUDA and as well. Not that you really should but anyway...

What's done so far:
Full GPU support in all core filters. Note that filters taking multiple clips as inputs need to have all of them or none on the GPU. The only exception is for clips used only for metadata.

To transfer a clip to/from the GPU run:
clip.std.GPUUpload()
clip.std.GPUDownload()

Only one GPU can be used per core. The most powerful one should be picked automatically.


VapourSynth's vulkan branch (https://github.com/vapoursynth/vapoursynth/tree/vukvuk2)
BestSource with GPU output
(https://github.com/vapoursynth/bestsource/tree/vktest) (use gpu=True, the arguments and hardware decoding options have been completely reworked)
BM3Dvk (https://github.com/myrsloik/Vapoursynth-BM3Dvk)
NNEDI3vk (apimod) (https://github.com/myrsloik/VapourSynth-nnedi3vk)
KNLMeansVk (https://github.com/myrsloik/vapoursynth-knlmeansvk)


Note that only VS and BestSource currently have precompiled artifacts you can use. I'll try to have binaries for the rest available soon.

What needs to be done before a full release:

API review and criticism by the public
Testing on different hardware configurations
Determine what a modern GPU actually is in practice

[Columbo]

Is there any provision for running chains of filters on the GPU without having to transfer intermediate results back to the CPU?

[Myrsloik]

Is there any provision for running chains of filters on the GPU without having to transfer intermediate results back to the CPU?

Yes. Otherwise the core wouldn't need to know about gpu filters at all. Ypu get warnings on implicit transfers.

[Columbo]

Sounds good. I'll do some testing.

What would cause an implicit transfer?

Can you please give an example of running a filter chain?

[Myrsloik]

Transfers are handled this way:
Every filter declares its argument types. For normal filters they specify the vnode type which is a normal video node with frames stored in RAM. As of the GPU changes filters can also declare that arguments are vnode:gpu (frames in VRAM) or vnode:all (both handled).

If a video node of an incompatible type is passed it's automatically transferred to make the filter chain work.

Simple example:
clip1 = core.bs.VideoSource("h264_1.mp4", gpu=True)
clip2 = core.bs.VideoSource("h264_2.mp4", gpu=True)
blended = clip.std.Merge(clip1, clip2)
blended.set_output()


As you can see above there's no difference at all if all filters have native gpu support.

clip = core.bs.VideoSource("exotic_format.avi", gpu=False)
clip = clip.std.GPUUpload()
clip = clip.nnedi3vk.NNEDI3(clip)
clip.set_output()


If you remove clip.std.GPUUpload() it will simply implicitly insert it and warn since nnedi3vk only accepts GPU clips as input.

You can check what kind of node it is with a simple print(clip.gpu_resident)

[Adub]

Very cool to see, thanks for the hard work on this.

I've added a tracking issue to support this in Zsmooth: https://github.com/adworacz/zsmooth/issues/30. I'm not sure when exactly I'll get around to porting all filters, but I definitely see the value in keeping as many operations on the GPU as possible, even for traditionally very fast CPU operations. I plan to use Zig's native SPIRV support, which has seen a lot of improvements in the yet-unreleased 0.17.0 release.

[Myrsloik]

Very cool to see, thanks for the hard work on this.

I've added a tracking issue to support this in Zsmooth: https://github.com/adworacz/zsmooth/issues/30. I'm not sure when exactly I'll get around to porting all filters, but I definitely see the value in keeping as many operations on the GPU as possible, even for traditionally very fast CPU operations. I plan to use Zig's native SPIRV support, which has seen a lot of improvements in the yet-unreleased 0.17.0 release.

For those if you who are curious you can either simply read all core plugins that use the gpufilter.h header you can freely copy for maximum abstraction. https://github.com/vapoursynth/vapoursynth/blob/vukvuk2/sdk/

For example the invert filter with maximum abstraction can be found here: https://github.com/vapoursynth/vapoursynth/blob/vukvuk2/sdk/gpu_invert_driver_example.cpp

[Adub]

Oh now that's interesting. So a GLSL kernel is compiled on the fly when the filter graph is setup.

A Zig version wouldn't be too difficult, and could live in https://github.com/dnjulek/vapoursynth-zig long term.

[Myrsloik]

Oh now that's interesting. So a GLSL kernel is compiled on the fly when the filter graph is setup.

A Zig version wouldn't be too difficult, and could live in https://github.com/dnjulek/vapoursynth-zig long term.

Yes, GLSL compilation available. Also various levels of abstraction. I think the "normal" invert gpu sample (not raw) is what you can integrate easily into zig.

Also note that it's internally cached so if you repeatedly pass it the same shader it instantly returns the precompiled copy. That's an important detail.

[wonkey_monkey]

pc.u[] and pc.f[] are the parameter block fill() writes below.

"pc" as in Push Constants, right?

[Myrsloik]

"pc" as in Push Constants, right?

Yes, indeed it is. You can also bind buffers if you need to pass more data.

Populated with the fill callback if constant every frame or the fillFrame callback if they change per frame.

Edit:
Now renamed to things to make slightly more sense, that's why the post doesn't correspond to the sample names anymore

[Myrsloik]

Big news for you who want to try things. You can now install the gpu build with pip install vapoursynth==80a3

If you want to use any of the 4 filters listed they now have artifacts you can download from github. (click actions, scroll all the way down to the artifacts list, if you're grabbing bestsource make sure it's from the vktest branch)