How to use Frequency Separation in VapourSynth

[Reclusive Eagle]

Frequency Separation is extremely useful in many circumstances.

A lot of the time certain filters don't have the ability to target specific planes. So you end up having to choose different methods.
Or maybe you just want finer control over which specific frequencies filters are applied to or have other uses for separating textures from color like denoising.


What is Frequency Separation?
Essentially you are separating an image into 2 different layers.

A High Frequency Texture layer
A Low Frequency Blur layer


Why do this?
The Texture layer only holds textures. So lines, noise etc.
The Blur Layer only holds Color and Luma information without any texture.

Combining these 2 creates your original image.

Unlike RGB or YUV it does not rely on Luma layers at all for texture.
All Color and Luma values are separated from texture allow you to target specifically only texture.


How to Frequency Separate in VapourSynth:

Basically what we will be doing is taking your clip, and applying a blur filter to it.
You can use any blur filter you want. The stronger the Blur the more amplified (clear/defined) the textures will be.
However you don't want to blur your clip too much. Only blur it till you stop seeing any noise/texture.
This is our Low Frequency layer.

I recommend you use planes=[0] on your blur filter for this step. If you don't do this you will see color in your high frequency layer. You don't want that.


To get our High Frequency layer we need to subtract our blur layer from our clip using MakeDiff. This gives us the difference which is the High Frequency.

After that we can recombine both layers to form our Original image.
However now you can specifically target only the high or low frequency layers. You can also view them to see what filters are doing to each.

Example

#Frequency Separation
Source=YourClip

#Color Layer
low = core.std.Convolution(Source, matrix=[1,1,1,1,1,1,1,1,1], planes=[0])

#Texture Layer
high = core.std.MakeDiff(Source, low, planes=[0,1,2])

#Combine Layers
merge= core.std.MergeDiff(low, high, planes=[0,1,2])

merge.set_output()



To get stronger blur you can use:

low = core.std.Convolution(Source, matrix=[1,4,6,4,1,4,16,24,16,4,6,24,-476,24,6,4,16,24,16,4,1,4,6,4,1], planes=[0])

or

low = core.std.Convolution(Source, matrix=[1,1,1,1,1,1,1,1,1], planes=[0]).std.Convolution(matrix=[1,1,1,1,1,1,1,1,1], planes=[0]).std.Convolution(matrix=[1,1,1,1,1,1,1,1,1], planes=[0])

or

low = core.std.BoxBlur(Source, hradius= 2, vradius = 2,planes=[0])

etc.

[lansing]

It doesn't really work, the low clip is just a blurred video, you can still see every details on the image. And the high clip is barely visible, not really usable.

[Reclusive Eagle]

It doesn't really work, the low clip is just a blurred video, you can still see every details on the image. And the high clip is barely visible, not really usable.

Point is not to use them individually as main clips but to separate information in a way that gives the user far more control than RGB or YUV. If you want more information on frequency separation, look at the photography community.

Frequency separation implemented in photoshop is an extremely powerful tool to modify images. Its the same here for modifying frames.
This is not a crash course on how or why to use Frequency sparation.

Its a short tutorial on how to frequency separate in VapourSynth