Vapoursynth

[Selur]

Using R52:


clip = core.resize.Bicubic(clip=clip, format=vs.RGB48, matrix_in_s="470bg", range_s="limited")
# color adjustment using TimeCube
clip = core.timecube.Cube(clip=clip, cube="I:/Hybrid/64bit/vsfilters/ColorFilter/TimeCube/PQ_to_BT709_slope.cube")
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235)

-> green screen

clip = core.resize.Bicubic(clip=clip, format=vs.RGB48, matrix_in_s="470bg", range_s="limited")
# color adjustment using TimeCube
clip = core.timecube.Cube(clip=clip, cube="I:/Hybrid/64bit/vsfilters/ColorFilter/TimeCube/PQ_to_BT709_slope.cube")
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P10, matrix_s="470bg", range_s="limited")
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235)

-> green screen
# adjusting color space from YUV420P10 to RGB48 for vscube
clip = core.resize.Bicubic(clip=clip, format=vs.RGB48, matrix_in_s="470bg", range_s="limited")
# color adjustment using TimeCube
clip = core.timecube.Cube(clip=clip, cube="I:/Hybrid/64bit/vsfilters/ColorFilter/TimeCube/PQ_to_BT709_slope.cube")
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, matrix_s="470bg", range_s="limited")
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235)
-> works

Same happens when using RGB30 in the scripts.


Levels documentation says:
clip

Clip to process. It must have integer sample type and bit depth between 8 and 16, or float sample type and bit depth of 32. If there are any frames with other formats, an error will be returned.

since there isn't an error I assume this is a bug in Vapoursynth or I'm doing something wrong when converting RGBX to YUV.

So:
a. can others reproduce this?
b. is this a bug in Vapoursynth or am I missing something?


Cu Selur

[poisondeathray]

@Selur , for the 10bit levels cases, it should not be 16 and 235. It should be 64 and 940. And 4096-60160 for 16bit

Also, you are adjusting all channels, not just Y - is that what you wanted ?

And for CbCr, in 8bit it would be 16-240 , 64-960 for 10bit, 4096-61440 for 16bit

[Selur]

@Poisongdeathray: Ah okay, I assumed Levels would to the scaling automatically.

[Selur]

Small question: is there a way to disable auto-loading of filters inside a script (on Linux and Mac) to be sure that only the filters which are explicitly loaded in the script are tried to load?

[Myrsloik]

Small question: is there a way to disable auto-loading of filters inside a script (on Linux and Mac) to be sure that only the filters which are explicitly loaded in the script are tried to load?

Implemented in the doodle1 branch but it's only accessible through the environment policy interface so it's very iffy. Unfortunately I can't give you an example either right now.

[Selur]

no problem, I'll wait till it hits the main branch, but happy there it's in the works. :)

[shph]

I am not an expert but this may be a bug in Vapoursynth (my original post here https://forum.selur.net/showthread.php?tid=1495&pid=9159#pid9159)
Gamma correction works normally if source transformed to RGB. But in YUV it produce green/purple (depends of gamma adjustment direction) color shift.

Examles:
planes: all Gamma adjusted in YUV. Green/Purle color shift problem (probably bug in Vapoursynth):
https://i.imgur.com/19167W1.jpg

planes: luma Gamma adjusted in YUV. No color shift problem, but "desaturation" effect that is essential for this sort of luma only adjustment.https://i.imgur.com/pU5xRll.jpg

planes: all Gamma adjusted in RGB. This is how any normal gamma adjustment usually look:
https://i.imgur.com/tx0WgKI.jpg

As a reference here is screengrab with gamma adjusted PhotoLine graphic editor. Some difference may be due video player screengrabs color management and range transformations, but overall look is near the same. It is dark but not desaturated:
https://i.imgur.com/CdztCXQ.jpg

[VS_Fan]

I am not an expert but this may be a bug in Vapoursynth (my original post here https://forum.selur.net/showthread.php?tid=1495&pid=9159#pid9159)
Gamma correction works normally if source transformed to RGB. But in YUV it produce green/purple (depends of gamma adjustment direction) color shift.It's not a bug. A brief suggestion: Don’t include the chroma planes if adjusting gamma with core.levels in vapoursynth

Recommended reading: Why gamma-correction is applied to RGB instead of YUV or similar? (https://stackoverflow.com/questions/38750069/why-gamma-correction-is-applied-to-rgb-instead-of-yuv-or-similar)

Also, here’s some info from Avisynth’s Levels (http://avisynth.nl/index.php/Levels)
When processing data in YUV mode, Levels only gamma-corrects the luma information, not the chroma. Gamma correction is really an RGB concept, and is only approximated here in YUV. If gamma=1.0 (unity), the filter should have the same effect in both RGB and YUV modes. For adjusting brightness or contrast in YUV mode, it may be better (depending on the effect you are looking for) to use Tweak or ColorYUV, because Levels changes the chroma of the clip

[Selur]

Vapoursynth documentation should also include some of this,.. ;)

[shph]

That makes sense. Y is Luma (tonality data only) and UV is saturation (color data only). Gamma is tonal curve only, so seems it is just useless and incorrect to apply it to UV color data or to YUV all together.

It is ok to apply Gamma only to Y in YUV model.
It is ok to apply Gamma to all RGB channels together or to separate R G B channels in RGB model.

And so we came to YRGB color correction concept used in DaVinci Resolve and some other color grading apps :)
https://i.imgur.com/Z6L91hz.jpg

[Cary Knoop]

It is ok to apply Gamma only to Y in YUV model.

It really is not ok!

[shph]

Cary Knoop, Can you explain further why?

As i noticed earlier Y gamma is not the same as RGB gamma. Y gamma is more like some special effect when you need adjust gamma without saturation.

[shph]

Here is another observation. Tweak is a part of VapourSynth? It was a discussed earlier that color correction results don't match exact to other image editors, but seems it is just due different math concepts of same adjustments in different apps https://forum.doom9.org/showthread.php?t=175093

But also i noticed that Tweak Contrast always shifts a lot to dark side of gradient. It feels like center Pivot point for that tool is way off. Could it be a bug? Maybe Tweak also require some special gamma transformation?
You can download Test Patterns generated in DaVinci Resolve here: https://www.dropbox.com/sh/bak63hnr7mnpgbj/AAAf-rMK0LvHYTAFjPuQaaxFa?dl=0
https://i.imgur.com/w2H580V.jpg

To compare, here is how normal contrast adjustment look s in Davinci Resolve or any other app:
https://i.imgur.com/IRduNq0.jpg

[Cary Knoop]

Cary Knoop, Can you explain further why?

As i noticed earlier Y gamma is not the same as RGB gamma. Y gamma is more like some special effect when you need adjust gamma without saturation.
You mess up your colors by modifying Y'.

If you want to do what I suspect you want to do you should use something like L*a*b*.
Unfortunately, Vapoursynth does not support this color model.

[poisondeathray]

Here is another observation. Tweak is a part of VapourSynth? It was a discussed earlier that color correction results don't match exact to other image editors, but seems it is just due different math concepts of same adjustments in different apps https://forum.doom9.org/showthread.php?t=175093

But also i noticed that Tweak Contrast always shifts a lot to dark side of gradient. It feels like center Pivot point for that tool is way off. Could it be a bug? Maybe Tweak also require some special gamma transformation?



tweak is just a port from avisynth tweak

If you want to increase contrast in a fashion similar to how Resolve and other NLE programs work, use havsfunc SigmoidDirect , for an increasing contrast "s-curve". The cont value units do not correlate directly with other programs

clip = haf.SigmoidDirect(clip, cont=blah, planes=[0])

gif demo, cont from 1 to 36
https://i.postimg.cc/mgf84bmJ/havsfunc-Sigmoid-Direct.gif (https://postimages.org/)

There is also VapourSynth-Curve, which can take photoshop acv files, and can work in YUV or RGB, and you can apply to each/all planes

[shph]

Yes, that sort of symmetrical S-curve variations usually expected when something named as "Contrast". Also center point position of that curve sometimes may be adjusted and that adjustment named "Pivot" in other apps. So it maybe a bug in Tweak, or Tweak may be just designed like this for some purpose.

[feisty2]

for some unknown reason, this seems to crash vspipe (like it's stuck in an infinite loop or something)

from vapoursynth import *
import os
clp = core.std.BlankClip(format=GRAYS)
clp.set_output(42)
os.system(f'vspipe -o 42 {__file__} tmp.bin -p')
clp = core.raws.Source('tmp.bin', 640, 480, src_fmt='GRAYS')
clp.set_output()

@Myrsloik what is happening here?

[feisty2]

I found a workaround to solve the problem

from vapoursynth import *
import os
clp = core.std.BlankClip(format=GRAYS)
clp.set_output(42)
if 'cache_flag' not in globals():
os.system(f'vspipe -o 42 -a "cache_flag=yes" {__file__} tmp.bin -p')
clp = core.raws.Source('tmp.bin', 640, 480, src_fmt='GRAYS')
clp.set_output()

it doesn't seem very pretty though...

[feisty2]

I wonder if there's a better (more accurate) way of doing this (https://github.com/IFeelBloated/VaporMagik/blob/master/VaporMagik.py#L70) with vs.core or the vs environment stuff:

def TraceFilePathOfTheRunningScript():
Frame = inspect.currentframe()
Callers = inspect.getouterframes(Frame)
del Frame
for x in Callers:
if x.filename.endswith('.vpy'):
return x.filename

it should work for most cases but it relies on the script being a .vpy file, which I guess might fail in some rare cases?

[Jukus]

I don't know if need to write about this, where and how correctly, but this file opens correctly with lsmas, but all the frames are mixed up when using ffms2.
https://1fichier.com/?w8vicd396ctnl4ds9e45

[Selur]

@Jukus: looks fine here using ffms2,...
Ah okay, I see it if I don't drop halve of the fields. :)
Seems like the field order gets mixed up.

[vcmohan]

Can I get link to source code of BlankClip function?

[feisty2]

Can I get link to source code of BlankClip function?

https://github.com/vapoursynth/vapoursynth/blob/master/src/core/simplefilters.c#L1184

[stax76]

On the github website there is a search field in the top left corner, enter BlankClip there or whatever you search.

Or download the code, open the code folder in Visual Studio Code and click on the search button in the side bar.

https://github.com/vapoursynth/vapoursynth/blob/master/src/core/simplefilters.c

[vcmohan]

https://github.com/vapoursynth/vapoursynth/blob/master/src/core/simplefilters.c#L1184 many thanks. I am trying to code a plugin where there will be no input clip but only outputs frames. I find in blankclip
VSNodeRef* node = vsapi->propGetNode(in, "clip", 0, &err);
if (!err) {
d.vi = *vsapi->getVideoInfo(node);
vsapi->freeNode(node);
hasvi = 1;
}
What should I do. I have no node as input, but have I to include a node and bind it to something? Or I should have no ref to node in my code?
in the arInitial part what is the purpose of the following?

if (d->vi.fpsNum > 0) {
VSMap* frameProps = vsapi->getFramePropsRW(frame);
vsapi->propSetInt(frameProps, "_DurationNum", d->vi.fpsDen, paReplace);
vsapi->propSetInt(frameProps, "_DurationDen", d->vi.fpsNum, paReplace);
}
Is there an instance where fps == 0?
What is keep?
May be I am testing your patience! Sorry for it.

[feisty2]

you could try using my c++ wrapper for the vs api, things are as simple, straightforward and obvious as it can get.
this is an example that has no input but has multiple outputs: https://github.com/IFeelBloated/vsFilterScript/blob/master/Examples/Palette.hxx

[Selur]

Small question about the whole Limiter&Levels filter:
Can someone clear this up?
Is it okay to use Limiter&Levels on YUV content, or should they only be used on RGB content and the planes-parameter is basically a bad idea?
Or is RGB only required for Levels when gamma is used?

Cu Selur

[Selur]

with:

# Loading F:\TestClips&Co\files\10bit Test.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TestClips&Co/files/10bit Test.mkv", format="YUV420P10", cache=0)
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg",range_s="limited")
# making sure frame rate is set to 25
clip = core.std.AssumeFPS(clip, fpsnum=25, fpsden=1)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# Color Adjustment
clip = core.std.Levels(clip=clip, min_in=160, max_in=1020, min_out=160, max_out=880)
# adjusting output color from: YUV420P10 to YUV420P8 for x264Model (i420@8-bit)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, range_s="limited")

(this should be the same as using In 40/255 and Out 40/220 in 8bit right?)
I get a green image, where as when using:
clip = core.std.Levels(clip=clip, min_in=0, max_in=1020, min_out=160, max_out=880)
(this should be the same as using In 0/255 and Out 40/220 in 8bit right?)
everything looks fine.
-> to me this looks like there is some problem/bug with the min_in restriction with YUV420P10

Cu Selur
iirc scaling should be:
bit depth TV scale PC scale
8 bit 16-235 0-255
10 bit 64-940 0-1023
12 bit 256-3760 0-4095

[poisondeathray]

@Selur ,

In YUV, usually levels means Y levels

For levels, add

planes=[0]

[shph]

poisondeathray, Y levels looks more like special effect than something commonly used. See my post here
https://forum.doom9.org/showthread.php?p=1923597#post1923597

[poisondeathray]

poisondeathray, Y levels looks more like special effect than something commonly used. See my post here
https://forum.doom9.org/showthread.php?p=1923597#post1923597

Your post deals with "gamma" - gamma is an RGB concept

(Gamma can refer to Y plane; it acts as "power" function - and you can check with a Y waveform; but usually not applied to CbCr planes because they are color difference channels)

Levels in YUV , generally refers to Y plane only; applying it to the CbCr planes is the reason for the green discoloration Selur is observing

[shph]

Levels in-out min-max applied in YUV to Y only also look subjectively incorrect and very different to Levels applied in RGB to all planes.
Same as Gamma, Levels in-out min-max applied in YUV to Y only look more like some sort of special effect but not like something that people usually expect from levels correction.

[poisondeathray]

Levels in-out min-max applied in YUV to Y only also look subjectively incorrect and very different to Levels applied in RGB to all planes.
Same as Gamma, Levels in-out min-max applied in YUV to Y only look more like some sort of special effect but not like something that people usually expect from levels correction.

That's expected because gamma is a RGB concept.

Levels in/out min/max and gamma in Y looks correct for a Y levels correction. That's what a Y levels adjustment means.

RGB is additive color model . If you start with x value of R and double it, it gets more "red" . Same with Y' (in terms of it increases in brightness, it's not necessarily double) . But this does not happen with CbCr because they are color difference channels - the color completely changes

[shph]

This is all correct and was explained few posts earlier. I just try to explain that there two options to apply Levels/Gamma/Limiter that give different results:

- RGB channels together
- Y in YUV model

Why do you think that Y in YUV model looks correct but RGB is incorrect? If you compare result to image editors it appears that Y in YUV model looks "incorrect". My personal understanding of this filter is that both results look interesting and may be useful depending of artistic taste or some special requirement. Levels/Gamma/Limiter in RGB model behave like classic Levels tool in any image editing app.
Levels/Gamma/Limiter - Y in YUV model probably behave more like some sort of analogue video filter in equipment that operates in YUV.

[shph]

Examples:

Limiter in RGB model (forced RGB input was created by TimeCube filter with applied empty identity.cube LUT):
https://i.imgur.com/JntKXVd.jpg

Limiter - Y in YUV model:
https://i.imgur.com/cuOn1Pz.jpg

[poisondeathray]

Why do you think that Y in YUV model looks correct but RGB is incorrect?


I never said it was incorrect. It's correct for RGB .

I said it was correct for Y, and working correctly for Y channel



If you compare result to image editors it appears that Y in YUV model looks "incorrect".


An image editor works in RGB

[Selur]

I guess the confusion is that one intuitively assumes that YUV + planes=all would behave like RGB.
Okay, so:
a. either applying level limits only in Y or RGB (Levels and Limiter)
b. when using gamma always use RGB (Levels)
or expect the unexpected?

[poisondeathray]

a. either applying level limits only in Y or RGB (Levels and Limiter)


Levels limits by clipping can technically be applied to CbCr too, but the range is different than "legal" range Y (CbCr 16-240 in 8bit values, whereas Y is 16-235).

But if you use min/max in/out parameters, or gamma, range compression/expansion is applied or a power function (for gamma) - and that's when you can get unexpected results because of the way CbCr's works (color difference)

RGB range limiting is rare. There are 2 situations where the RGB range would be limited - for r103 compliance or studio range RGB. RGB is typically "full range" in 99.99999% of scenarios.



b. when using gamma always use RGB (Levels)
or expect the unexpected?


Not always; Adjusting "Gamma" - the parameter - can be used on Y channel as a power function . eg. you might want to roughly brighten shadows to a higher degree than highlights

The math for the Y curve is the same as , R, G, B curves (ie. the shape of the curve) when you adjust "gamma" for the latter. YUV obviously isn't RGB, so there is no reason to assume you'd get something visually similar

If you view a Y waveform, you can see what levels (any of the parameters, including gamma) in Y is actually doing to the image. People have used levels or smoothlevels in avisynth (since forever) . You know this, Selur

There are scenarios/ reason why one might prefer manipulations in YUV, but others in RGB

NLE's that work in YUV - their YUV levels "gamma" parameter works the same way, and their R,G,B "gamma" parameter for RGB levels works like an image editor

[HuBandiT]

I guess the confusion is that one intuitively assumes that YUV + planes=all would behave like RGB.
Okay, so:
a. either applying level limits only in Y or RGB (Levels and Limiter)
b. when using gamma always use RGB (Levels)
or expect the unexpected?

The answer depends on what you are trying to do.

What are you trying to do?

[Jukus]

Is there no way to use ffmpeg filters inside a script?

[poisondeathray]

Is there no way to use ffmpeg filters inside a script?


What filter(s) do you need that are not available natively ?

If you need to apply an exclusive ffmpeg filter, and there isn't something similar already available - ffmpeg output can be piped to a vs script (vsrawsource input), and that can be piped back into ffmpeg with vspipe or native ffmpeg vpy demuxer

[Jukus]

What filter(s) do you need that are not available natively ?
I searched for something autolevels and didn't find it.

I am trying to fix videos where good, normal scenes and very dark scenes are constantly alternating. I tried using "-vf pp=al", it just got a little better. Even if try to manually set the brightness and contrast values for a specific frame, then get disgusting grains, noise and others. I would be glad to have advice for this situation.

[poisondeathray]

I searched for something autolevels and didn't find it.

I am trying to fix videos where good, normal scenes and very dark scenes are constantly alternating. I tried using "-vf pp=al", it just got a little better. Even if try to manually set the brightness and contrast values for a specific frame, then get disgusting grains, noise and others. I would be glad to have advice for this situation.

I looked and couldn't find any either...

There are several "auto" leveling plugins in avisynth; if you're on windows you can use those in the vpy script. Avisynth+ is supposed to be crossplatform now, but I'm not sure if you can load avs plugins into vpy scripts on other platforms than Win

The ffmpeg/vpy piping back and forth workaround is not ideal; lots of overhead piping back and forth.

[_Al_]

Just as a curiosity sort of, you could pipe ffmpeg cmd line in vapoursynth directly in your script.
It is not ideal, it is one way only, no searching etc.
vs script is always python script so all python woo-doo is available in vs script as well. Not sure how many folks realize that.
import vapoursynth as vs
from vapoursynth import core
import subprocess
import ctypes

ffmpeg = r'C:\tools\ffmpeg.exe'
source_path=r'C:\videos\video.mp4'
clip = core.lsmas.LibavSMASHSource(source_path) #this clip is not not needed, just to get width and height
clip = core.std.BlankClip(clip)

w = clip.width
h = clip.height
Ysize = w * h
UVsize = w * h//4
frame_len = w * h * 3 // 2 #YUV420

command = [ ffmpeg, '-i', source_path,'-vcodec', 'rawvideo', '-pix_fmt', 'yuv420p', '-f', 'rawvideo', '-']
pipe = subprocess.Popen(command, stdout = subprocess.PIPE, bufsize=frame_len)

def load_frame(n,f):
try:
vs_frame = f.copy()
for i, size in enumerate([Ysize, UVsize, UVsize]):
ctypes.memmove(vs_frame.get_write_ptr(i), pipe.stdout.read(size), size)
pipe.stdout.flush()
except Exception as e:
raise ValueError(repr(e))
return vs_frame

try:
clip = core.std.ModifyFrame(clip, clip, load_frame)
except ValueError as e:
pipe.terminate()
print(e)

clip.set_output()

[feisty2]

VideoNode has a member function called output(), see: https://github.com/IFeelBloated/Oyster/blob/alpha/Alpha.py#L49

[_Al_]

yes, thats much better, using some filters on clip and then give it to ffmpeg:
import vapoursynth as vs
from vapoursynth import core
import subprocess

ffmpeg = r'C:\tools\ffmpeg.exe'
source_path=r'C:\videos\video.mp4'
clip = core.lsmas.LibavSMASHSource(source_path)

clip = clip.std.Expr(['x 20 -','','']) #just to demonstrate using some filters before ffmpeg

ffmpeg_cmd = [ffmpeg, '-f', 'yuv4mpegpipe', '-i', '-','-c:v', 'libx264', 'F:\OUT\output.mp4']
process = subprocess.Popen(ffmpeg_cmd, stdin=subprocess.PIPE)
clip.output(process.stdin, y4m = True)
process.communicate()

[_Al_]

or how about this,
vs -> ffmpeg -> prores - > vs again -> vspipe -> whatever
is it too crazy?
import vapoursynth as vs
from vapoursynth import core
import subprocess

ffmpeg = r'C:\tools\ffmpeg.exe'
INTERMEDIATE = 'F:\OUT\prores.mov'
source_path=r'C:\videos\video.mp4'
clip = core.lsmas.LibavSMASHSource(source_path)
clip = clip.std.Expr(['x 20 -','','']) #just to demonstarte using some filter before ffmpeg

ffmpeg_cmd = [ffmpeg, '-f', 'yuv4mpegpipe', '-i', '-', '-c:v', 'prores', '-an', '-y', INTERMEDIATE]
process = subprocess.Popen(ffmpeg_cmd, stdin=subprocess.PIPE)
clip.output(process.stdin, y4m = True)
process.communicate()

clip = core.lsmas.LibavSMASHSource(INTERMEDIATE)
clip = clip.std.Expr(['x 20 +','','']) #yet some more filters
clip.set_output()
technically , that ffmpeg could be piped into vs again like my example #1, not needing prores ...

[feisty2]

prores is lossy, ffv1 is probably better for your purpose

[_Al_]

VideoNode has a member function called output(), see: https://github.com/IFeelBloated/Oyst...a/Alpha.py#L49
What would be the purpose for that Materialize()? Function takes a clip, stores raw data on disk, then reads those data into same clip and that is returned. What is the Inject wrapper?

Oh, I guess it is used for more instances, only one time created.

[feisty2]

@Inject can dynamically inject any property into native types in python (stuff like list, or VideoNode), this is not normally allowed by python
Materialize() is used to speed up very slow scripts, it materializes intermediate results so they won’t be re-evaluated once things exceed max memory limit