Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th November 2023, 20:57   #4841  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 113
Anythig numpy can go turbo with Numba njit. And in this case even benefit of parallelism thanks to independant processing of the planes. However, muvsfunc would have to be rewritten extensively to only access Python and numpy primitives in its core muvsfunc_numpy module.

Is the sample you posted incomplete? There are no acccess to cv2 and the import is unused.

Last edited by cubicibo; 13th November 2023 at 21:02.
cubicibo is online now   Reply With Quote
Old 13th November 2023, 21:05   #4842  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,312
You are right, I don't need openCV, I first had it in there for some image handling, but I noticed that just using numpy was faster.
Okay, so the next question is: is numpy really needed?
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 13th November 2023 at 21:08.
Selur is offline   Reply With Quote
Old 13th November 2023, 21:31   #4843  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 113
You can drop Pillow, not numpy.

Code:
def convert_temp(frame: np.ndarray, temp: int) -> np.ndarray:
    return frame*(np.array(kelvin_table[temp], dtype=float)/255.0)
You may additionally want to np.round() before casting back to np.uint8.

Last edited by cubicibo; 13th November 2023 at 21:33.
cubicibo is online now   Reply With Quote
Old 14th November 2023, 00:09   #4844  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
colortemperature can be implemented using only vanilla std.Expr + std.ShufflePlanes.
WolframRhodium is offline   Reply With Quote
Old 14th November 2023, 03:14   #4845  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 325
Quote:
Originally Posted by cubicibo View Post
You can drop Pillow, not numpy.

Code:
def convert_temp(frame: np.ndarray, temp: int) -> np.ndarray:
    return frame*(np.array(kelvin_table[temp], dtype=float)/255.0)
You may additionally want to np.round() before casting back to np.uint8.
I tried this and it seams to work, more testing maybe needed. Basics are used, just numpy and vapoursynth, no muvsfunc_numpy , no PIL:
Code:
import vapoursynth as vs
from vapoursynth import core
import numpy as np

class Temperature:
    KELVIN_TABLE = {
        1000: (255,56,0),
        1500: (255,109,0),
        2000: (255,137,18),
        2500: (255,161,72),
        3000: (255,180,107),
        3500: (255,196,137),
        4000: (255,209,163),
        4500: (255,219,186),
        5000: (255,228,206),
        5500: (255,236,224),
        6000: (255,243,239),
        6500: (255,249,253),
        7000: (245,243,255),
        7500: (235,238,255),
        8000: (227,233,255),
        8500: (220,229,255),
        9000: (214,225,255),
        9500: (208,222,255),
        10000: (204,219,255)}
        
    def __init__(self, temp):
        self.rgb = self.KELVIN_TABLE[temp]

    def change(self, n, f):
        f_out = f.copy()
        for p in range(3):
            npArray = np.asarray(f[p])
            npArray = npArray*(np.array(self.rgb[p], dtype=float)/255.0)
            np.copyto(np.asarray(f_out[p]), npArray[:,:])
        return f_out

clip = core.lsmas.LibavSMASHSource('video.mp4')
clip = clip.resize.Point(format=vs.RGBS, matrix_in_s = '709')
clip = core.std.ModifyFrame(clip, clip, Temperature(4000).change)
clip = clip.resize.Point(format=vs.YUV420P8, matrix_s = '709')
clip.set_output(0)
_Al_ is offline   Reply With Quote
Old 14th November 2023, 05:29   #4846  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 325
Quote:
Originally Posted by WolframRhodium View Post
colortemperature can be implemented using only vanilla std.Expr + std.ShufflePlanes.
this seams to perform about the same speed as example above:
Code:
import vapoursynth as vs
from vapoursynth import core

KELVIN_TABLE = {
    1000: (255,56,0),
    1500: (255,109,0),
    2000: (255,137,18),
    2500: (255,161,72),
    3000: (255,180,107),
    3500: (255,196,137),
    4000: (255,209,163),
    4500: (255,219,186),
    5000: (255,228,206),
    5500: (255,236,224),
    6000: (255,243,239),
    6500: (255,249,253),
    7000: (245,243,255),
    7500: (235,238,255),
    8000: (227,233,255),
    8500: (220,229,255),
    9000: (214,225,255),
    9500: (208,222,255),
    10000: (204,219,255)}

def change_temperature(clip, temp):
    rgb = KELVIN_TABLE[temp]
    rgb = [value/255.0 for value in rgb]
    planes = core.std.SplitPlanes(clip)
    planes = [core.std.Expr(plane, expr=[f"x {rgb[i]} *"]) for i, plane in enumerate(planes)]
    return core.std.ShufflePlanes(clips=planes, planes=[0], colorfamily=vs.RGB)

clip = core.lsmas.LibavSMASHSource('video.mp4')
clip = clip.resize.Point(format=vs.RGBS, matrix_in_s = '709')
clip = change_temperature(clip, 4000)
clip = clip.resize.Point(format=vs.YUV420P8, matrix_s = '709')
clip.set_output(0)

Last edited by _Al_; 14th November 2023 at 05:31.
_Al_ is offline   Reply With Quote
Old 14th November 2023, 06:06   #4847  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 325
to not use KELVIN_TABLE, and using Selur's code in python to use any temperature, not just selected from a table:
Code:
import vapoursynth as vs
from vapoursynth import core
import math

def get_rgb(temp):
    temp = temp / 100
    if temp <= 66:
        r = 255
    else:
        r = temp - 60
        r = 329.698727466 * math.pow(r, -0.1332047592)
        r = min(max(0, r), 255)

    if temp <= 66:
        g = temp
        g = 99.4708025861 * math.log(g) - 161.1195681661
    else:
        g = temp - 60
        g = 288.1221695283 * math.pow(g, -0.0755148492)
    g = min(max(0, g), 255)

    if temp >= 66:
        b = 255
    else:
        if temp <= 19:
            b = 0
        else:
            b = temp - 10
            b = 138.5177312231 * math.log(b) - 305.0447927307
            b = min(max(0, b), 255)

    return round(r), round(g), round(b)

def change_temperature(clip, temp):
    rgb = get_rgb(temp)
    r, g, b = [value/255.0 for value in rgb]
    return core.std.Expr([clip], expr=[f"x {r} *", f"x {g} *", f"x {b} *"])

clip = core.lsmas.LibavSMASHSource('video.mp4')
clip = clip.resize.Point(format=vs.RGBS, matrix_in_s = '709')
clip = change_temperature(clip, 4345)
clip = clip.resize.Point(format=vs.YUV420P8, matrix_s = '709')
clip.set_output(0)
Expr seams to perform faster than working with numpy arrays.

Last edited by _Al_; 14th November 2023 at 06:39.
_Al_ is offline   Reply With Quote
Old 14th November 2023, 06:46   #4848  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,312
Nice, thanks.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 23rd November 2023, 10:56   #4849  |  Link
rgr
Registered User
 
Join Date: Jun 2022
Posts: 56
Vapoursynth needs a specific version of Python (3.11)? I have 3.13 and it doesn't want to install.
rgr is offline   Reply With Quote
Old 23rd November 2023, 11:06   #4850  |  Link
cubicibo
Registered User
 
Join Date: Feb 2022
Posts: 113
You need 3.11.
Why would you even use Python 3.13? It is in an early alpha and the Python fundation strongly advise to not use it [for production].
cubicibo is online now   Reply With Quote
Old 19th December 2023, 23:00   #4851  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,312
Are there color constants that can be used with AddBorders?
'float[] color=<black>' seems to suggest they are, if there are, is there a list somewhere?
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 20th December 2023, 09:16   #4852  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,569
Quote:
Originally Posted by Selur View Post
Are there color constants that can be used with AddBorders?
'float[] color=<black>' seems to suggest they are, if there are, is there a list somewhere?
No, it's just a simpler way than trying to express black in all different formats and bitdepths.

I've chosen to directly pass through values in all functions unlike avisynth which usually takes 8 bit RGB values and converts.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 20th December 2023, 09:17   #4853  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,312
Thanks for clearing that up.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 4th March 2024, 21:27   #4854  |  Link
Adub
Fighting spam with a fish
 
Adub's Avatar
 
Join Date: Sep 2005
Posts: 2,708
I had a few plugin development questions I wanted to pose to the community, and I figured I'd just post here instead of opening a new thread (although I'm happy to start one).

My questions:
1. When it comes to parameter handling based on bit-depth, what's the preferred behavior? For example, if a parameter normally ranges from 0-255 for 8-bit content, should plugins default to autoscaling parameters based on the input clip's bit depth, or should they not scale the parameter value and let the user handle it? A third option would be to make the scaling behavior configurable, with something like a boolean "scale_param" parameter that would let a user toggle autoscaling behavior if they want to be extra precise.
2. What open source licenses are preferred by this community? I see a broad mix with plugins, where some don't specify a license, some use MIT, and Vapoursynth itself uses LGPL (I believe). So I was wondering if there was a preferred standard license.
3. Are there any preferences for "skinny" vs "fat" plugins? In other words, creating a new plugin for every filter, or creating one plugin that houses a number of (at least semi-related) filters?

For reference, I'm experimenting with a new plugin that I'm writing in my free time, which currently consists of various smoothing/denoising related functions (things like averages, medians, etc). I'm kind of leaning towards making it "fat" by bundling a bunch of these filters into the same code base (think of having something like TemporalMedian, TemporalSoften2, and FluxSmooth all offered by the same code base).
Adub is offline   Reply With Quote
Old 4th March 2024, 21:42   #4855  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,569
Quote:
Originally Posted by Adub View Post
I had a few plugin development questions I wanted to pose to the community, and I figured I'd just post here instead of opening a new thread (although I'm happy to start one).

My questions:
1. When it comes to parameter handling based on bit-depth, what's the preferred behavior? For example, if a parameter normally ranges from 0-255 for 8-bit content, should plugins default to autoscaling parameters based on the input clip's bit depth, or should they not scale the parameter value and let the user handle it? A third option would be to make the scaling behavior configurable, with something like a boolean "scale_param" parameter that would let a user toggle autoscaling behavior if they want to be extra precise.
2. What open source licenses are preferred by this community? I see a broad mix with plugins, where some don't specify a license, some use MIT, and Vapoursynth itself uses LGPL (I believe). So I was wondering if there was a preferred standard license.
3. Are there any preferences for "skinny" vs "fat" plugins? In other words, creating a new plugin for every filter, or creating one plugin that houses a number of (at least semi-related) filters?

For reference, I'm experimenting with a new plugin that I'm writing in my free time, which currently consists of various smoothing/denoising related functions (things like averages, medians, etc). I'm kind of leaning towards making it "fat" by bundling a bunch of these filters into the same code base (think of having something like TemporalMedian, TemporalSoften2, and FluxSmooth all offered by the same code base).
1. Unscaled if there's actually some kind of direct relation to the pixel values.

2. MIT or GPL for plugins. If you use AGPL I will mock you for being special.

3. Not really.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 6th March 2024, 15:36   #4856  |  Link
efschu
Registered User
 
Join Date: Mar 2024
Posts: 2
Hello,

I'm modifying my jellyfin right now and replaced the ffmpeg binary with a script that calls vspipe together with ffmpeg. So I'm upscaling my content with realesr and interpolate it with RIFE in realtime on the server and watch the improved content on my clients.

This works fine for local files on server. But I also want to do this for live TV stream. But I did not figure out how to use a stream with vapoursynth. As I understood neither ffms2 nor l-smash nor bestsource support a stream as input.

Then I tried a workaround, to first save the stream in segments to disk, and then call vapoursynth script with a loop over all temporary stream files, but the loop didnt wait the files to finish one by one, instead the loop started to process all files at once (which resulted in out of gpu memory and forsure in not sequencial raw output frames) and crashed.

Sooo ladys and gents, do you have an idea how I could process live TV stream with vapoursynth? Or how I could process segmential files one by one with vapoursynth directly?
efschu is offline   Reply With Quote
Old 6th March 2024, 15:46   #4857  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,827
Hi.

I believe in your case, a custom build of ffmpeg or mpv with VPY filter support might be suitable, instead of frameserving.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th March 2024, 12:10   #4858  |  Link
efschu
Registered User
 
Join Date: Mar 2024
Posts: 2
Do you mean running it with
Quote:
ffmpeg -f vapoursynth - i script.vpy
But how would that help me with source dont have to be a "finished" file?

This is my script right now:
Quote:
from vsrife import RIFE
from pathlib import Path
import vapoursynth as vs
import sys
import os

core = vs.core
core.num_threads = 14

core.std.LoadPlugin(path='/usr/lib/x86_64-linux-gnu/libffms2.so')
core.std.LoadPlugin(path="/mnt/hts/libs/libvstrt.so")

sys.path.append("/home/efeu/Downloads/VSGAN-tensorrt-docker/")
from src.rife_trt import rife_trt

clip = core.ffms2.Source(source='/tmp/tmpvideo', cache=False)

clip = core.resize.Bicubic(clip, width=640, height=480, format=vs.RGBS, matrix_in_s='709') # RGBS means fp32, RGBH means fp16

clip = core.trt.Model(clip, engine_path="/workspace/realesr-general-wdn-x4v3_opset16_640_480.engine", num_streams=4, device_id=0)

clip = core.resize.Bicubic(clip, width=1280, height=960, format=vs.RGBS, matrix_in_s='709') # RGBS means fp32, RGBH means fp16

clip = rife_trt(clip, multi = 2, scale = 1.0, device_id = 1, num_streams=4, engine_path="rife46_ensembleFalse_op18_clamp_1280x960_P40.engine")

clip = core.resize.Bicubic(clip, format=vs.YUV420P8, matrix_s='709')
clip.set_output()
Btw, do you have an idea why vspipe sometimes only runs on one core, then I abort command and try it again, then it runs fine over "all cores"? (it mostly only runs on one core)



efschu is offline   Reply With Quote
Old 8th March 2024, 23:03   #4859  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,569
R66 is finally out. REQUIRES PYTHON 3.12!

See this blog post for an easier to read summary.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 9th March 2024, 09:00   #4860  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,312
Thanks!

Cu Selur
Ps.: Argh, there are no TensorRT Python bindings for 3.12 from the looks of it, so no R66 for me atm. (tensorrt-8.6.1-cp311-none-win_amd64.whl is the latest currently )

PPs.: seems like R66 portable is broken: https://github.com/vapoursynth/vapoursynth/issues/1033 -> turns out the install instructions have changed und you need now to also install the right wheel file matching your Python version.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 9th March 2024 at 11:10.
Selur is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:19.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.