Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th November 2022, 06:41   #1  |  Link
jlw_4049
Registered User
 
Join Date: Sep 2018
Posts: 387
Convert video node frame to numpy array or pillow image object (python)

I was wondering if someone could point me in the right direction. I've been struggling with it a portion of the day.

Currently I'm only able to output greyscale images from the frames.

I was attempting to use code like this

Code:
rgb_clip = vs.core.ffms2.Source(video.mkv) #1frame clip

#number of planes , for rgb should be 3 anyway
planes = rgb_clip.format.num_planes

#making numpy array from vapoursynths videonode
list_of_arrays = [np.array(rgb_clip.get_frame(0).get_read_array(i), copy=False) for i in range(planes)]
numpy_array = np.dstack(list_of_arrays)
But it is not what I need

Last edited by jlw_4049; 27th November 2022 at 06:53.
jlw_4049 is offline   Reply With Quote
Old 27th November 2022, 08:42   #2  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 268
Not sure what you aim for. Video should be in rgb, maybe your really is, to go frame by frame - vs frame to np array and back to vs frame:
Code:
import numpy as np
#from PIL import Image
#import cv2

def format_shuffles(n,f):
    npArray = np.dstack([np.asarray(f.get_read_array(p)) for p in range(3)]) #latest vs: f[p] instead of f.get_read_array(p)
    #work here with npArray image
    #hsv = cv2.cvtColor(npArray, cv2.COLOR_BGR2HSV)
    #PIL_img = Image.fromarray(npArray, 'RGB')
    #or work with PIL image than convert it back to np array
    #npArray = np.array(PIL_img)
    f_out = f.copy()
    [np.copyto(np.asarray(f_out.get_write_array(p)), npArray[:, :, p]) for p in range(3)]
    return f_out
clip = core.std.BlankClip(format=vs.YUV420P8) #here loading YUV video with whatever source plugin
clip = clip.resize.Point(format=vs.RGB24, matrix_in_s = '709')
clip = core.std.ModifyFrame(clip, clip, format_shuffles)
clip.set_output(0)

Last edited by _Al_; 27th November 2022 at 08:52.
_Al_ is offline   Reply With Quote
Old 27th November 2022, 18:17   #3  |  Link
jlw_4049
Registered User
 
Join Date: Sep 2018
Posts: 387
Thank you. I plan to make a viewer of sorts to view a video frame by frame and modify the script and visually see the changes

So this should help me. I got a 14 hour shift today so likely cannot test until tomorrow.
jlw_4049 is offline   Reply With Quote
Old 27th November 2022, 19:10   #4  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 268
You can just use view.py ,
just design your script in some python editor and just run the script
Code:
import vapoursynth as vs
clip = vs.core.lsmas.LibavSMASHSource('source.mp4')
clip = ... some filters ....
clip.set_output()
if __name__ == "__main__":
    import view
    view.Preview(clip)
it is a good idea to have that preview set in that if block, if you forget that there and actually using that py script for encoding , that block would not run

Last edited by _Al_; 27th November 2022 at 19:23.
_Al_ is offline   Reply With Quote
Old 29th November 2022, 15:57   #5  |  Link
jlw_4049
Registered User
 
Join Date: Sep 2018
Posts: 387
I actually played around with view.py for a bit. I did get it to work by editing the code, but it does not work as it stands. So I figured there was no support for it anymore.

I see now that you are the developer haha!

So, I still haven't had a chance to get to my computer thanks to work, but I plan to try to play around with the code tonight/tomorrow after I'm off. I'll respond back here the results, also I'll look at view.py again!
jlw_4049 is offline   Reply With Quote
Old 29th November 2022, 16:08   #6  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 268
Yes, it was for API3, Vapoursynth version R55 (not sure 100%) and below, not for API4, I thought because it is run in Python, fix would be easy for anyone, because it shows errors. Not using it, I use other versions which are not published. I'll look into it.
_Al_ is offline   Reply With Quote
Old 30th November 2022, 23:26   #7  |  Link
Quadratic
Registered User
 
Join Date: Jul 2021
Posts: 26
Quote:
Originally Posted by jlw_4049 View Post
Thank you. I plan to make a viewer of sorts to view a video frame by frame and modify the script and visually see the changes
.
https://github.com/Irrational-Encodi...dry/vs-preview
https://github.com/quietvoid/vspreview-rs
Quadratic is offline   Reply With Quote
Old 1st December 2022, 01:37   #8  |  Link
jlw_4049
Registered User
 
Join Date: Sep 2018
Posts: 387
Quote:
Originally Posted by _Al_ View Post
Not sure what you aim for. Video should be in rgb, maybe your really is, to go frame by frame - vs frame to np array and back to vs frame:
Code:
import numpy as np
#from PIL import Image
#import cv2

def format_shuffles(n,f):
    npArray = np.dstack([np.asarray(f.get_read_array(p)) for p in range(3)]) #latest vs: f[p] instead of f.get_read_array(p)
    #work here with npArray image
    #hsv = cv2.cvtColor(npArray, cv2.COLOR_BGR2HSV)
    #PIL_img = Image.fromarray(npArray, 'RGB')
    #or work with PIL image than convert it back to np array
    #npArray = np.array(PIL_img)
    f_out = f.copy()
    [np.copyto(np.asarray(f_out.get_write_array(p)), npArray[:, :, p]) for p in range(3)]
    return f_out
clip = core.std.BlankClip(format=vs.YUV420P8) #here loading YUV video with whatever source plugin
clip = clip.resize.Point(format=vs.RGB24, matrix_in_s = '709')
clip = core.std.ModifyFrame(clip, clip, format_shuffles)
clip.set_output(0)
I tried this but it didn't result in anything.

Code:
import numpy as np
#from PIL import Image
#import cv2

def format_shuffles(n,f):
    npArray = np.dstack([np.asarray(f[p]) for p in range(3)]) #latest vs: f[p] instead of f.get_read_array(p)
    print(npArray)
    #work here with npArray image
    #hsv = cv2.cvtColor(npArray, cv2.COLOR_BGR2HSV)
    #PIL_img = Image.fromarray(npArray, 'RGB')
    #or work with PIL image than convert it back to np array
    #npArray = np.array(PIL_img)
    f_out = f.copy()
    [np.copyto(np.asarray(f_out.get_write_array(p)), npArray[:, :, p]) for p in range(3)]
    return f_out
clip = core.lsmas.LWLibavSource(r"C:\Users\jlw_4\Desktop\rugrats test\The Rugrats Movie (1998).mkv")
# clip = core.std.BlankClip(format=vs.YUV420P8) #here loading YUV video with whatever source plugin
clip = clip.resize.Point(format=vs.RGB24, matrix_in_s = '709')
clip = core.std.ModifyFrame(clip, clip, format_shuffles)
format_shuffles(0, clip)
clip.set_output(0)
print(npArray) doesn't return anything. Because the function isn't actually running as far as I can tell.
jlw_4049 is offline   Reply With Quote
Old 1st December 2022, 04:51   #9  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 268
Quote:
Originally Posted by jlw_4049 View Post
format_shuffles(0, clip)
should be:
format_shuffles(0, clip.get_frame(0))

simplest basic opencv player, it will play without delay frame by frame:
Code:
import cv2
import numpy as np

clip = core.lsmas.LWLibavSource(r"C:\Users\jlw_4\Desktop\rugrats test\The Rugrats Movie (1998).mkv")
#assuming clip is loaded correctly from mkv
rgb = core.resize.Point(clip, format=vs.RGB24,    matrix_in_s = '709')
for f in rgb.frames():     
    img = np.dstack([np.asarray(f.get_read_array(p)) for p in [2,1,0]]) #or f[p] ...
    cv2.imshow('movie', img)
    cv2.waitKey(1)

Last edited by _Al_; 1st December 2022 at 04:54.
_Al_ is offline   Reply With Quote
Old 1st December 2022, 16:52   #10  |  Link
jlw_4049
Registered User
 
Join Date: Sep 2018
Posts: 387
Thanks for the reply. I did look these over, I'm not looking for a completed GUI to do this. I just am trying to understand the conversion/utilize it in a very basic window in python to complete some projects

Quote:
Originally Posted by _Al_ View Post
should be:
format_shuffles(0, clip.get_frame(0))

simplest basic opencv player, it will play without delay frame by frame:
Code:
import cv2
import numpy as np

clip = core.lsmas.LWLibavSource(r"C:\Users\jlw_4\Desktop\rugrats test\The Rugrats Movie (1998).mkv")
#assuming clip is loaded correctly from mkv
rgb = core.resize.Point(clip, format=vs.RGB24,    matrix_in_s = '709')
for f in rgb.frames():     
    img = np.dstack([np.asarray(f.get_read_array(p)) for p in [2,1,0]]) #or f[p] ...
    cv2.imshow('movie', img)
    cv2.waitKey(1)
This set me on the right track. I do have this working now at least. I appreciate the information so far!
jlw_4049 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:45.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.