Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th March 2019, 00:40   #3301  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
When I use the vcmove rotate function, vs doesn't flush the memory after I close the preview in the editor.

Code:
clip = core.ffms2.Source(video_file)
rotate = core.vcmove.Rotate(clip, clip, angle=45)
rotate.set_output()
I got this message

Code:
Core freed but 4838400 bytes still allocated in framebuffers
lansing is offline   Reply With Quote
Old 30th March 2019, 00:54   #3302  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by lansing View Post
When I use the vcmove rotate function, vs doesn't flush the memory after I close the preview in the editor.

Code:
clip = core.ffms2.Source(video_file)
rotate = core.vcmove.Rotate(clip, clip, angle=45)
rotate.set_output()
I got this message

Code:
Core freed but 4838400 bytes still allocated in framebuffers
It's a vcmove bug. Report it to the author. The frame reference bkg in line 144 of moveRotate.cpp is never freed. Other operations may have the same typo so check all filters.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 30th March 2019, 00:59   #3303  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by Myrsloik View Post
It's a vcmove bug. Report it to the author. The frame reference bkg in line 144 of moveRotate.cpp is never freed. Other operations may have the same typo so check all filters.
Okay.
lansing is offline   Reply With Quote
Old 7th April 2019, 21:18   #3304  |  Link
thedangle
Registered User
 
Join Date: Jan 2018
Posts: 3
Not sure if this is a VS or waifu2x caffe thing, but using scripts with caffe seems to cause large amounts of virtual memory to be committed, even with core.max_cache_size set to 5124. Physical memory use seems to adhere to the memory limit, though, even if there's about 6gb of "available" physical memory and 3gb free VRAM.

I'd just ignore it since memory is meant to be used anyway but when it hits my commit cap virtualdub64 crashes. Example script (source res is 720x480):

Quote:
import vapoursynth as vs
import havsfunc as haf
import adjust
from vapoursynth import core
core.num_threads = 8
core.max_cache_size = 5124

clp = core.lsmas.LWLibavSource(r'test.avi',threads=8)
clp = adjust.Tweak(clp,sat=1.05)
clp = core.fft3dfilter.FFT3DFilter(clp, sigma = 2.2, planes=[1,2])
clp = core.pp7.DeblockPP7(clp, qp=1.8, mode=2)
clp = core.fmtc.matrix (clp, mat="601",col_fam=vs.RGB)
clp = core.fmtc.bitdepth (clp,bits=32,dmode=0)
clp = core.caffe.Waifu2x(clp, noise=2, model=6, scale=2, block_w=320, block_h=240, cudnn=True, tta=False, batch=3) # crashes at 720 wblock
clp = core.fmtc.bitdepth(clp, bits=16,dmode=0)
clp = core.knlm.KNLMeansCL(clp, d=0, a=12, s=0, h=.23, wmode=0)
clp = core.f3kdb.Deband(clp,random_algo_ref=2,random_algo_grain=2,blur_first=True,dynamic_grain=True,sample_mode=1,range=16,dither_algo=1,y=64,cb=80,cr=80,grainy=0,grainc=0,output_depth=16)
clp.set_output()
Looking at it further it seems windows 10 considers vram use as part of total usable memory, but does not add it to the calculation of max memory available. For example if I have 16gb ram, 6gb virtual memory and 8gb vram, my commit max is only 22gb, but if I use 8gb of vram and 1gb of ram windows considers 9gb of memory committed. Maybe this is working as intended? Not sure how separate pools are handled.

Last edited by thedangle; 10th April 2019 at 19:19.
thedangle is offline   Reply With Quote
Old 16th April 2019, 15:05   #3305  |  Link
Tima
Registered User
 
Join Date: Aug 2004
Location: Russia, Novosibirsk
Posts: 176
DoubleWeave doesn't seem to pick up field props:

DoubleWeave: field order could not be determined from frame properties

Code:
import vapoursynth as vs
import havsfunc as haf
import nextfunc as nextf
core = vs.get_core()
std = core.std

clip = core.avisource.AVISource(src)

clip = std.SetFieldBased(clip, 2)

woven = std.DoubleWeave(clip)
clip = std.SelectEvery(woven, 2, 0)

clip.set_output()
Tima is offline   Reply With Quote
Old 16th April 2019, 15:17   #3306  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
@Tima - if your AVI clip doesn't have field order in the file metadata, you can set the frame props

clip = core.std.SetFrameProp(clip, prop="_FieldBased", intval=0) #0=frame based (progressive), 1=bottom field first, 2=top field first.
poisondeathray is offline   Reply With Quote
Old 16th April 2019, 20:42   #3307  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Until R46 you will have to avoid using DoubleWeave's tff parameter. If you use it and the _Field property is not present/usable, DoubleWeave will swap the fields. Or I guess you can say tff=1 when you mean tff=0, and vice versa, but then your script will break when R46 appears.

A solution that will work both before and after R46 is to set the _Field property:
Code:
clip = core.avisource.AVISource(src)

tff = True

even = clip.std.SelectEvery(cycle=2, offsets=0)
even = even.std.SetFrameProp(prop="_Field", intval=tff)

odd = clip.std.SelectEvery(cycle=2, offsets=1)
odd = odd.std.SetFrameProp(prop="_Field", intval=not tff)

clip = std.Interleave(clips=[even, odd])

clip = clip.DoubleWeave()
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 16th April 2019, 22:07   #3308  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Would it be possible to add 'pop' or some other means to remove the top of the stack in Expr. I'm trying to make a sorting algorithm and it's kinda hard without that instruction. Well, it's not exactly easy with it either, sorting 5 values will take about 100 instructions, but at least it would be possible...
zorr is offline   Reply With Quote
Old 17th April 2019, 10:44   #3309  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by zorr View Post
Would it be possible to add 'pop' or some other means to remove the top of the stack in Expr. I'm trying to make a sorting algorithm and it's kinda hard without that instruction. Well, it's not exactly easy with it either, sorting 5 values will take about 100 instructions, but at least it would be possible...
At this point I'm surprised you don't propose a special sorting instruction. Something like "v1 v2 v3 v4 <number of previous values to sort on the stack> sortasc/sortdesc".

Also consider using a real compiler. If you have the patience to sort things using Expr then you should find that plugin writing isn't that bad.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 18th April 2019, 00:12   #3310  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by Myrsloik View Post
At this point I'm surprised you don't propose a special sorting instruction. Something like "v1 v2 v3 v4 <number of previous values to sort on the stack> sortasc/sortdesc".
Yes, that would be awesome! I did consider it but I know you're a busy guy. But if you're willing to entertain the idea of aggregate functions then I do think they would make Expr more... Expressive and useful. Median would be useful too (hard to implement), perhaps even a reverse (of top n values). The syntax you proposed seems good. Or you could do it in the style of SwapN and there would be Sort2, Sort3 etc.

Quote:
Originally Posted by Myrsloik View Post
Also consider using a real compiler. If you have the patience to sort things using Expr then you should find that plugin writing isn't that bad.
Not knowing much about the plugin development I think that would be quite a lot more time consuming endeavour than fiddling with the stack. But yeah, 100 instructions is not going to be too hot performance-wise when applied to every pixel... Are there good tutorials / introductions to VapourSynth plugin development? There was a reference to SDK directory, is that part of the full installation package only (didn't find it in the FATPACK)?
zorr is offline   Reply With Quote
Old 21st April 2019, 00:10   #3311  |  Link
AzraelNewtype
Registered User
 
AzraelNewtype's Avatar
 
Join Date: Oct 2007
Posts: 135
Quote:
Originally Posted by jackoneill View Post
Until R46 you will have to avoid using DoubleWeave's tff parameter. If you use it and the _Field property is not present/usable, DoubleWeave will swap the fields. Or I guess you can say tff=1 when you mean tff=0, and vice versa, but then your script will break when R46 appears.

A solution that will work both before and after R46 is to set the _Field property:
Code:
clip = core.avisource.AVISource(src)

tff = True

even = clip.std.SelectEvery(cycle=2, offsets=0)
even = even.std.SetFrameProp(prop="_Field", intval=tff)

odd = clip.std.SelectEvery(cycle=2, offsets=1)
odd = odd.std.SetFrameProp(prop="_Field", intval=not tff)

clip = std.Interleave(clips=[even, odd])

clip = clip.DoubleWeave()
Using SetFrameProp in this manner is throwing the exact error it was supposed to be avoiding. DoubleWeave only does anything at all if I set tff explicitly.
AzraelNewtype is offline   Reply With Quote
Old 21st April 2019, 11:21   #3312  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by AzraelNewtype View Post
Using SetFrameProp in this manner is throwing the exact error it was supposed to be avoiding. DoubleWeave only does anything at all if I set tff explicitly.
I can't see any problem there. What does your script look like?
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 21st April 2019, 17:48   #3313  |  Link
AzraelNewtype
Registered User
 
AzraelNewtype's Avatar
 
Join Date: Oct 2007
Posts: 135
Code:
res = core.std.Interleave(clips=[re, ro])
    res = res.std.SeparateFields(True)
    res = res.std.SelectEvery(cycle=4, offsets=[2, 1])
    res = res.std.SetFrameProp(prop="_Field", intval=1)
    # res = res.text.FrameProps()
    # res = res.std.SetFrameProp(prop="_FieldBased", intval=2)
    # res = res.std.SetFieldBased(2)
    res = res.std.DoubleWeave(tff=False)[::2]
It sets the prop just fine, but taking out the explicit tff=False and or using any of the commented lines instead/in addition just reports that it can't figure out field order from frame properties.

Edit: and of course I've just realized that I probably should have set this at the re/ro level one top and one bottom, because _Field and _FieldBased aren't really saying the same thing.

Last edited by AzraelNewtype; 21st April 2019 at 17:54.
AzraelNewtype is offline   Reply With Quote
Old 28th April 2019, 16:24   #3314  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
VapourSynth works in Wine now. Maybe this is of interest to someone.

I was able to use 64 bit VSEdit r19 with VapourSynth r45 portable and Python 3.7.3 portable in Wine 4.3.
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 29th April 2019, 13:09   #3315  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
How fast/slow is it compared to Windows?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 29th April 2019, 17:09   #3316  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 760
Quote:
Originally Posted by ChaosKing View Post
How fast/slow is it compared to Windows?
I don't know. I don't have Windows on this computer. I assume it's mostly the same once it's started. Wine takes a second or two to start up.
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 10th May 2019, 09:21   #3317  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
I'm trying to mass apply hundreds of lut files to segments of a clip. I ran a test loading 100 lut files in the script, each lut file is about 8MB, and it took a minute 45 seconds to load, that is not good. Considering the size of 300/400 lut will take over 5 minutes and more. Can I improve the loading speed with some lazy loading?

Code:
path = r"my file path"
file_list = os.listdir(path) # saving the lut file names to an array

clip = core.lsmas.LWLibavSource(file)
clip = clip[0:990]
rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.RGBS)

accu_file = core.std.BlankClip(rgb_clip, length=1) #dummy initial file for concat

for cube_file, frame in zip(file_list, range(0, 991, 10)):
	cube_file_path = path + '//' + cube_file
	accu_file += core.timecube.Cube(rgb_clip[frame: frame+10], cube=cube_file_path)

accu_file.set_output()
lansing is offline   Reply With Quote
Old 10th May 2019, 11:32   #3318  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
Quote:
Originally Posted by lansing View Post
I'm trying to mass apply hundreds of lut files to segments of a clip. I ran a test loading 100 lut files in the script, each lut file is about 8MB, and it took a minute 45 seconds to load, that is not good. Considering the size of 300/400 lut will take over 5 minutes and more. Can I improve the loading speed with some lazy loading?

Code:
path = r"my file path"
file_list = os.listdir(path) # saving the lut file names to an array

clip = core.lsmas.LWLibavSource(file)
clip = clip[0:990]
rgb_clip = core.resize.Bicubic(clip, matrix_in_s="709", format=vs.RGBS)

accu_file = core.std.BlankClip(rgb_clip, length=1) #dummy initial file for concat

for cube_file, frame in zip(file_list, range(0, 991, 10)):
	cube_file_path = path + '//' + cube_file
	accu_file += core.timecube.Cube(rgb_clip[frame: frame+10], cube=cube_file_path)

accu_file.set_output()
Does FrameEval() work?

Code:
from functools import partial

def apply_lut(n, f, clip, file_list):
    cube_file = file_list[n % 10]
    cube_file_path = path + '//' + cube_file

    return core.timecube.Cube(clip, cube=cube_file_path)

accu_file = core.std.FrameEval(rgb_clip, partial(apply_lut, clip=rgb_clip, file_list=file_list))
WolframRhodium is offline   Reply With Quote
Old 10th May 2019, 17:26   #3319  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by WolframRhodium View Post
Does FrameEval() work?

Code:
from functools import partial

def apply_lut(n, f, clip, file_list):
    cube_file = file_list[n % 10]
    cube_file_path = path + '//' + cube_file

    return core.timecube.Cube(clip, cube=cube_file_path)

accu_file = core.std.FrameEval(rgb_clip, partial(apply_lut, clip=rgb_clip, file_list=file_list))
Thanks, this works when take out the "f" from the function. Each frame took about a second to load now. But can I make it even more faster? Since one cube file would be applied to a range of frames, so the same cube doesn't have to be reloaded again on frames that lie within that range.
lansing is offline   Reply With Quote
Old 10th May 2019, 19:03   #3320  |  Link
WolframRhodium
Registered User
 
Join Date: Jan 2016
Posts: 162
Quote:
Originally Posted by lansing View Post
Thanks, this works when take out the "f" from the function. Each frame took about a second to load now. But can I make it even more faster? Since one cube file would be applied to a range of frames, so the same cube doesn't have to be reloaded again on frames that lie within that range.
One of the solutions is to stack frames together:
Code:
from functools import partial

rgb_clip_stacked = core.std.StackVertical([rgb_clip[i::10] for i in range(10)])

h = rgb_clip.height
sh = rgb_clip_stacked.height

def apply_lut(n, clip, file_list): 
    cube_file = file_list[n]
    cube_file_path = path + '//' + cube_file 
    return core.timecube.Cube(clip, cube=cube_file_path) 

accu_file_stacked = core.std.FrameEval(
    rgb_clip_stacked, 
    partial(apply_lut, clip=rgb_clip_stacked, file_list=file_list)
)
accu_file = core.std.Interleave(
    [core.std.Crop(accu_file_stacked, top=h*i, bottom=sh-h*(i+1)) for i in range(10)]
)
WolframRhodium is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.