Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd May 2018, 14:31   #3021  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Seems it only happens if mvtools-sf is in the filter chain.
Edit:
Is there a way (hack) to avoid this problem without modifying mvtools? I mean except putting it outside of FrameEval, bcs it's very slow and I need it only for many short scenes.

I will report this bug to feisty2 but who knows when and if he's gonna fix it.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 2nd May 2018 at 14:50.
ChaosKing is offline   Reply With Quote
Old 2nd May 2018, 15:39   #3022  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Quote:
Originally Posted by HolyWu View Post
Generally I don't recommend invoking filters within FrameEval unless unavoidable, because that introduces extra overhead.

PHP Code:
from vapoursynth import core
import vapoursynth 
as vs
import functools

def frame_eval
(nclip):
    return 
core.misc.AverageFrames(clipweights=[1] * 5)

clip core.std.BlankClip(width=720height=480format=vs.YUV420P8length=10000)

#clip = core.misc.AverageFrames(clip, weights=[1] * 5)
# I get 1014.60 fps

#clip = core.std.FrameEval(clip, functools.partial(frame_eval, clip=clip))
# I get 566.09 fps

clip.set_output() 
You should assign the filtered result to a variable outside FrameEval and then pass it in via functools.partial() as additional argument if possible.
Why didn't I think of that... it works perfectly.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 2nd May 2018, 16:45   #3023  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
I get the same speed up ~3x with your script. But what kind of cpu do you have to get 3000fps? My Ryzen 1700 8core @3.6ghz gets only half of that (VS R43)
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 5th May 2018, 18:27   #3024  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
R44-test2

More large page fixes and updated zimg. Test again because this will probably work better.

WINDOWS 10 APRIL UPDATE IS REQUIRED FOR PROPER LARGE PAGE TESTING

I forgot to mention that the cache logic has been improved so settings max memory usage to yuuuuuge numbers shouldn't be necessary anymore.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet

Last edited by Myrsloik; 5th May 2018 at 18:29.
Myrsloik is offline   Reply With Quote
Old 5th May 2018, 23:55   #3025  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Yep, I get the same speed (64-65fps, 4fps more than test1) with and without core.max_cache_size = 32768, cpu @ 99%, April 1803 update is installed.
With no max_cache set this is shown: Script exceeded memory limit. Consider raising cache size.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 7th May 2018, 16:31   #3026  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Did something break with the Windows 10 April 1803 update? I'm unable to load KNLMeansCL anymore; the plugin has been in the plugins64 folder all the time and now VSEdit is saying "AttributeError: No attribute with the name knlm exists. Did you mistype a plugin namespace?"

Code:
import vapoursynth as vs
core = vs.get_core()
clp = core.dgdecodenv.DGSource(r'C:\Temp\testclip.dgi')
clp = core.knlm.KNLMeansCL(clp)
clp.set_output()
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 7th May 2018, 16:39   #3027  |  Link
DJATOM
Registered User
 
DJATOM's Avatar
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 377
Check if your video drivers properly installed... Massive updates usually breaks things for me.
__________________
Me on GitHub
PC Specs: Ryzen 5950X, 64 GB RAM, RTX 2070
DJATOM is offline   Reply With Quote
Old 7th May 2018, 16:41   #3028  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Quote:
Originally Posted by DJATOM View Post
Check if your video drivers properly installed... Massive updates usually breaks things for me.
Yes, just found out that the stupid update rendered the graphics card useless and the NVIDIA driver package didn't even find an older version from the computer. Because I don't play games with the computer, I never noticed until now.

__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 7th May 2018, 17:13   #3029  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Quote:
Originally Posted by ChaosKing View Post
With no max_cache set this is shown: Script exceeded memory limit. Consider raising cache size.
I get this message with a 1080p source and some MVTools-based denoising + contrasharpening and advanced downsizing stuff in the script. I've tried setting the cache at 12000 but at some point the warning appears in the encoder log. Of course, with that value being so high (I have 16 GB of memory), the computer ends up swapping a lot so I need to tune it back to 10240 or so.

After some time, the memory usage of the vspipe process goes down to 3-4 GB of memory and stays there until the end. Does it mean that it's actually not useful to set a high value to start the encoding with?

EDIT: Except that with r44t2, the memory usage stays high all the time.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 7th May 2018 at 18:45. Reason: Additional info after running the encode for over an hour
Boulder is offline   Reply With Quote
Old 7th May 2018, 17:39   #3030  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Side info...

MABS is supposed to be able to build ffmpeg with VapourSynth support (for video filtering, like in mpv, IIRC). But it is important to know: If you build ffmpeg with VS support, VS also becomes a requirement to even start this specific ffmpeg build, it won't run on a PC where VS is not installed.

So I doubt I would provide such a build, as I won't need it for this kind of usage.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th May 2018, 20:19   #3031  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Kinnarps Chair
Posts: 2,548
Quote:
Originally Posted by Boulder View Post
I get this message with a 1080p source and some MVTools-based denoising + contrasharpening and advanced downsizing stuff in the script. I've tried setting the cache at 12000 but at some point the warning appears in the encoder log. Of course, with that value being so high (I have 16 GB of memory), the computer ends up swapping a lot so I need to tune it back to 10240 or so.

After some time, the memory usage of the vspipe process goes down to 3-4 GB of memory and stays there until the end. Does it mean that it's actually not useful to set a high value to start the encoding with?

EDIT: Except that with r44t2, the memory usage stays high all the time.
Basically the logic is closer to Avisynth behavior now. All leftover memory is used for recycled framebuffers. I have some ideas on how to improve things later. I don't think it's ideal but it does improve another corner case a lot... But if it stops at such a low total memory usage you can set it lower to begin with and it won't cause any problems.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 18th May 2018, 23:46   #3032  |  Link
Pat357
Registered User
 
Join Date: Jun 2006
Posts: 452
Quote:
Originally Posted by LigH View Post
Side info...

MABS is supposed to be able to build ffmpeg with VapourSynth support (for video filtering, like in mpv, IIRC). But it is important to know: If you build ffmpeg with VS support, VS also becomes a requirement to even start this specific ffmpeg build, it won't run on a PC where VS is not installed.

So I doubt I would provide such a build, as I won't need it for this kind of usage.
Appearently can ffmpeg with --enable vapoursynth be build that it will run also without any VS installed. This is done by building everything as static and load the stuff as delayed import.

The guys here build a binary ffmpeg this way (with gcc) and this version, unlike many other builds (ie the media-suite version) works very well.
The build from media-suite needs indeed that VS is installed, but even then I can't make it output any video from a valid VPY script.

See https://forum.doom9.org/showthread.php?t=175341

Last edited by Pat357; 18th May 2018 at 23:57.
Pat357 is offline   Reply With Quote
Old 20th May 2018, 15:19   #3033  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Supporting VS optionally when the environment is available ... that will be much more interesting.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 21st May 2018, 20:10   #3034  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
I'm wondering can vapoursynth read color value in hex?
lansing is offline   Reply With Quote
Old 22nd May 2018, 01:14   #3035  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Color is only ever read as a Python list of each value per plane. There's nothing keeping you from implementing a convenience function, however, like:

Code:
def Hex2List(colorhex):
  digits = math.ceil(colorhex ** (1/16.))
  colorlist = []
  for plane in range(0,digits):
    colorlist.insert(0, colorhex % 256)
    colorhex = colorhex // 256
  return colorlist
And calling it as Hex2List(0x778899).

You can always use an actual list like [0x77, 0x88, 0x99], so the value of this convenience is questionable.

Last edited by foxyshadis; 22nd May 2018 at 07:13.
foxyshadis is offline   Reply With Quote
Old 22nd May 2018, 02:32   #3036  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by foxyshadis View Post
Color is only ever read as a Python list of each value per plane. There's nothing keeping you from implementing a convenience function, however, like:

Code:
def Hex2List(colorhex):
  digits = math.ceil(colorhex ** (1/16.))
  colorlist = []
  for plane in range(0,digits):
    colorlist.append(colorhex % 256)
    colorhex = colorhex // 256
  return colorlist
And calling it as Hex2List(0x778899).

You can always use an actual list like [0x77, 0x88, 0x99], so the value of this convenience is questionable.
I'm talking about things like using it in blankclip like this, instead of the longer rgb value.

Code:
clip = core.std.BlankClip(color="#ffffff")
lansing is offline   Reply With Quote
Old 22nd May 2018, 04:57   #3037  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by lansing View Post
I'm talking about things like using it in blankclip like this, instead of the longer rgb value.

Code:
clip = core.std.BlankClip(color="#ffffff")
Is BlankClip(color=[0xff, 0xff, 0xff]) that much harder? If it is, use that convenience function in my last post.
foxyshadis is offline   Reply With Quote
Old 22nd May 2018, 05:23   #3038  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Quote:
Originally Posted by foxyshadis View Post
Is BlankClip(color=[0xff, 0xff, 0xff]) that much harder? If it is, use that convenience function in my last post.
I will need to load in a bunch of color, so the shorter the representation the better.

And Hex2List(0x778899) and [0x77, 0x88, 0x99] give me different color.
lansing is offline   Reply With Quote
Old 22nd May 2018, 07:15   #3039  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Whoops, edited. I forgot to test. If you're going to get a lot of use out of it, might as well rename it H2L or something.
foxyshadis is offline   Reply With Quote
Old 22nd May 2018, 08:43   #3040  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,657
Thanks foxyshadis and Holy, the function works now
lansing is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:27.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.