Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th July 2019, 22:15   #3541  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
Quote:
Originally Posted by lansing View Post
Is there a max speed limit set in vapoursynth? I'm testing out my new cpu with source filter like ffms2 on my dvd source, and the max speed I got is 1800 fps, while the same ffms2 in avisynth reaches 5200 fps.
How did you test the speed?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 26th July 2019, 22:41   #3542  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,159
Quote:
Originally Posted by Myrsloik View Post
How did you test the speed?
I open the script in vs editor and run the benchmark function, for avs+ I use avsmeter.
lansing is offline   Reply With Quote
Old 26th July 2019, 23:17   #3543  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
Quote:
Originally Posted by lansing View Post
I open the script in vs editor and run the benchmark function, for avs+ I use avsmeter.
"vspipe script.vpy ." <- try that

If it's still slow lower the number of threads. You're effectively creating a worst possible scenario if you throw a lot of threads at only one source filter with no processing.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 27th July 2019, 02:16   #3544  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,159
Quote:
Originally Posted by Myrsloik View Post
"vspipe script.vpy ." <- try that

If it's still slow lower the number of threads. You're effectively creating a worst possible scenario if you throw a lot of threads at only one source filter with no processing.
Using vspipe gives 4200 fps, though it is still 25% slower than avs+.

Then I ran another test on a longer sd video (2 hours), avs+ got 4700 fps and vs got 4400 fps, the difference is much closer, while both took less than 25% cpu usage.
lansing is offline   Reply With Quote
Old 27th July 2019, 02:55   #3545  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 654
Using different tools (avsmeter and vspipe) for performance comparison won't give you very precise result, since the mechanism for measuring the speed in each application is not the same. You can't really conclude your result unless the mechanism in each application is identical.

Last edited by HolyWu; 27th July 2019 at 07:52.
HolyWu is offline   Reply With Quote
Old 27th July 2019, 11:45   #3546  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,033
Quote:
Originally Posted by HolyWu View Post
Using different tools (avsmeter and vspipe) for performance comparison won't give you very precise result, since the mechanism for measuring the speed in each application is not the same. You can't really conclude your result unless the mechanism in each application is identical.
I can confirm it. It just tested via vspipe, vsedit, avsmeter and vdub2 with a 1h h264 DV res video.

vsedit is the "slowest" (always ~220fps slower). avsmeter always wins by a couple of sec (3-5sec) or 1586fps vs 1404fps in vspipe. This is also confirmed with a stopwatch in powershell.

In vdub2 on the other hand, avsiynth takes 58sec while vapoursynth is done in 54sec.

Conclusion: ffms2 is more or less equally fast in VS and AVS+ and the tools to measure the speed can have a significant impact on the speed and/or the results.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is online now   Reply With Quote
Old 28th July 2019, 00:53   #3547  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
Quote:
Originally Posted by lansing View Post
Using vspipe gives 4200 fps, though it is still 25% slower than avs+.

Then I ran another test on a longer sd video (2 hours), avs+ got 4700 fps and vs got 4400 fps, the difference is much closer, while both took less than 25% cpu usage.
Go back a few years and you'll see similar questions to this. Anyway...

The 25% cpu usage is because you reach the multithreading limit of ffmpeg. This is a very artificial experiment.

The remaining speed difference is due to differences in how memory allocation works. You're basically benchmarking the respective memory pools in the most artificial way possible. This is an extremely artificial experiment.

Actually I think the ffmpeg binary would win this competition if you just find the right options. I'm generally not interested in benchmarks like these since people rarely even get the options right. See my first answer.

Go benchmark something meaningful next time.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th July 2019, 22:36   #3548  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,159
Something weird I just noticed after more testings, in the beginning of my benchmarks, task manager shows all drives at 0% read/write activity while the cpu was running. But some 30 seconds in, the drive that contains the source video began to have some read activities for no reason. And this had effectively bottlenecked my speed. In my script here, it was running at 100+ fps with 100% cpu load for the first 30 seconds, then it had dropped to 70 fps when the drive started to have some 40% reading.

Code:
clip = core.ffms2.Source(file)
clip = core.dfttest.DFTTest(clip)
I have tested on both vs and avs+, same behavior.

Can anyone reproduce it?
lansing is offline   Reply With Quote
Old 28th July 2019, 22:42   #3549  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,033
Quote:
Originally Posted by lansing View Post
Something weird I just noticed after more testings, in the beginning of my benchmarks, task manager shows all drives at 0% read/write activity while the cpu was running. But some 30 seconds in, the drive that contains the source video began to have some read activities for no reason. And this had effectively bottlenecked my speed. In my script here, it was running at 100+ fps with 100% cpu load for the first 30 seconds, then it had dropped to 70 fps when the drive started to have some 40% reading.

Code:
clip = core.ffms2.Source(file)
clip = core.dfttest.DFTTest(clip)
I have tested on both vs and avs+, same behavior.

Can anyone reproduce it?
Sound like (os) caching...
Need more info: ram size, file size, ssd or hdd?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is online now   Reply With Quote
Old 28th July 2019, 23:21   #3550  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,159
Quote:
Originally Posted by ChaosKing View Post
Sound like (os) caching...
Need more info: ram size, file size, ssd or hdd?
ram is 32GB, source video is 6.5G and I tested on both 5400rpm and 7200rpm hdd.
lansing is offline   Reply With Quote
Old 31st July 2019, 03:38   #3551  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
Why is that even a concern?

Encoders typically encode the video in a percentage of that 4000 fps speed. If you are talking about difference between avs/vs to the actual encoding, it is becoming a basis point.

For a 1 hour encode you save not more than 10 seconds.

It might be the way avs/vs fetching and caching the data.
It might be the advantage that avs is C++ compiled while vs is dependent on Python, which in turn depends on the explainator.
It might be ... wait I'm wasting too much time than that 10 seconds.

VapourSynth is powerful for many reasons, but raw speed on absolutely simplest script doesn't fall into the list.

Last edited by littlepox; 31st July 2019 at 03:45.
littlepox is offline   Reply With Quote
Old 31st July 2019, 22:33   #3552  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
R47 RC1

Test it so it's stable. Especially the makediff/mergediff/merge/maskedmerge filters since they were converted to intrinsics from asm.

64bits
32bits

Changes:
Code:
fixed a crash in vdecimate when both dryrun and clip2 is set (no1d)
updated zimg to 2.9.2 to fix a crash that would happen on certain invalid input combinations
improved message handler api and core info api
removed dependency on nasm
various installer improvements including a warning if the vs2019 runtimes aren't installed
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 4th August 2019, 23:20   #3553  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
R47 has been released. The usual blog post here.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 7th August 2019, 16:48   #3554  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,623
Quote:
Originally Posted by Myrsloik
You should never have put things in the vapoursynth subdir. That was reserved for VS and nothing else. Make your own directory if you want to do things that way or simply let VSRepo stuff everything into the right place...
Quote:
Originally Posted by Boulder View Post
I think that once it was required to put your .py files containing your own VS functions under site-packages to get them to autoload so you could just import them in your script. It has worked perfectly fine until R46, hence the confusion.
So what is the proper way of doing things in R47?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 7th August 2019, 21:52   #3555  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
Quote:
Originally Posted by Boulder View Post
So what is the proper way of doing things in R47?
Use vsrepo to install things. Then it's always in the right place.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 8th August 2019, 05:38   #3556  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 81
Quote:
I think that once it was required to put your .py files containing your own VS functions under site-packages to get them to autoload so you could just import them in your script. It has worked perfectly fine until R46, hence the confusion.
It should still work, it is python thing. Your .py file within site-packages of the same Python version you are running. Then it could be imported as module.
With new Python version installed , you move those files as well. So organize it well, always gather them in one directory, like site-packages/vapoursynth/.

I do not know where vsrepo puts those .py files, maybe somewhere else, into AppData.

Last edited by _Al_; 8th August 2019 at 05:40.
_Al_ is offline   Reply With Quote
Old 8th August 2019, 08:34   #3557  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,623
Quote:
Originally Posted by _Al_ View Post
It should still work, it is python thing. Your .py file within site-packages of the same Python version you are running. Then it could be imported as module.
With new Python version installed , you move those files as well. So organize it well, always gather them in one directory, like site-packages/vapoursynth/.

I do not know where vsrepo puts those .py files, maybe somewhere else, into AppData.
I thought so as well, but it doesn't work. The exact same Python version installed (3.7.3) and Vapoursynth R45 works but R47 doesn't.

Using VSRepo is fine, but my own custom .py files need to be taken care of as well and the problem lies there.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 9th August 2019, 00:25   #3558  |  Link
_Al_
Registered User
 
Join Date: May 2011
Posts: 81
For example put just that line into your script:
import os
That would mean you can import standard library modules but not yours? Seems odd. Or it still pulls from somewhere else than you think.
Maybe domain conflict? For example you cannot name your module as subprocess.py, because there is already standard library modul named subprocess, I did it once and error message was not particularly clear about that.
_Al_ is offline   Reply With Quote
Old 12th August 2019, 03:25   #3559  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,978
Quote:
Originally Posted by Boulder View Post
I thought so as well, but it doesn't work. The exact same Python version installed (3.7.3) and Vapoursynth R45 works but R47 doesn't.

Using VSRepo is fine, but my own custom .py files need to be taken care of as well and the problem lies there.
I just upgraded, and it complained something about python not being installed for all users

Short version - I reinstalled python for all users now, but it's in a different location now; so that's where you have to copy over the scripts to the new site-packages location . When you install for "all users" it's Program Files => Python37 => Lib => site-packages (at least on Windows , for x64)
poisondeathray is offline   Reply With Quote
Old 12th August 2019, 05:49   #3560  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,623
The only easy way to make it work was to use the folder which VSRepo uses (%appdata%\Roaming\Python\Python37\site-packages). I've had Python installed in that mentioned directory under Program Files for a long time now but I don't know why it stopped working now.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.