Troubleshooting portable 'install'

[markfilipak]

I seek to install svpflow1,dll & svpflow2.dll. I'm totally new to VapourSynth and don't know AviSynth at all. Python & portable VapourSynth are together here: 'c:\CMD & tiny apps\VapourSynth\'. I plan to pipe to ffmpeg.

I've gotten essentially nowhere. What am I not understanding? Thanks!

C:\CMD & tiny apps\VapourSynth>python vsrepo.py available
Failed to open vspackages3.json. Run update command.

C:\CMD & tiny apps\VapourSynth>update
'update' is not recognized as an internal or external command,
operable program or batch file.

C:\CMD & tiny apps\VapourSynth>dir /s *update*.*
Volume in drive C is \Device\Harddisk0\DR0
Volume Serial Number is D8AD-00FF
File Not Found

[lansing]

the command is

python vsrepo.py update

[markfilipak]

the command is

python vsrepo.py update


Thank you. I ran it and got this:

C:\CMD & tiny apps\VapourSynth>python vsrepo.py update
Traceback (most recent call last):
File "urllib\request.py", line 1350, in do_open
File "http\client.py", line 1255, in request
File "http\client.py", line 1301, in _send_request
File "http\client.py", line 1250, in endheaders
File "http\client.py", line 1010, in _send_output
File "http\client.py", line 950, in send
File "http\client.py", line 921, in connect
File "socket.py", line 808, in create_connection
File "socket.py", line 796, in create_connection
OSError: [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "vsrepo.py", line 909, in <module>
update_package_definition('http://www.vapoursynth.com/vsrepo/vspackages3.zip')
File "vsrepo.py", line 713, in update_package_definition
with urllib.request.urlopen(req_obj) as urlreq:
File "urllib\request.py", line 222, in urlopen
File "urllib\request.py", line 525, in open
File "urllib\request.py", line 542, in _open
File "urllib\request.py", line 502, in _call_chain
File "urllib\request.py", line 1379, in http_open
File "urllib\request.py", line 1353, in do_open
urllib.error.URLError: <urlopen error [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions>


Clue: 'http://www.vapoursynth.com/vsrepo/vspackages3.zip'

I download that. Inside was 'vspackages3.json'. Hurrah! Now, where do I put it?

[markfilipak]

UPDATE:

After putting copies everywhere and failing, I put a copy of 'vspackages3.json' here:
'c:\Users\Administrator\AppData\Roaming\VapourSynth\vsrepo'
and this command:
python vsrepo.py available
then succeeded.

Next, I copied svpflow1.dll & svpflow2.dll to here:
c:\Users\Administrator\AppData\Roaming\VapourSynth\plugins64\svpflow1.dll
c:\Users\Administrator\AppData\Roaming\VapourSynth\plugins64\svpflow2.dll
and I reran this:
python vsrepo.py available
but the resulting list did not include 'svpflow1.dll' & 'svpflow2.dll'.

So autoloading didn't work.

How do I install svpflow1 & svpflow2?

[markfilipak]

UPDATE:

None of these succeeded.

C:\CMD & tiny apps\VapourSynth>python vsrepo.py install svpflow1
Package svpflow1 not found

C:\CMD & tiny apps\VapourSynth>python vsrepo.py install svpflow1.dll
Package svpflow1.dll not found

C:\CMD & tiny apps\VapourSynth>python vsrepo.py install c:\Users\Administrator\AppData\Roaming\VapourSynth\plugins64\svpflow1.dll
Package c:\Users\Administrator\AppData\Roaming\VapourSynth\plugins64\svpflow1.dll not found

C:\CMD & tiny apps\VapourSynth>python vsrepo.py install c:/Users/Administrator/AppData/Roaming/VapourSynth/plugins64/svpflow1.dll
Package c:/Users/Administrator/AppData/Roaming/VapourSynth/plugins64/svpflow1.dll not found

[lansing]

vsrepo is a command line plugin manager so that you don't have to hunt them yourself. You should also use this vsrepogui (https://forum.doom9.org/showthread.php?t=176313) along with it so you get a nice gui and be able to install plugin in one click. vspackage3.json is the plugin info file that vsrepo read from, and it is being keep up to date often.

If you're new, just get an installer, it does everything for you.

[markfilipak]

vsrepo is a command line plugin manager so that you don't have to hunt them yourself.
It's not succeeding.
You should also use this vsrepogui (https://forum.doom9.org/showthread.php?t=176313) along with it so you get a nice gui and be able to install plugin in one click.
I will give it a try of course.
vspackage3.json is the plugin info file that vsrepo read from, and it is being keep up to date often.
vspackage3.json is not in the portable package -- why isn't it? I've tried to follow the instructions in the documentation step by step, but it's partly wrong and partly missing. An error message pointed me to a vspackage3.json download and from there I hacked where to put vspackage3.json:
c:\Users\Administrator\AppData\Roaming\VapourSynth\vsrepo\
The developers appear to have bollixed the portible and have included parts of the installer version. It's a mess.

If you're new, just get an installer, it does everything for you.
I'm running in Windows. It does not have access to the Internet.

[markfilipak]

... You should also use this vsrepogui (https://forum.doom9.org/showthread.php?t=176313) ...

This is getting to be quite a tragic comedy.

I put VSRepoGUI.exe here:
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe

When I ran it, I got this:
"Can not find your VapourSynth installation(s)."
It's in the VapourSynth directory!

Oh, brother.

[lansing]

You can't use vsrepo without the internet, I mean where else can it get the plugins?

[markfilipak]

You can't use vsrepo without the internet, I mean where else can it get the plugins?
Erm... Direct download in Linux and then sneakernet?

[ChaosKing]

In your portable version you should have python.exe vsrepo.py and vsrepogui.exe in the same folder => inside "VapourSynth64" (I assume you're using 64bit)
But vsrepo won't help you here as svpflow is not in the repository. That means there no entry in vspackage3.json for where to download it.

Alternative: You can try the "VapourSynth Portable FATPACK" here https://forum.doom9.org/showthread.php?t=175529
Or look at the build script to see which steps are necessary to make a "portable" version https://github.com/theChaosCoder/vapoursynth-portable-FATPACK/blob/master/build.ps1

The FATPACK contains svpflow1_vs64.dll & svpflow2_vs64.dll so I guess it should work out of the box I guess. I never tried this filter.

[markfilipak]

In your portable version you should have python.exe vsrepo.py and vsrepogui.exe in the same folder => inside "VapourSynth64" (I assume you're using 64bit)
c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\doc\
c:\CMD & tiny apps\VapourSynth\sdk\
c:\CMD & tiny apps\VapourSynth\vapoursynth64\
But vsrepo won't help you here as svpflow is not in the repository. That means there no entry in vspackage3.json for where to download it.

Alternative: You can try the "VapourSynth Portable FATPACK" here https://forum.doom9.org/showthread.php?t=175529
Or look at the build script to see which steps are necessary to make a "portable" version https://github.com/theChaosCoder/vapoursynth-portable-FATPACK/blob/master/build.ps1

The FATPACK contains svpflow1_vs64.dll & svpflow2_vs64.dll so I guess it should work out of the box I guess. I never tried this filter.
Thanks for that. Thanks so much. I'll give it a go.

[markfilipak]

Alternative: You can try the "VapourSynth Portable FATPACK" here https://forum.doom9.org/showthread.php?t=175529
The github download page is:
'https://github.com/theChaosCoder/vapoursynth-portable-FATPACK/releases'.

I'm looking for 'vapoursynth-portable-FATPACK', right? All I can see there is this:
VapourSynth64Portable_2020_09_06.7z
But I already have this:
VapourSynth64-Portable-R52.7z
'installed' (actually, copied to 'c:\CMD & tiny apps\VaporSynth\').

[ChaosKing]

c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\doc\
c:\CMD & tiny apps\VapourSynth\sdk\
c:\CMD & tiny apps\VapourSynth\vapoursynth64\

Thanks for that. Thanks so much. I'll give it a go.

This is not a correct "vapoursynth portable" configuration.

You basically need to throw everything into vapoursynth64.


The github download page is:
'https://github.com/theChaosCoder/vapoursynth-portable-FATPACK/releases'.

I'm looking for 'vapoursynth-portable-FATPACK', right? All I can see there is this:
VapourSynth64Portable_2020_09_06.7z
But I already have this:
VapourSynth64-Portable-R52.7z
'installed' (actually, copied to 'c:\CMD & tiny apps\VaporSynth\').
VapourSynth64-Portable-R52.7z is just the vapoursynth part. You need python as well. Look at the structure in VapourSynth64Portable_2020_09_06.7z to understand what belongs where :-)

[markfilipak]

UPDATE

I went ahead and downloaded VapourSynth64Portable_2020_09_06.7z. The is no vspackages3.json in it. So I'm obviously doing something wrong.

[markfilipak]

VapourSynth64-Portable-R52.7z is just the vapoursynth part. You need python as well.
I knew that. I had already copied the embedded version of python into 'c:\CMD & tiny apps\VapourSynth\'.

[ChaosKing]

UPDATE

I went ahead and downloaded VapourSynth64Portable_2020_09_06.7z. The is no vspackages3.json in it. So I'm obviously doing something wrong.

Why do you need one? If you don't have internet access it's useless... It is only used by vsrepo to download plugins.

With
python vsrepo.py update
you can download the latest version of vspackages3.json
That means it does not make sense to bundle an old version of the package file.

[markfilipak]

c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\doc\
c:\CMD & tiny apps\VapourSynth\sdk\
c:\CMD & tiny apps\VapourSynth\vapoursynth64\
This is not a correct "vapoursynth portable" configuration.

You basically need to throw everything into vapoursynth64.

I'm confused. A vapoursynth64 directory inside a vapoursynth64 directory inside 'c:\CMD & tiny apps\VapourSynth\'? I must be misunderstanding you.

1st- & 2nd-level directories of VapourSynth64Portable_2020_09_06.zip

\apps_other\
\bin\
\extras\
\Scripts\
\VapourSynth64\
\VapourSynth64\bearer\
\VapourSynth64\doc\
\VapourSynth64\iconengines\
\VapourSynth64\imageformats\
\VapourSynth64\Lib\
\VapourSynth64\platforms\
\VapourSynth64\Scripts\
\VapourSynth64\sdk\
\VapourSynth64\sdk\examples\
\VapourSynth64\sdk\include\
\VapourSynth64\sdk\lib32\
\VapourSynth64\sdk\lib64\
\VapourSynth64\Settings\
\VapourSynth64\styles\
\VapourSynth64\vapoursynth64\

[markfilipak]

c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\doc\
c:\CMD & tiny apps\VapourSynth\sdk\
c:\CMD & tiny apps\VapourSynth\vapoursynth64\
This is not a correct "vapoursynth portable" configuration.
What I did was simply copy the contents of VapourSynth64-Portable-R52.7z into 'c:\CMD & tiny apps\VapourSynth\'.

That was wrong, eh?

[ChaosKing]

c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\doc\
c:\CMD & tiny apps\VapourSynth\sdk\
c:\CMD & tiny apps\VapourSynth\vapoursynth64\

What I did was simply copy the contents of VapourSynth64-Portable-R52.7z into 'c:\CMD & tiny apps\VapourSynth\'.

That was wrong, eh?

Ok maybe I added some confusion with VapourSynth64.

It seems you had it already. So in your case: your VapourSynth folder = my VapourSynth64


c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\<contents of VapourSynth64-Portable-R52.7z>

[markfilipak]

Ok maybe I added some confusion with VapourSynth64.

It seems you had it already. So in your case: your VapourSynth folder = my VapourSynth64


c:\CMD & tiny apps\VapourSynth\python.exe
c:\CMD & tiny apps\VapourSynth\vsrepo.py
c:\CMD & tiny apps\VapourSynth\VSRepoGUI.exe
c:\CMD & tiny apps\VapourSynth\<contents of VapourSynth64-Portable-R52.7z>

Now I'm even more confused. ... :-)
Your setup has directories relative to the 1st level VapourSynth64 directory as '..\apps_other\' and '..\bin\' and '..\extras\' and '..\Scripts\'
so I'm lost (and that structure may break if I change it). Kindly advise. I'll do whatever you advise. I just want success, not necessarily beauty.

Edit: You see why I prefer installers for Windows and portables for Linux?

[ChaosKing]

Anti confusion spell
https://i.imgur.com/4YhxGE8.png


Edit: You see why I prefer installers for Windows and portables for Linux?
There is a installer for vapoursynth ;)

[markfilipak]

Anti confusion spell
(https://i.imgur.com/4YhxGE8.png)
I don't know what you're trying to show me.


There is a installer for vapoursynth ;)
I couldn't use it. I don't know whether the problem was Python or the C++ runtimes [1]. I was advised to use the portable version of VS and put the VS stuff and the python stuff together.

[1] I do have the C++ 2019 runtimes installed but the VS install nonetheless failed. I was advised to download the runtimes again, but the runtimes are at the Microsoft Store -- the Microsoft Store is inaccessable to me because Windows doesn't have access to the Internet (which makes the Microsoft Store useless).

[ChaosKing]

You were confused by the folder structure. So I showed you the correct one, so you can compare it with yours...

You can download 2019 runtimes without the store. They were never ever in the ms store.

Here is a direct link https://download.visualstudio.microsoft.com/download/pr/b29e54d1-216d-423a-8160-742b2ec94cc1/FA3E2A414D47D2C9691719795E5A3C66E515D0FFF9C3144D7DDBDFD0351F74DB/VC_redist.x64.exe
Or download a bundle here with all latest runtimes here https://github.com/abbodi1406/vcredist/releases

[markfilipak]

You were confused by the folder structure. So I showed you the correct one, so you can compare it with yours...
Not to burden the subject... Your zip has 4 directories below the 1st-level vapoursynth46 directory which will put them below 'c:\CMD & tiny apps\VapourSynth\'... Oh, never mind, I'll fix up the links and batch files. I hope there's nothing that'll break.

You can download 2019 runtimes without the store. They were never ever in the ms store.
You mean someone gave me bogus info? Shocking!

Here is a direct link https://download.visualstudio.microsoft.com/download/pr/b29e54d1-216d-423a-8160-742b2ec94cc1/FA3E2A414D47D2C9691719795E5A3C66E515D0FFF9C3144D7DDBDFD0351F74DB/VC_redist.x64.exe
Or download a bundle here with all latest runtimes here https://github.com/abbodi1406/vcredist/releases
Thanks.

Hmm... It sure looks like you've already packaged python in your VapourSynth64Portable_2020_09_06.7z

[ChaosKing]

Hmm... It sure looks like you've already packaged python in your VapourSynth64Portable_2020_09_06.7z
It's a ready to use package for beginners, so you don't have to "install" or download everything yourself. It has plugins, script, vsedior and some examples on how to encode with x265 x264

EDIT:
All you need is to extract it (maybe have runtimes installed, but whats it :D)

[markfilipak]

ChaosKing. I'm in trouble.

I wiped my previous portable VS and copied yours in its place. I need to get svpflow going, but look:

C:\CMD & tiny apps\VapourSynth\VapourSynth64>python vsrepo.py installed
Name Namespace Installed Latest Identifier
...svpflow1_vs64.dll is not there.
...svpflow2_vs64.dll is not there.

C:\CMD & tiny apps\VapourSynth\VapourSynth64>python vsrepo.py available
Name Namespace Installed Latest Identifier
...svpflow1_vs64.dll is not there.
...svpflow2_vs64.dll is not there.

C:\CMD & tiny apps\VapourSynth\VapourSynth64>python vsrepo.py install svpflow1_vs64
Package svpflow1_vs64 not found

C:\CMD & tiny apps\VapourSynth\VapourSynth64>python vsrepo.py install svpflow1_vs64.dll
Package svpflow1_vs64.dll not found

C:\CMD & tiny apps\VapourSynth\VapourSynth64>python vsrepo.py install "c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll"
Package c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll not found

These DO exist:
c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll
c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow2_vs64.dll

What is wrong?

[ChaosKing]

You should read my posts more carefully https://forum.doom9.org/showthread.php?p=1935533#post1935533

The FATPACK already contains svpflow.dll . It's called fat pack for a reason, because it already contains most plugins and scripts.
vsrepo can not install/download svpflow because the plugin is NOT in the vsrepo repository. It does not know about svpflow plugin at all.

[markfilipak]

You should read my posts more carefully https://forum.doom9.org/showthread.php?p=1935533#post1935533

The FATPACK already contains svpflow.dll.
I know that. You wrote that.
It's called fat pack for a reason, because it already contains most plugins and scripts.
And I know that, too.
vsrepo can not install/download svpflow because the plugin is NOT in the vsrepo repository. It does not know about svpflow plugin at all.

Thank you. You're assuming I know what you know. I don't. I'm trying to follow the instructions in the documentation. It says to run 'python vsrepo.py available'. I've jumpped the line a bit by also running 'python vsrepo.py installed'. Those svp filters aren't listed in either list.

[markfilipak]

You should read my posts more carefully https://forum.doom9.org/showthread.php?p=1935533#post1935533

ChaosKing. I appreciate that you care for novices so much that you created your own distribution and I fear that my responses may have discouraged you. I sincerely apologize for any perceived discredit. You don't deserve that and I didn't mean that. Kindly let me illuminate my problems as I attempt to hack VapourSynth use.

I found these posts that include "svpflow1_vs64.dll":

https://www.svp-team.com/forum/viewtopic.php?id=3917
https://forum.selur.net/showthread.php?tid=1422&page=2
https://github.com/staxrip/staxrip/issues/83#issue-517054016
https://github.com/Revan654/staxrip/issues/60
https://forum.videohelp.com/threads/392552-Soccer-matches-CBR-vs-VBR-decision/page2#post2545675
https://forum.doom9.org/showthread.php?t=181597
http://forum.doom9.org/archive/index.php/t-164554-p-2.html

The problem with them is that they have been submitted by users who are in trouble. So, which of them are good code and which of them are bad code? I don't know. But given that their correctness is in doubt, can they serve as examples? I don't think so. As I read the cited threads it is clear to me that the people who submitted them are groping and that the people who responded don't see why.

I've previously cited the instructions that appear here: http://www.vapoursynth.com/doc/. What more could be needed, eh? To the informed, everything a novice might need appears to be right there, before them.
What's missing are example scripts that are known to be correct and are known to work.
What's missing are details of the VapourSynth architecture and how it initializes and how it processes commands so that novices can gain understanding.
What's missing are samples that novices can run that begin simple and proceed to complicated (just as such samples usually appear in "How To" & "For Dummies" books) so that novices can go from success to success and thereby learn VapourSynth's requirements and operation and gain confidence. They need illuminative samples. They need a lot of them.

I know that the documentation is not your responsibility. But I suspect you have (or could) influence it.

You cited https://forum.doom9.org/showthread.php?p=1935533#post1935533. In that post, you allude to features of vsrepo that I can only guess about. I'm not stupid. I'm ignorant.

I now have a portable installation that, thanks to you, is complete (I presume).

I have the VapourSynth documentation. That documentation sets out descriptions of the parts of VapourSynth, but mere descriptions of the parts is inadequate. Novices need to know how the parts go together. They need to know how to form work flows that do the things they need done.

Now, based on clues. I'm going to presume that svpflow1_vs64.dll & svpflow2_vs64.dll don't need to be installed -- that their mere presence in .\vapoursynth64\vapoursynth64\plugins\ will suffice. Based on clues, I believe this is correct, but there's nothing in the documentation that supports that belief. Certainly, any Windows application with which I'm familiar has no such "mere presence" feature.

Despite my "mere presence" presumptions, I still have no confidence that I can create viable python scripts. I continue to struggle.

[stax76]

Nothing wrong with having ambitions but for this kind of task you are simply not ready.

[markfilipak]

Nothing wrong with having ambitions but for this kind of task you are simply not ready.
You're exactly correct. Like all novices, I'm not ready. How do I get ready? Where do I find known-good sample scripts? What elements of scripts perform what functions? I don't know the architecture. What are the objects and methods that VS adds? How are they used? Is there a browsing tool that I can use to discover objects and methods?

I'm seeking to pipe to ffmepg. Until today, here are the best fragments I could find while searching:
'python SCRIPT.vpy | ffmpeg -f yuv4mpegpipe -'
'vspipe --y4m SCRIPT.vpy - | ffmpeg -i pipe:'
'ffmpeg -f vapoursynth -i SCRIPT.vpy' -- this can't work because ffmpeg distros aren't compiled with vapoursynth support.

Today, I stumbled across this:
'vspipe --y4m SCRIPT.vpy - | ffmpeg -i pipe: -i SOURCE -c:v prores -profile:v 3 -c:a copy -map 0:0 -map 1:1 TARGET'

So, I guess that the ouput of vspipe is raw frames? So, ffmpeg's input is raw frames?

The VS documentation is written by people who intimately understand the VS architecture. I'm ignorant. I'm reduced to guessing.

Have compassion, please. I'm not stupid or inexperienced with either object-oriented languages or with python (though it has been several years since I studied python). I'm just ignorant.

[lansing]

So you're just confused about installing plugins.

Here's a quick summary. Vapoursynth uses two kind of plugins, normal vs plugins (*.dll) and python scripts (*.py). To install them you just have to grab the dll and put them in their respective folders. The dll goes to path like vapoursynth/plugin64 folder and python scripts goes to the python folder, mostly these two:


C:\Users\your name\AppData\Roaming\VapourSynth\plugins64
C:\Users\your name\AppData\Roaming\Python\Python38\site-packages


And you're done. The vsrepo is there to do all of those work for you, like download/extracting/copying/pasting all the plugins. But since you don't have internet, this is not an option for you. You'll have to do all those things manually. And the documentation did provide a script for getting started (http://www.vapoursynth.com/doc/gettingstarted.html). To preview the script you'll need Vapoursynth editor.

[markfilipak]

'Mark's_script_1.cmd':
vspipe --y4m Mark's_script_1.vpy - | ffmpeg -i pipe: Mark's_script_1.mkv

'Mark's_script_1.vpy' (based on 'Getting Started', 'Example Script'):
from vapoursynth import core
video = core.ffms2.Source(source='Mark\'s_source.mkv')
video.set_output()

Well, what do you know. My first success.

And the result of 'Script check':
Script was successfully evaluated. Output video info:
Frames: 480 | Time: 0:00:20.020 | Size: 1080x1920 | FPS: 24000/1001 = 23.976 | Format: YUV420P8

What is this: 'Mark's_script_1.mkv.ffindex'? "ffindex" is not in the documentation.

'Mark's_script_2.cmd':
vspipe --y4m Mark's_script_2.vpy - | ffmpeg -i pipe: -map 0 -codec:v libx265 -x265-params "crf=16:qcomp=0.60" -codec:a copy -codec:s copy Mark's_script_2.mkv

'Mark's_script_2.vpy':
from vapoursynth import core
video = core.ffms2.Source(source='Mark\'s_source.mkv')
core.std.LoadPlugin(path="c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll")
core.std.LoadPlugin(path="c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow2_vs64.dll")
import havsfunc as havsfunc
video = havsfunc.InterFrame(video, Preset="fastest", NewNum=60000, NewDen=1001, GPU=True)
video.set_output()

Failed with error:
C:\CMD & tiny apps\VapourSynth\VapourSynth64>vspipe --y4m Mark's_script_2.vpy - | ffmpeg -i pipe: -map 0 -codec:v libx265 -x265-params "crf=16:qcomp=0.60" -codec:a copy -codec:s copy Mark's_script_2.mkv
ffmpeg started on 2021-02-10 at 21:27:30
Report written to "ffmpeg-20210210-212730.log"
Log level: 32
ffmpeg version N-100851-g9f38fac053 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 9.3-win32 (GCC) 20200320
configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static --pkg-config=pkg-config --cross-prefix=x86_64-w64-mingw32- \
--arch=x86_64 --target-os=mingw32 --enable-gpl --enable-version3 --disable-debug --disable-w32threads --enable-pthreads \
--enable-iconv --enable-zlib --enable-libxml2 --enable-libfreetype --enable-libfribidi --enable-gmp --enable-lzma --enable-fontconfig \
--enable-opencl --enable-libvmaf --enable-vulkan --enable-libvorbis --enable-amf --enable-libaom --enable-avisynth --enable-libdav1d \
--enable-libdavs2 --enable-ffnvcodec --enable-cuda-llvm --enable-libglslang --enable-libass --enable-libbluray --enable-libmp3lame \
--enable-libopus --enable-libtheora --enable-libvpx --enable-libwebp --enable-libmfx --enable-libopencore-amrnb --enable-libopencore-amrwb \
--enable-libopenjpeg --enable-librav1e --enable-librubberband --enable-schannel --enable-sdl2 --enable-libsoxr --enable-libsrt \
--enable-libsvtav1 --enable-libtwolame --enable-libuavs3d --enable-libvidstab --enable-libx264 --enable-libx265 --enable-libxavs2 \
--enable-libxvid --enable-libzimg --extra-cflags=-DLIBTWOLAME_STATIC --extra-cxxflags= --extra-ldflags=-pthread --extra-libs=-lgomp
libavutil 56. 64.100 / 56. 64.100
libavcodec 58.119.100 / 58.119.100
libavformat 58. 65.101 / 58. 65.101
libavdevice 58. 11.103 / 58. 11.103
libavfilter 7.100.100 / 7.100.100
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100
Script evaluation failed:
Python exception: Failed to load c:\CMD & tiny apps\VapourSynth\VapourSynth64apoursynth64\plugins\svpflow1_vs64.dll. GetLastError() returned 126. The file you tried to load or one of its dependencies is probably missing.

Traceback (most recent call last):
File "src\cython\vapoursynth.pyx", line 2244, in vapoursynth.vpy_evaluateScript
File "src\cython\vapoursynth.pyx", line 2245, in vapoursynth.vpy_evaluateScript
File "Mark's_script_2.vpy", line 3, in <module>
core.std.LoadPlugin(path="c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll")
File "src\cython\vapoursynth.pyx", line 2069, in vapoursynth.Function.__call__
vapoursynth.Error: Failed to load c:\CMD & tiny apps\VapourSynth\VapourSynth64apoursynth64\plugins\svpflow1_vs64.dll. GetLastError() returned 126. The file you tried to load or one of its dependencies is probably missing.

pipe:: Invalid data found when processing input

What do you make of this part:
Python exception: Failed to load c:\CMD & tiny apps\VapourSynth\VapourSynth64apoursynth64\plugins\svpflow1_vs64.dll.
In the actual text (but not in the preview of this posting), "\VapourSynth64apoursynth64" is actually "\VapourSynth64_apoursynth64" where _ is supposed to be "\v" but is actually 0x000B. What? is \v some sort of magic word?

[poisondeathray]

You probably need to escape the amepersand character in the path

You can use better path names and avoid symbols

or

Another way is to use the r' notation for paths


core.std.LoadPlugin(r'c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll')
core.std.LoadPlugin(r'c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow2_vs64.dll')

[markfilipak]

from vapoursynth import core
video = core.ffms2.Source(source='Mark\'s_source.mkv')
core.std.LoadPlugin(r'c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll')
core.std.LoadPlugin(r'c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow2_vs64.dll')
import havsfunc as havsfunc
video = havsfunc.InterFrame(video, Preset="fastest", NewNum=60000, NewDen=1001, GPU=True)
video.set_output()
I got this:
Python exception: Plugin c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll already loaded (com.svp-team.flow1) from C:/CMD & tiny apps/VapourSynth/VapourSynth64/vapoursynth64/plugins/svpflow1_vs64.dll

[poisondeathray]

If they are already auto loaded, then comment out those LoadPlugin lines or delete them



Code:

from vapoursynth import core
video = core.ffms2.Source(source='Mark\'s_source.mkv')
#core.std.LoadPlugin(r'c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow1_vs64.dll')
#core.std.LoadPlugin(r'c:\CMD & tiny apps\VapourSynth\VapourSynth64\vapoursynth64\plugins\svpflow2_vs64.dll')
import havsfunc as havsfunc
video = havsfunc.InterFrame(video, Preset="fastest", NewNum=60000, NewDen=1001, GPU=True)
video.set_output()

[markfilipak]

Thank you, friend.

[poisondeathray]

What is this: 'Mark's_script_1.mkv.ffindex'? "ffindex" is not in the documentation.


This is an index file

The source filter you used to load and decode the file, ffms2 produces an index for frame accuracy and seek purposes.

This is in the ffms2 official documentation, but it might not be included in the bundled plugin you downloaded

When you first load a file or run the script, it will pause as it's indexing the file. A large file can take a few minutes depending on your hardware specs, it's normal behaviour. Subsequent accesses do not require re-indexing the file

[markfilipak]

This is an index file

The source filter you used to load and decode the file, ffms2 produces an index for frame accuracy and seek purposes...
Ah! I think I remember reading something like that, and virtues (if any) of pre-indexing.

Thanks again.

You and ChaosKing are so generous with your time I hesitate to ask more questions. But in an effort to find primary sources, do you know how I can query the capabilities of InterFrame? I tried a textual search of all the files for "havsfunc" hoping to extract strings from the file (much as you might extract the version number from an executable's StringFileInfo), but the search didn't get any hits. I have 2, slightly differing copies of 'InterFrame2.avsi' (same version #: 2.8.2 by SubJunk) and that's helped, but it seems from them that the 'Cores=<n>' option is not there. I don't know whether you diddle with InterFrame.

[poisondeathray]

But in an effort to find primary sources, do you know how I can query the capabilities of InterFrame? I tried a textual search of all the files for "havsfunc" hoping to extract strings from the file (much as you might extract the version number from an executable's StringFileInfo), but the search didn't get any hits. I have 2, slightly differing copies of 'InterFrame2.avsi' (same version #: 2.8.2 by SubJunk) and that's helped, but it seems from them that the 'Cores=<n>' option is not there. I don't know whether you diddle with InterFrame.


for "capabilities" did you mean settings or switches?

cores=<n> is for the avisynth version, running on CPU , with GPU=false

For vapoursynth, using CPU, there is automatic threading so cores=<n> switch is irrelevant. Interframe for vapoursynth was a direct port from the avisynth version, and the avisynth version has a html documentation file for the settings , and for the most part it's applicable and "human readable"

But the best way for that type of info is to look at the havsfunc.py for any information because that's the actual version you are using. Open it up in notepad++ or similar text edtior. Yes, it's not exactly user friendly readable, but it gives you the actual switches you can use. Look at the InterFrame section and compare with the avs html docs

Interframe is a convenience wrapper function, to simply commands . It does not give you access to all the individual settings - if wanted explictly change the horizontal or vertical block size or subpixel motion estimation, forward or backward vectors etc.. you'd have to use svpflow or mvtools outside of interframe

[ChaosKing]

Yep. Easiest way is too look at the parameters here https://github.com/HomeOfVapourSynthEvolution/havsfunc/blob/e56f8f787bd4b679e6d14c9b08bf0460a3480ea4/havsfunc.py#L3834


def InterFrame(Input, Preset='Medium', Tuning='Film', NewNum=None, NewDen=1, GPU=False, InputType='2D', OverrideAlgo=None, OverrideArea=None, FrameDouble=False):
....
if Preset not in ['medium', 'fast', 'faster', 'fastest']:
raise vs.Error(f"InterFrame: '{Preset}' is not a valid preset")

if Tuning not in ['film', 'smooth', 'animation', 'weak']:
raise vs.Error(f"InterFrame: '{Tuning}' is not a valid tuning")

if InputType not in ['2D', 'SBS', 'OU', 'HSBS', 'HOU']:
raise vs.Error(f"InterFrame: '{InputType}' is not a valid InputType")

And at the avisynth doc https://www.spirton.com/uploads/InterFrame/InterFrame2.html

[markfilipak]

for "capabilities" did you mean settings or switches?
I guess what I meant was how do I 'talk' to SVSuper, SVAnalyse, SVConvert, and SVSmoothFps (though I didn't know their names at the time).
Interframe is a convenience wrapper function, to simply commands . It does not give you access to all the individual settings - if wanted explictly change the horizontal or vertical block size or subpixel motion estimation, forward or backward vectors etc.. you'd have to use svpflow or mvtools outside of interframe
Yes, thanks, I understand.

I've made a lot of slow progress. FYI, I've diddled InterFrame extensively... maybe 50 differing tests. I've hit some limits...

You undoubtedly tread my path long ago, and I don't want to burden you, so read the '====='-delimited section quickly just to get the flavor.

=====

The source video is 23.976fps, 1920x1080. It has strong 'picket fence'-type backgrounds during panning shots that track a walking man -- the man appears stationary as the background pans behind him -- then he stops. There appears to be 3 problems:

1, After the man stops, the 'picket fence' then exhibits large, flashing, 'phantoms': single-frame, blocky reverse-image areas that are somewhat like afterimages. The 'phantoms' are nowhere near the man but are way off to the right, overlaying parts of the 'picket fence' -- it's actually a wooden Chinese screen but 'picket fence' is the term I've read in Doom9 Avisynth posts from 2017. Curiously, the 'phantom' affects only the verticals in the 'picket fence', not the horizontals (which remain sharp and well defined). During the 'phantom' image distortions, the 'picket fence' is either stationary or is moving very slightly (almost pel-fractionally).

2, Before the man stops (i.e. as he 'walks' during the panning shot), he seems to 'drag' the patterned background with him, but only within the approximately 20 to 30 pels that directly surround his body. This is what the SVSmoothFps documentation calls 'halo'.

The 2 problems above exist for all values of 'SVSmoothFps.algo:' (i.e. 'algo:0', 'algo:1', 'algo:11', 'algo:13', 'algo:21', and 'algo:23').

If I raise 'SVSmoothFps.mask.area:' (default: 0) to 100, then 200, then 300, then 400, the 'phantom' and the 'halo' are both substantially less noticeable at 'area:200', then are overcome (but not really) at 'area:300'. However, doing so exposes the 3rd problem.

3, At higher 'area' settings, pixel blending seems to kick in (even for 'algo:0' and 'algo:1'). Inside the blending, shiny highlights on vertical columns in the scene begin to judder at 'area:300' and get worse at 'area:400'.

At 'area:300', single stepping frame-by-frame shows that the judder is contained within the general area of the vertical highlights -- that the highlights are 'vibrating' -- but the judder doesn't affect the sides of the columns.

Then, if 'area:' is raised to 400, the actual bodies and sides of the columns themselves exhibit small but substantial judder. Oddly, the judder is at 24 judders/s., not at 60 judders/s.

So it appears that I can either avoid judder at 'area:' values below 300 (and than suffer some 'phantoms'), or I can avoid 'phantoms' at 'area:' values above 300 (and then suffer some judder), but there is no 'sweet spot' that avoids both 'phantoms' and judder. Both 'phantoms' and 'judder' (and to a lesser extent, 'halo's) appear to depend on video image content, not rates or other frame mechanics, so trying to find a 'sweet spot' that works for all videos is probably a fool's errand.

Regarding the 'halo's, higher 'area:' values don't have much effect. What does have effect is 'algo' values. 'algo:11', 'algo:13', 'algo:21', and 'algo:23' fuzz the 'halo's, but they are there for all 'algo:'s.

For what it's worth, I don't like the fuzzing that 'algo:11', 'algo:13', 'algo:21', and 'algo:23' do. I consider motion vector interpolation to be a mechanical approach while the fuzzing is a cosmetic approach. I don't know why the higher 'algo:' settings apply any cosmetics and I wish they didn't.

=====

Here's the 'meat' of this meal:

Issue #1:
Can I call SVSmoothFps (and perhaps SVSuper, SVAnalyse, and SVConvert) myself, outside of InterFrame? Would I need to write my own InterFrame-style python script? Would I put that script into Scripts\havsfunc.py? Would I need to 'install' it? Or would I 'compile' it into Scripts\__pycache__\havsfunc.cpython-38.pyc? Generally, how would I make VapourSynth 'aware' of its existence? (A link to documentation would do, and if it's already there, I apologize for asking but I'm getting pretty wasted groping in the dark).

Issue #2:
It appears that SVSmoothFps.scene.mode = '1m' interpolates (via MVs) from 24fps to 48fps (and then repeats some of the interpolated frames to get to 60fps), or that SVSmoothFps.scene.mode = '2m' interpolates 24fps to 72fps (and then discards some of the interpolated frames to get to 60fps). I think something better can be done: 1-to-5, direct interpolation from 24fps to 60fps via scaled, temporal vector interpolation (like a weighted 'mix' filter in ffmpeg, and without 'dynamic medians'). I think that doing so could eliminate/avoid all of the 'phantom' 'halo' and judder problems I've seen.

In that regard, do you know of anyone who would like to help and who might like a bounty?

Warm Regards,
Mark.

[poisondeathray]

Both 'phantoms' and 'judder' (and to a lesser extent, 'halo's) appear to depend on video image content, not rates or other frame mechanics, so trying to find a 'sweet spot' that works for all videos is probably a fool's errand.

Yes, those are typical "fail" scenarios for optical flow - all kinds - even expensive professional OF solutions

One set of settings that improves frames from one section, will make another section worse.

When it works, it can be ok. But sometimes artifacts are too distracting. Either live with it, or don't do anything.





Here's the 'meat' of this meal:

Issue #1:
Can I call SVSmoothFps (and perhaps SVSuper, SVAnalyse, and SVConvert) myself, outside of InterFrame? Would I need to write my own InterFrame-style python script? Would I put that script into Scripts\havsfunc.py? Would I need to 'install' it? Or would I 'compile' it into Scripts\__pycache__\havsfunc.cpython-38.pyc? Generally, how would I make VapourSynth 'aware' of its existence? (A link to documentation would do, and if it's already there, I apologize for asking but I'm getting pretty wasted groping in the dark).



The SVPFlow plugins need to be loaded; you already have them loaded if you used interframe before

There are example scripts on the site
https://www.svp-team.com/wiki/Manual:SVPflow#Basic_Vapoursynth_script



Issue #2:
It appears that SVSmoothFps.scene.mode = '1m' interpolates (via MVs) from 24fps to 48fps (and then repeats some of the interpolated frames to get to 60fps), or that SVSmoothFps.scene.mode = '2m' interpolates 24fps to 72fps (and then discards some of the interpolated frames to get to 60fps). I think something better can be done: 1-to-5, direct interpolation from 24fps to 60fps via scaled, temporal vector interpolation (like a weighted 'mix' filter in ffmpeg, and without 'dynamic medians'). I think that doing so could eliminate/avoid all of the 'phantom' 'halo' and judder problems I've seen.



Repeating patterns aka the protoypical "picket fence" scenario - causes problems with ALL optical flow algorithms. The solution in other programs is some manual guidance - motion tracking and masking. Motion tracking to guide the estimation, masking to indicate object boundaries and reduce boundary artifacts. It's just not possible with some automatic script - You will always get some artifacts.

"AI" research methods such as DAIN, RIFE can sometimes produce cleaner results, especially on rotational vectors, but the "picket fence" issues are still very much a problem with them too .

Object boundaries and occlusions are still very much a problem for all algorithms. It's just impossible for computer at this point in time to track and distinguish when objects change shape or go behind other objects and reappear



For cadence issues, svpflow is closed plugin. You can ask in the svpflow forum.

But MVTools2 has "time" parameter for functions like mflowinter, where you should be able to adjust the phase for interpolated frames


time

Interpolation time position between frames, in percent. Default value of 50.0 is half-way.