Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th March 2018, 04:47   #1  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 531
ffmpeg using .vpy scripts directly one day, like avisynth ?

Do you think there's any chance of ffmpeg one day building with something like --enable-vapoursynth so that it can consume .vpy scripts directly, sort of like avisynth ? Is there a feature in the works of any kind ?

It could mean that VS may need to be amenable to cross-compilation, so I figure the chances are remote. What do you reckon ?

Yes there's vspipe (video, no audio unless I'm mistaken ?) and HDR can be piped per this https://forum.doom9.org/showthread.p...37#post1831137

This person had a start at having a go with what appears to be limited success
Code:
'vapoursynth': { # not cross compiling ( yet )
and
Code:
'vapoursynth_libs': {
https://github.com/DeadSix27/python_...ss_compiler.py

Last edited by hydra3333; 24th March 2018 at 05:25.
hydra3333 is offline   Reply With Quote
Old 24th March 2018, 09:11   #2  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,882
sure it's possible in theory, but whether someone is motivated enough to implement it is a totally different thing
-> either implement it yourself and send a patch to the ffmpeg dev team or create a feature request through the official channels and hope for the best

The repository you link there doesn't seem to aim to add Vapoursynth support to ffmpeg.
It's compiles for example MPV on Linux for Windows.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 24th March 2018 at 09:13.
Selur is offline   Reply With Quote
Old 24th March 2018, 09:39   #3  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
That must be a very old comment about cross compiling, I know people who've cross compiled it many times to compare gcc vs msvc for speed. Time to write the code I guess...
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 24th March 2018, 14:07   #4  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 531
Quote:
Originally Posted by Myrsloik View Post
That must be a very old comment about cross compiling, I know people who've cross compiled it many times to compare gcc vs msvc for speed.
Oh. Would you be able to name someone here, or maybe post or a github link or something ?

Quote:
Originally Posted by Myrsloik View Post
Time to write the code I guess...
hydra3333 is offline   Reply With Quote
Old 25th March 2018, 13:30   #5  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 725
The core library and the included plugins (minus ocr) can be cross-compiled pretty easily. You pass
Code:
--host=x86_64-w64-mingw32
to configure.

The Python module is the annoying part, because you must also cross-compile Python, and I think the Python developers don't make it easy.

But this is all irrelevant, because VapourSynth has a C API, so you can just use the official binaries compiled with MSVC. You don't have to cross-compile it for ffmpeg.

Correction: VapourSynth has a reasonable C API, unlike Python.
__________________
Buy me a "coffee" and/or hire me to write code!

Last edited by jackoneill; 25th March 2018 at 13:35.
jackoneill is offline   Reply With Quote
Old 25th March 2018, 14:07   #6  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,882
Also instead of cross compiling you could also compile it on Windows using media-autobuild_suite.
(Cross-Compiling won't allow ffmpeg to .vpy files so this might not be what you are looking for.)
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 25th March 2018 at 14:10.
Selur is offline   Reply With Quote
Old 27th March 2018, 06:01   #7  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 531
Quote:
Originally Posted by Selur View Post
Also instead of cross compiling you could also compile it on Windows using media-autobuild_suite.
(Cross-Compiling won't allow ffmpeg to .vpy files so this might not be what you are looking for.)
Thank you. Yes I've used jb's under Windows - and may still resort to that if cross-compilation of a static ffmpeg and its dependencies (3rd party libraries) is out of the question.

My objective was to cross-compile a static ffmpeg for windows whilst as far as possible keeping a minimalist Windows PC environment free from bits needed for compilation (eg stuff as needed for jb's approach and/or visual studio bits for vapoursynth). There are various linux cross-compilation approaches (eg zeranoe's which is a widely used build, rdp's, deadsix27's) eg using "free" ubuntu in a clean VM. The market appears to be out there.

Quote:
Originally Posted by jackoneill View Post
The core library and the included plugins (minus ocr) can be cross-compiled pretty easily. You pass
Code:
--host=x86_64-w64-mingw32
to configure.
Ah, good oh. I guess one needs to do the usual things like setup the same folder structures as compiling it under windows, with sources from https://github.com/pinterf/AviSynthPlus/tree/MT and https://github.com/sekrit-twc/zimg/releases

Quote:
Originally Posted by jackoneill View Post
The Python module is the annoying part, because you must also cross-compile Python, and I think the Python developers don't make it easy.
But this is all irrelevant, because VapourSynth has a C API, so you can just use the official binaries compiled with MSVC. You don't have to cross-compile it for ffmpeg.
Correction: VapourSynth has a reasonable C API, unlike Python.
Oh. Time for some newbie googling, to see if anyone's cross-compiled python libraries in a way acceptable to an ffmpeg build process.

I suppose setting up for cross-compiling would need to take into account the "correct" way to do these these things, to result in a cross compiled Win build:

From the current windows build instructions
Quote:
Needs Visual Studio 2017 and vsnasm (https://github.com/ShiftMediaProject/VSNASM)
It also needs both 32 and 64 bit Python 3.6 series with recent setuptools, cython, sphinx and sphinx-intl installed
Inno setup is needed to create the installer (default installation path assumed)
7-zip is needed to compress the portable version (default installation path assumed)
-
Clone VapourSynth
Clone zimg v2.7 branch into the VapourSynth dir (https://github.com/sekrit-twc/zimg)
Clone avs+ pinterf mt branch into the VapourSynth dir (https://github.com/pinterf/AviSynthPlus/tree/MT)
Compile 32 and 64 bit release using the VapourSynth solution
Run cython_build.bat to compile the Python modules
Run make_docs.bat to compile the documentation
I had to make "novice's home notes" while experimenting building vapoursynth under windows on a very old and exhausted "spare" PC ... so I suppose these would also need to be catered for as a part of cross compilation

Quote:
A. Under Windows, I found that for other dependencies like
cython, sphinx and sphinx-intl this seemed to not crash:
1. install 64 bit python (3.6 in this case) as I only need 64bit stuff
2. start an Admin CMD dos box (or it can't copy files into protected folders) then use the pip command to install the packages, which apparently gets installed with python:
pip uninstall cython
pip --no-cache-dir install cython
pip uninstall sphinx-intl
pip --no-cache-dir install sphinx-intl
# uninstall and then install, to get the latest versions

B. Under Windows I also got this error building avscompat
Quote:
LINK : fatal error LNK1181: cannot open input file 'gdi32.lib'
which is apparently a known error per http://stackoverflow.com/questions/3...file-gdi32-lib
Quote:
If anyone has this problem on Visual Studio 2017, it is an issue with the Windows 10 Creators Update.
A workaround is to select the "Windows 10 SDK (10.0.15063.0) for Desktop C++ x86 and x64" under the "Desktop development with C++" workload in the Visual Studio Installer.
C. Other source to clone before building, into EXACTLY the right folders
1. For vapoursynth to build, the avs+ mt branch placed in the root of vapoursynth folder tree must be called "AviSynthPlus". Source: https://github.com/pinterf/AviSynthPlus/tree/MT
2. and the zimg v2.7.1 branch placed in the root of vapoursynth folder tree must be called "zimg". Source: https://github.com/sekrit-twc/zimg/releases
Looking at that, I'm agreeing with you, jb's Windows-based build method looks like it may be the only way for a newbie to get something up and going once the VS coding etc is done. Unless someone has already done it and is willing to share the process in detail for others to learn
hydra3333 is offline   Reply With Quote
Old 27th March 2018, 18:38   #8  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 725
VapourSynth's Avisynth(+) compatibility module must be compiled with MSVC, because it uses the Avisynth(+) C++ API, and Avisynth(+) itself is compiled with MSVC. If you manage to compile the Avisynth(+) compatibility module with clang and its -fms-extensions parameter, that may also work, but I don't think anyone tested it.
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 27th March 2018, 19:18   #9  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,031
Quote:
Originally Posted by jackoneill View Post
VapourSynth's Avisynth(+) compatibility module must be compiled with MSVC, because it uses the Avisynth(+) C++ API, and Avisynth(+) itself is compiled with MSVC. If you manage to compile the Avisynth(+) compatibility module with clang and its -fms-extensions parameter, that may also work, but I don't think anyone tested it.
Clang can compile that too now after avs+ actually made the headers follow the c++ standard... I think it works but didn't test it that much.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th April 2018, 21:15   #10  |  Link
DJATOM
Registered User
 
DJATOM's Avatar
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 187
I spent some time on making x264 support vpy scripts internally. Patch here.
The ideas and some code taken from ffmpeg demuxer proposed here, avs.c input from x264 and NVEnc's vpy input module.
Not sure if that's faster than piping, but looks like complex scripts uses all CPU cores.
Code will definitely compile well with icl 2018 or gcc 7.3.0. I didn't tried another compilers.
__________________
Me on GitHub | My Telegram
PC Specs: Ryzen 3900X (no OC with 250W Air cooling), Asus ROG Crosshair Hero VII (WiFi) @ chipset x470, 32 GB RAM @ 3333MHz OC, Gigabyte RTX 2070, Kingston A1000 @ 240 GB
DJATOM is offline   Reply With Quote
Old 29th April 2018, 02:06   #11  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 531
Quote:
Originally Posted by DJATOM View Post
I spent some time on making x264 support vpy scripts internally. Patch here.
The ideas and some code taken from ffmpeg demuxer proposed here, avs.c input from x264 and NVEnc's vpy input module.
Not sure if that's faster than piping, but looks like complex scripts uses all CPU cores.
Code will definitely compile well with icl 2018 or gcc 7.3.0. I didn't tried another compilers.
Thank you !


Separately, here's a link explaining why building a static ffmpeg with vapoursynth can't be done.
https://forum.doom9.org/showthread.p...31#post1840531
hydra3333 is offline   Reply With Quote
Old 29th April 2018, 02:15   #12  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,073
http://lists.ffmpeg.org/pipermail/ff...il/229125.html
TheFluff is offline   Reply With Quote
Old 29th April 2018, 12:33   #13  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 653
Quote:
Originally Posted by hydra3333 View Post
Separately, here's a link explaining why building a static ffmpeg with vapoursynth can't be done.
https://forum.doom9.org/showthread.p...31#post1840531
IMHO the best we can do is build VS core and VSScript statically and let FFmpeg link with the two static libraries, plus delay-loading python's dll. So users of FFmpeg that don't need vpy demuxing won't see error message about missing DLLs, and the other users that need vpy demuxing can use either installed or portable Python+VS on their own.
HolyWu is offline   Reply With Quote
Old 29th April 2018, 17:07   #14  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 531
OK, if that could be done and for example that meant that if the ffmpeg exe was run in the same folder as "portable VS" and it worked then that could well be close enough for some purposes ( eg mine ).

Do you know of the current ffmpeg change proposal (per the link above) achieves that including delayed loading of python or are there more changes needed ?

Also, I wonder if the same portable VS folder structure is needed for plugins DLLs etc which I guess can't be statically linked in either, I suppose it is.

Last edited by hydra3333; 29th April 2018 at 17:10.
hydra3333 is offline   Reply With Quote
Old 29th April 2018, 18:09   #15  |  Link
JEEB
もこたんインしたお!
 
JEEB's Avatar
 
Join Date: Jan 2008
Location: Finland / Japan
Posts: 514
It loads the script into memory, and then uses libvapoursynth's vsscript_evaluateScript() to evaluate it. Whatever libvapoursynth does in the background regarding python etc. is what matters, not really that patch.
__________________
[I'm human, no debug]
JEEB is offline   Reply With Quote
Old 29th April 2018, 18:39   #16  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 653
Quote:
Originally Posted by hydra3333 View Post
OK, if that could be done and for example that meant that if the ffmpeg exe was run in the same folder as "portable VS" and it worked then that could well be close enough for some purposes ( eg mine ).

Do you know of the current ffmpeg change proposal (per the link above) achieves that including delayed loading of python or are there more changes needed ?

Also, I wonder if the same portable VS folder structure is needed for plugins DLLs etc which I guess can't be statically linked in either, I suppose it is.
Well, It indeed can be done because I have built static VS and FFmpeg with gcc and the resulting binary works fine with both installed and portable VS. But I don't know (and highly doubt) that the procedure to generate a delay-import library of python will be included in the configure script. I generate it myself using gendef and dlltool. By the way the patch on FFmpeg mailing list seems buggy, it gives an error when trying to open vpy file. But Stephen's patch works fine for me.

Last edited by HolyWu; 29th April 2018 at 18:44.
HolyWu is offline   Reply With Quote
Old 30th April 2018, 03:29   #17  |  Link
hydra3333
Registered User
 
Join Date: Oct 2009
Location: crow-land
Posts: 531
Your own, good oh.
If I may, do you have a link to your procedure which I could look and learn from (and pinch) ?
hydra3333 is offline   Reply With Quote
Old 30th April 2018, 05:53   #18  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 653
Quote:
Originally Posted by hydra3333 View Post
Your own, good oh.
If I may, do you have a link to your procedure which I could look and learn from (and pinch) ?
Well, I think I'll just post them here.

Firstly grab python36.dll from (portable) Python's folder. In msys2, use
Code:
gendef python36.dll
to produce python36.def. Then use
Code:
dlltool --output-delaylib libpython3.6m.a --input-def python36.def
to produce the delay-import library we need and put it in mingw64/lib.

When configuring VS, you can optionally pass
Code:
--disable-shared --disable-vspipe --disable-python-module --disable-plugins
if you only need the static core and vsscript libraries for ffmpeg.

When configuring ffmpeg, pass the options below along with the other options you need. VS_CORE_EXPORTS needs to be defined due to static linking and hence symbol import is unneeded. For unknown reason I have to pass -lpython3.6m -lstdc++ as extra libs when linking with static vsscript, otherwise undefined references occur.
Code:
--enable-vapoursynth --extra-cflags="-DVS_CORE_EXPORTS" --extra-ldflags="-static" --extra-libs="-lpython3.6m -lstdc++"
HolyWu is offline   Reply With Quote
Old 1st May 2018, 13:21   #19  |  Link
jackoneill
unsigned int
 
jackoneill's Avatar
 
Join Date: Oct 2012
Location: 🇪🇺
Posts: 725
Quote:
Originally Posted by HolyWu View Post
For unknown reason I have to pass -lpython3.6m -lstdc++ as extra libs when linking with static vsscript, otherwise undefined references occur.
Code:
--enable-vapoursynth --extra-cflags="-DVS_CORE_EXPORTS" --extra-ldflags="-static" --extra-libs="-lpython3.6m -lstdc++"
You have to pass -lstdc++ because I assume ffmpeg has no C++ code, or at least the library that uses VSScript doesn't. Thus gcc is used for linking, not g++. gcc doesn't link stdc++ because it thinks it's all C code, but VSScript uses C++ internally.

And you need to pass -lpython3.6m most likely because ffmpeg's configure doesn't call pkg-config with the --static parameter. If it did, VSScript's pkg-config file would give it -lpython3.6m already.
__________________
Buy me a "coffee" and/or hire me to write code!
jackoneill is offline   Reply With Quote
Old 1st May 2018, 17:10   #20  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,976
Will this one day be implemented in common distributed builds , such as zeranoe's ? Or the autobuild scripts ? Or will this be something you have to compile yourself ?
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:44.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.