ffmpeg using .vpy scripts directly one day, like avisynth ?

[qyot27]

Unfortunately I can't help you with this specific issue, but I have a request :
could you consider to make a new FFMS2 compile with the libaom decoder replaced by dav1d ?
On relative simple scripts, the just decoding from AV1 takes a lot of CPU with libaom, while dav1d is at least 4x faster on modern CPU's.
You mean like the builds I released in January and March of this year? (https://forum.doom9.org/showthread.php?p=1863450#post1863450)

[hydra3333]

... since I could never seem to get the official implementation to cross-compile right, despite following the steps provided earlier to get the Python lib encapsulated correctly.

ffmpeg-4.2-dev-N-93696-g45048ece81-win64-static
(https://drive.google.com/open?id=1wtUE-k8DkBrESZKkCLpgVNPr6MdDi9QS)
ffmpeg version N-93696-g45048ece81 Copyright (c) 2000-2019 the FFmpeg developers

built with gcc 9.0.1 (Built by Wolfberry) 20190426 (prerelease)

configuration: --enable-amf --enable-avisynth --enable-bzlib --enable-cuda --enable-d3d11va --enable-ffnvcodec --enable-frei0r --enable-gray --enable-iconv --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libcdio
--enable-libdav1d --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-liblensfun --enable-libmfx --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband
--enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libssh --enable-libtesseract --enable-libvidstab --enable-libvmaf --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg
--enable-libzmq --enable-lzma --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-openssl --enable-pthreads --enable-sdl2 --enable-vapoursynth --enable-zlib --enable-gpl --enable-version3


Would it be possible to post the build scripts ? That'd likely help the many.

[richardpl]

That build is unsafe for using.

[Wolfberry]

Do you mean the auto-detection of .vpy extension?

That's in the original implementation found at post #26

I can try revert that part to the official implementation by wm4.

Or do you mean something else?

If you feel unsafe then don't use it. It's simple :)

[Pat357]

That build is unsafe for using.

Why is that ?

[hydra3333]

Thanks !

[sl1pkn07]

that patch is safe to push to upstream?

[Pat357]

@Wolfberry
Your latest FFmpeg build N-93755-ga5387f983d (20190504) seems to be broken for me.

If I do "ffplay -i "d:\path.to\a.mkv", I get the normal text output on screen, but the window that should show the video is white with the typical "windows wait" icon that stays there for about 5 seconds after the window closes.
The 5 seconds seems to be interdependent from the video-length: same for a 20s fragment or a full film (90-210min).

Also Avisynth scripts or VS scripts produce the same scenario, even when I specify "ffplay -f vapoursynth -i a.vpy"

Your previous version N-93696-g45048ece81 posted from 2019-04-30 is working perfect.

I have VS R45 installed on my system (not port.)

From my Windows log I have :
Foutbucket 1498974882443012356, type 4
Naam van gebeurtenis: APPCRASH
Antwoord: Niet beschikbaar
Id van CAB-bestand: 0

Handtekening van probleem:
P1: ffplay.exe
P2: 0.0.0.0
P3: 00000000
P4: msvcrt.dll
P5: 7.0.17134.1
P6: 5cbba6fd
P7: c0000005
P8: 000000000005cc53
P9:
P10:

[Wolfberry]

After some investigation, the culprit was LTO.

I saw there are some improvements in LTO optimizations in GCC 9, so I decided to give it a try.

It turns out that LTO was still more or less broken for ffmpeg, at least in MinGW.

[stax76]

Thanks for the built. Any idea what could be wrong here?


C:\Users\frank\Desktop>C:\Users\frank\Daten\Projekte\VB\staxrip\bin\Apps\Encoders\ffmpeg\ffmpeg.exe -i "C:\Users\frank\Daten\Misc\Video Samples\test_temp\test.vpy" -c:v libx264 -an -y -hide_banner "C:\Users\frank\Daten\Misc\Video Samples\test_temp\test_out.mkv"

C:\Users\frank\Daten\Misc\Video Samples\test_temp\test.vpy: Invalid data found when processing input



import os
import sys
ScriptPath = 'C:/Users/frank/Daten/Projekte/VB/staxrip/bin/Apps/Plugins/VS/Scripts'
sys.path.append(os.path.abspath(ScriptPath))
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(r"C:\Users\frank\Daten\Projekte\VB\staxrip\bin\Apps\Plugins\vs\vslsmashsource\vslsmashsource.dll")
clip = core.lsmas.LibavSMASHSource(r"C:\Users\frank\Daten\Misc\Video Samples\test.mp4")
clip.set_output()



General

CompleteName : C:\Users\frank\Daten\Misc\Video Samples\test.mp4
Format/String : MPEG-4
Format_Profile : Base Media
CodecID/String : isom (isom/iso2/avc1/mp41)
FileSize/String : 60.9 MiB
Duration/String : 2 min 49 s
OverallBitRate/String : 3 007 kb/s
Encoded_Application/String: Lavf57.19.100

Video

ID/String : 1
Format/String : AVC
Format/Info : Advanced Video Codec
Format_Profile : High@L4
Format_Settings : CABAC / 3 Ref Frames
Format_Settings_CABAC/String: Yes
Format_Settings_RefFrames/String: 3 frames
CodecID : avc1
CodecID/Info : Advanced Video Coding
Duration/String : 2 min 49 s
BitRate/String : 2 876 kb/s
Width/String : 1 920 pixels
Height/String : 1 080 pixels
DisplayAspectRatio/String: 16:9
FrameRate_Mode/String : Variable
FrameRate/String : 23.976 (24000/1001) FPS
FrameRate_Minimum/String : 23.974 FPS
FrameRate_Maximum/String : 23.981 FPS
ColorSpace : YUV
ChromaSubsampling/String : 4:2:0
BitDepth/String : 8 bits
ScanType/String : Progressive
Bits-(Pixel*Frame) : 0.058
StreamSize/String : 58.2 MiB (96%)
colour_range : Limited
colour_primaries : BT.709
transfer_characteristics : BT.709
matrix_coefficients : BT.709
Codec configuration box : avcC

Audio

ID/String : 2
Format/String : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
CodecID : mp4a-40-2
Duration/String : 2 min 49 s
BitRate_Mode/String : Constant
BitRate/String : 126 kb/s
Channel(s)/String : 2 channels
ChannelLayout : L R
SamplingRate/String : 44.1 kHz
FrameRate/String : 43.066 FPS (1024 SPF)
Compression_Mode/String : Lossy
StreamSize/String : 2.54 MiB (4%)
Default/String : Yes
AlternateGroup/String : 1

[stax76]

Same error using ffms2 as src filter.

edit:

I missed using : -f vapoursynth

[stax76]

The built works very well and it's really helpful being able to open vpy with ffmpeg.

I wonder if mpv can open vpy too? I tried it a few days before but couldn't get it working.

[stax76]

@Wolfberry

I would like to suggest using --enable-libmp3lame in the next built.

[stax76]

@Wolfberry

Thanks for the new built. There is an issue, I'm not able to use this command line:

ffmpeg.exe -f vapoursynth -i C:\test.vpy -c:v libx265 -an -y -hide_banner C:\test.mkv

The problem seem to be arbitrary because sometimes it works and it happens only with x265 and not with x264.

[Revan654]

@Wolfberry

Thanks for the new built. There is an issue, I'm not able to use this command line:

ffmpeg.exe -f vapoursynth -i C:\test.vpy -c:v libx265 -an -y -hide_banner C:\test.mkv

The problem seem to be arbitrary because sometimes it works and it happens only with x265 and not with x264.

from what I read there been some changes to x265 ffmpeg encoder.

If all else fails you could just do this:

"C:\Program Files (x86)\VapourSynth\core64\vspipe.exe" -y "to\your\VS\Script.vpy" - | "x265.exe" [Options] -

I know it defeats the purpose of using ffmpeg internal encoder, but cmd line might help come up with proper cmdline.

I know this code works with x265, NVEnc, and mpv player.

[Revan654]

The built works very well and it's really helpful being able to open vpy with ffmpeg.

I wonder if mpv can open vpy too? I tried it a few days before but couldn't get it working.

Unless something has changed, It's Not possible. I've tried to get it to work last year, Every single test failed. It would just refuse to open file or just crash the program.

However using the following line will play back the vpy script in mpv, It will lack the ability to properly scrub the file forward or back.

"C:\Program Files (x86)\VapourSynth\core64\vspipe.exe" -y "to\your\VS\Script.vpy" - | "mpv.exe" [Options] -

This was the basic set to the script:

import os
import sys
ScriptPath = 'Path/to/your/VS/Scripts/Filters'
sys.path.append(os.path.abspath(ScriptPath))
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(r"SourceFilterHere|I'm Using DGIndexNV")
clip = core.dgdecodenv.DGSource(r"C:\to\your\Source\File.mkv")
clip.set_output()

However Potplayer can open vpy script and play it back.

[_Al_]

clip could be output to mpv player right from script itself using output() and using subprocess modul. VSEdit might need clip.set_output() at the end.
import vapoursynth as vs
from vapoursynth import core
import subprocess

clip = core.avisource.AVISource(r'C:\video.avi')

#import shutil
#mpv = shutil.which('mpv')
mpv = r'C:\tools\mpv.exe'
mpv_cmd = [mpv, '-']
process = subprocess.Popen(mpv_cmd, stdin=subprocess.PIPE)
file_handle=process.stdin
clip.output(file_handle, y4m = True)
process.communicate()

[Wolfberry]

The problem seem to be arbitrary because sometimes it works and it happens only with x265 and not with x264.

Yes, I am able to reproduce it.

I adjusted the LAVF patches for x265 and it seems to be better, can you confirm?

[stax76]

@Wolfberry

It's either fixed or improved, didn't happen in a couple of test runs, much appreciated, thanks.

[unix]

I tried to use Wolfberry FFmpeg Builds but I couldn't encode, did I missed something?

vpy script:

import os
import sys
from vapoursynth import core
import vapoursynth as vs
core = vs.get_core()

src = core.ffms2.Source(source='dcr.mkv').std.AssumeFPS(fpsnum=24000,fpsden=1001).std.Trim(60,75)
src.set_output()


ffmpeg.exe -f vapoursynth -i TEST.vpy -c:v libx264 -an test1.mkv

[stax76]

Try with the most simple script, meaning only the source filter, what does ffmpeg output?

[unix]

But my script so simple

from vapoursynth import core
import vapoursynth as vs
core = vs.get_core()

src = core.ffms2.Source(source='drc.mkv')
src.set_output()


Still I didn't get any output file!!!

[stax76]

There is no output so it could be the same problem I had before. Since you are using relative paths, are you sure all files can be found? Does it work with a real input file (mkv)?

[unix]

There is no output so it could be the same problem I had before. Since you are using relative paths, are you sure all files can be found? Does it work with a real input file (mkv)?

All files are in the same folder which is VapourSynth64Portable

from vapoursynth import core
import vapoursynth as vs
core = vs.get_core()

src = core.ffms2.Source(source='C:/Users/:)/Desktop/VapourSynth64Portable/cdl.mkv')
src.set_output()


I tried with avi as well, but there's no output, also I used ffplay but nothing happens!

ffplay.exe -f vapoursynth -i test.vpy

[jackoneill]

VapourSynth and VSScript are now statically linked into ffmpeg.

By the way, did you need to patch VapourSynth to accomplish this?

[ChaosKing]

All files are in the same folder which is VapourSynth64Portable



I tried with avi as well, but there's no output, also I used ffplay but nothing happens!

I think VS needs to be installed. ffmpeg & ffplay are working fine.
Tested with: ffplay.exe -f vapoursynth -i 'D:\test.vpy'

[Wolfberry]

By the way, did you need to patch VapourSynth to accomplish this?

I use gendef and dlltool to generate delay-import libraries from official VapourSynth and Python dlls, so no patch is needed.

What I mean by static is that ffmpeg will not require extra dlls at startup.

I think VS needs to be installed. ffmpeg & ffplay are working fine.
Tested with: ffplay.exe -f vapoursynth -i 'D:\test.vpy'

Portable VapourSynth should also works since this is the only one I tested.

I just extracted VapourSynth64-Portable-R45, python-3.7.3-embed-amd64 and ffmpeg to the same directory and use that as a testing environment.

[unix]

Thank you Wolfberry.

By the way using direct vapoursynth method is it faster than vspipe to ffmpeg, or it depends on the filters that using?

[Wolfberry]

By the way using direct vapoursynth method is it faster than vspipe to ffmpeg, or it depends on the filters that using?

Usually the native vapoursynth demuxer is faster than piping, but different implementation of the demuxer can have a impact on speed, see post #47 (http://forum.doom9.org/showthread.php?p=1867060#post1867060) and #48 (http://forum.doom9.org/showthread.php?p=1867074#post1867074) for details.

[qyot27]

Since I've now hammered out the kinks (hopefully), I'm able to answer this based on actually cross-compiling VapourSynth instead of using gendef on Windows dlls.
By the way, did you need to patch VapourSynth to accomplish this?
Not really. Most of the weirdness is in how to handle Python (it still needs to be gendeffed, et al.). VapourSynth itself has only three issues at play here:

src/vsscript/vsscript.cpp has an include of Windows.h that trips up cross-compiles due to case sensitivity.
No matter what I tried, I couldn't force NASM to recognize the multilib MinGW-w64 environment and actually compile 32-bit objects, so I have to force-override the Makefile to compile the correct object format.
The Requires.private invocation of python in vapoursynth-script.pc screws up FFmpeg's configure tests; Libs.private does work as expected.
A possible fourth issue is whether there's a case for including -lstdc++ in the .pc file's Libs(.private) stuff (it's not a problem when a bunch of other external libraries are enabled, since one/several of them pull in -lstdc++, but a 'simple' build of FFmpeg with VapourSynth and AviSynth as the only enabled external libraries fails unless --extra-libs is used to provide libstdc++ as detailed way earlier in this thread). Or whether a Cflags.private entry should be added for pkgconf users so -DVS_CORE_EXPORTS can be used only in the case of static linking. Essentially, this 'point' basically is just bikeshedding.

The entire process is documented here (link goes directly to the VapourSynth section of the guide):
https://github.com/qyot27/mpv/blob/extra-new/DOCS/crosscompile-mingw-tedious.txt#L4396


One distinction is that when I'd tested with using gendef to handle VapourSynth too, I could get FFmpeg to link to it, but mpv refused to link to libavformat. gendeffing only Python and then cross-compiling VapourSynth like any other library results in both FFmpeg and mpv being able to link to it (importantly, mpv being able to directly play back scripts this way).

32-bit and 64-bit FFmpeg and mpv with static VapourSynth (http://www.mediafire.com/file/dv34bighibg2wcx/ffmpegmpv_vs_static_20190530.7z/file)

Said binaries include *both* VS demuxers. The upstream demuxer is still 'vapoursynth', and the other one is 'vapoursynth_alt'. It's easy to tell them apart because they return different data types as input: '-f vapoursynth' returns wrapped_avframe, '-f vapoursynth_alt' returns rawvideo. The upstream demuxer also doesn't honor relative file paths across directories (in just FFMS2? maybe in general?), while the alt demuxer does.

[Natty]

ffmpeg-20190729-43891ea-win64-static
(https://drive.google.com/open?id=1iCiKNbyQfME6E9x89YsnXqdAI-Jcs6va)


404. That’s an error.

The requested URL was not found on this server. That’s all we know.
:(

[ChaosKing]

404. That’s an error.

The requested URL was not found on this server. That’s all we know.
:(

Click on his signature link, you can find a newer version there.

[Natty]

Click on his signature link, you can find a newer version there.

got it. :thanks:

[kolak]

Since I've now hammered out the kinks (hopefully), I'm able to answer this based on actually cross-compiling VapourSynth instead of using gendef on Windows dlls.

Not really. Most of the weirdness is in how to handle Python (it still needs to be gendeffed, et al.). VapourSynth itself has only three issues at play here:

src/vsscript/vsscript.cpp has an include of Windows.h that trips up cross-compiles due to case sensitivity.
No matter what I tried, I couldn't force NASM to recognize the multilib MinGW-w64 environment and actually compile 32-bit objects, so I have to force-override the Makefile to compile the correct object format.
The Requires.private invocation of python in vapoursynth-script.pc screws up FFmpeg's configure tests; Libs.private does work as expected.
A possible fourth issue is whether there's a case for including -lstdc++ in the .pc file's Libs(.private) stuff (it's not a problem when a bunch of other external libraries are enabled, since one/several of them pull in -lstdc++, but a 'simple' build of FFmpeg with VapourSynth and AviSynth as the only enabled external libraries fails unless --extra-libs is used to provide libstdc++ as detailed way earlier in this thread). Or whether a Cflags.private entry should be added for pkgconf users so -DVS_CORE_EXPORTS can be used only in the case of static linking. Essentially, this 'point' basically is just bikeshedding.

The entire process is documented here (link goes directly to the VapourSynth section of the guide):
https://github.com/qyot27/mpv/blob/extra-new/DOCS/crosscompile-mingw-tedious.txt#L4396


One distinction is that when I'd tested with using gendef to handle VapourSynth too, I could get FFmpeg to link to it, but mpv refused to link to libavformat. gendeffing only Python and then cross-compiling VapourSynth like any other library results in both FFmpeg and mpv being able to link to it (importantly, mpv being able to directly play back scripts this way).

32-bit and 64-bit FFmpeg and mpv with static VapourSynth (http://www.mediafire.com/file/dv34bighibg2wcx/ffmpegmpv_vs_static_20190530.7z/file)

Said binaries include *both* VS demuxers. The upstream demuxer is still 'vapoursynth', and the other one is 'vapoursynth_alt'. It's easy to tell them apart because they return different data types as input: '-f vapoursynth' returns wrapped_avframe, '-f vapoursynth_alt' returns rawvideo. The upstream demuxer also doesn't honor relative file paths across directories (in just FFMS2? maybe in general?), while the alt demuxer does.

Both versions crash very quickly with my deinterlacing script. ffmpeg simply hangs and does nothing.
Wolfberry version does absolutely nothing- just ends without any print or error.

[poisondeathray]

Both versions crash very quickly with my deinterlacing script. ffmpeg simply hangs and does nothing.
Wolfberry version does absolutely nothing- just ends without any print or error.

Is it valid script ? Does it work in other programs?

what does info say ?

vspipe --info script.vpy -

Post the script.

[qyot27]

Roll back VapourSynth (git) and Python (release) to versions that were current at the end of May. Trying to mix and match versions will fail spectacularly.

Or don't use a build from nearly 5 months ago; use a current one built against the now-current versions of those environments.

[kolak]

Where can I find current build?
(script is absolutely fine)

[unix]

Where can I find current build?
(script is absolutely fine)

https://drive.google.com/drive/folders/1xZQABtoaSFgGu11YstmHKYLzO3elemlC

[kolak]

I've used this one already:
ffmpeg-20190926-87ddf9f-win64-static.7z

and it does nothing. Maybe my PY is too new as it's the latest one (3.7.4).

[qyot27]

FFmpeg and mpv built with static VapourSynth/Python, 2019-10-10 (http://www.mediafire.com/file/fblizu9b89zwdyz/ffmpegmpv-vsstatic_20191010.7z/file)

VapourSynth: git-bae5d1a
Python: 3.7.4

The incompatibility is most likely tied to the version of Python, but VS itself probably incurs *some* of the restriction too (it matters which version of Python the VS .dlls were built against, just as much as it matters which one is linked into FFmpeg). I still have Python 3.7.3 and the official release build of R47 installed system-wide, but built FFmpeg with Python 3.7.4 and VapourSynth's current git HEAD. It worked just fine. It seems to be okay as long as the Python and VapourSynth linked into FFmpeg are newer than the ones on the system, but the inverse situation will almost certainly fail.

As before, both VapourSynth demuxers are present. The upstream demuxer is vapoursynth, the rawvideo demuxer patch provides vapoursynth_alt.

[kolak]

THX.
Any difference in demuxers stability wise?

[kolak]

Seems to be working. Just getting this:
"Script exceeded memory limit" which I assume is about some cache in vs. How do I increase it ?
With vapoursynth_alt speed is 3x better.

[Pat357]

The FFmpeg binaries work fine to play/process .VPY files directly ; the vapoursynth_alt demuxer seems faster than the build-in.

How to feed a.vpy directly to MPV ?
Is there a way to specify which demuxer/format to use when playing a .vpy file using MPV ?

[qyot27]

--demuxer-lavf-format=vapoursynth (or vapoursynth_alt). The 64-bit mpv from the package I posted on the 10th also supports using VapourSynth as a video filter (for say, 24->60fps motion interpolation).

.vpy can associated with the above command in the config file, so you don't have to remember to use it:
[extension.vpy]
demuxer-lavf-format=vapoursynth

[stax76]

I'm trying to pipe from ffmpeg to the crappy SvtAv1EncApp, with avisynth it works using cmd, it sucks that it doesn't work with Windows Terminal and PowerShell, creates distorted output.

With VapourSynth it does not work with cmd either, vspipe doesn't support format conversation apparently. I don't know if the crappy Intel encoder supports y4m input, it's not documented at least.

ffmpeg.exe -loglevel fatal -hide_banner -i aaa.avs -nostdin -f rawvideo -pix_fmt yuv420p - | SvtAv1EncApp.exe -i stdin -n 328 -w 1280 -h 720 -q 20 -b test.ivf

ffmpeg.exe -loglevel fatal -hide_banner -f vapoursynth -i aaa.vpy -nostdin -f rawvideo -pix_fmt yuv420p - | SvtAv1EncApp.exe -i stdin -n 328 -w 1280 -h 720 -q 20 -b test.ivf


edit:

found a way:

ffmpeg -loglevel fatal -hide_banner -f vapoursynth -i aaa.vpy -nostdin -f yuv4mpegpipe -strict -1 -pix_fmt yuv420p - | SvtAv1EncApp -i stdin -n 328 -w 1280 -h 720 -q 20 -b aaa.ivf

[Selur]

@stax76: that looks strange,... isn't '-1' only needed when handling 10+bit content?
and shouldn't the output pixel format then be yuv420p10le (for example) ?
Analog to SvtHevcEncApp, I would try something like:
ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "F:\TestClips&Co\files\10bitTest.mkv" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | SvtAv1EncApp -i stdin -fps-num 25 -fps-denom 1 -n 429 -b "E:\Temp\10bitTest_06_05_51_8410_01.ivf"
to handle 10bit content,..

Cu Selur

[stax76]

My code relies on the fact that for every progress a line break is written to the output stream, I've never seen a console app that don't do it and most .NET apps rely on this and working around it will take at least a day, I made a request and quit for now.

[poisondeathray]

Are there any updated ffmpeg windows binaries complied with vapoursynth demuxer available ? Thanks

[stax76]

In the staxrip apps dialog you can find binaries and download links. Last Patman build is from this month.

[poisondeathray]

In the staxrip apps dialog you can find binaries and download links. Last Patman build is from this month.

Thanks