AviSynth not working on Linux

[forgetfool]

Hi, people.

Just a short history before my question. I have been using AviSynth/AviSynth+ on Windows quite heavily with Audacity being my sound editor and VirtualDub - a renderer. At some point I fully switched to Linux (currently it's ArchLinux).

Then I discovered there is AviSynth+ fork for Linux! But it does not work for me apart from a basic test... Among piles of stuff on the web I found, for example, this guide: https://github.com/AviSynth/AviSynthPlus/blob/master/distrib/docs/english/source/avisynthdoc/contributing/posix.rst.

I use ArchLinux, so I got all the packages (didn't need to compile anything manually). My ffmpeg configuration includes --enable-avisynth flag. Version() works when I do "ffplay script.avs"

As for FFMS2, trying "FFVideoSource('file')" produced only -- Script error: There is no function named 'FFVideoSource'. And with with the recent install of avisynthplus I get: Cannot load file '/usr/lib/avisynth/libimageseq.so' when I run "ffplay -i test.avs". The .so file actually exists...

Thanks for any help in advance.

[Selur]

Script error: There is no function named 'FFVideoSource'
Never used Avisynth+s Linux port, but looks like auto-loading doesn't work, so you would need to load the plugin manually. (http://avisynth.nl/index.php/Plugins#LoadPlugin)
This assumes you have a FFVideoSource plugin compiled.

Cu Selur

[forgetfool]

Perhaps, I need a different AVISYNTH+ package...

I tried various combinations of:
AddAutoloadDir("...")
LoadPlugin(".../libimageseq.so")

It tries the right file (judging by the path -- I actually tried copying the .so file around), but the output is always the same (except for the path on the "Cannot load file" line):

[verbose] Initialized opengl renderer.
[NULL @ 0x7f001c000bc0] [debug] Opening 'film-test-linux.avs' for reading
[file @ 0x7f001c0017c0] [debug] Setting default whitelist 'file,crypto,data'
[avisynth @ 0x7f001c000bc0] [debug] Format avisynth probed with size=2048 and score=50
[avisynth @ 0x7f001c000bc0] [error] Cannot load file '/usr/lib/avisynth/libimageseq.so'.
(film-test-linux.avs, line 4)
[AVIOContext @ 0x7f001c009b80] [verbose] Statistics: 1596 bytes read, 0 seeks
[error] film-test-linux.avs: Unknown error occurred

[qyot27]

ImageSeq requires DevIL, which apparently is only listed as an optional dependency.
pacman -S community/devil

Not sure what the issue with FFMS2 is, aside from not having installed it, unless that's just the recursive shared library problem cropping up. If you install DJATOM/avs2yuv, you can pipe an FFMS2-using script to FFmpeg or mpv without the shared library issue causing a deadlock (alternatively, build FFMS2 against a minimal, static build of FFmpeg that you don't install to the system; then there won't be a conflict with the system's shared version of FFmpeg and anything that links to that).

[forgetfool]

THANKS! DevIL moved me further. Now I get

[verbose] Initialized opengl renderer.
[NULL @ 0x7ff65c000bc0] [debug] Opening 'film-test-linux.avs' for reading
[file @ 0x7ff65c0017c0] [debug] Setting default whitelist 'file,crypto,data'
[avisynth @ 0x7ff65c000bc0] [debug] Format avisynth probed with size=2048 and score=50
[info] nan : 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0

...and it hangs forever. Is this the deadlock you've mentioned?

Btw, before your message I gave up and tried AUR avisynthplus-git and fmms2-git. That got me the same result as now (i.e. with latest + DevIL)

[qyot27]

Yup.

https://github.com/FFMS/ffms2/issues/369

You can test that you really get video by invoking ImageSource (since DevIL is installed now, so ImageSeq will work) to open a JPG or PNG.

Essentially, FFmpeg is built as shared. FFMS2 links to it, so it's going to use FFmpeg's shared libraries. FFmpeg loads AviSynth+, which (when using FFMS2) loads FFMS2. So FFmpeg is confronted with both itself linked to its own libraries and dlopened FFMS2 linking to those same libraries, and the process stalls (sometimes; see my response there and in the issue on AviSynth+'s issues tracker that I reference).

The most direct way to avoid that conflict is to make sure the FFmpeg libraries FFMS2 uses don't overlap with the system libraries, which is both good and (kind of) bad. Good, because in reality FFMS2 doesn't *need* a lot of those things enabled in FFmpeg, so using a build of FFmpeg for FFMS2 that only enables what it needs results in a much faster build process and a smaller FFMS2. Bad, because static linking is usually viewed as a huge no-no due to the maintenance burden of having to rebuild the entire thing when the underlying libraries need to update (although you'd still have to build FFmpeg again anyway, and FFMS2 is very small). Further, if you don't want that static version of FFMS2 conflicting with the version from the repo, you can build it but not install it to the system. AviSynth+ accepts both ~/.avisynth and ~/.local/lib/avisynth as directories to stash user-specific plugins in, although I haven't really tested to see whether those take absolute precedence over the presence of a symlink to the system FFMS2 in /usr/lib/avisynth.

As I mentioned, though, avs2yuv can work fine with the Community packages of all of these; the only downside is that you have to pipe rather than opening in FFmpeg or mpv directly (it wouldn't be such a problem if we had a NUT-producing pipe tool in the AviSynth+ tree (https://github.com/AviSynth/AviSynthPlus/issues/193)).

[Richard1485]

(alternatively, build FFMS2 against a minimal, static build of FFmpeg that you don't install to the system; then there won't be a conflict with the system's shared version of FFmpeg and anything that links to that).

That seems to be the way that ffms2 is built in the guide (https://github.com/AviSynth/AviSynthPlus/blob/master/distrib/docs/english/source/avisynthdoc/contributing/posix.rst)...

#Ubuntu

PKG_CONFIG_PATH=$HOME/ffmpeg_build/lib/pkgconfig \
CPPFLAGS="-I/usr/local/include/avisynth" \
./autogen.sh --enable-shared --enable-avisynth && \
make -j$(nproc) && \
sudo checkinstall --pkgname=ffms2 --pkgversion="1:$(./version.sh)-git" \
--backup=no --deldoc=yes --delspec=yes --deldesc=yes --strip=yes --stripso=yes \
--addso=yes --fstrans=no --default


...judging by the fact that PKG_CONFIG_PATH points to the ffmpeg created in $HOME when not installing to the system...

#Ubuntu

./configure --prefix=$HOME/ffmpeg_build --enable-gpl --enable-version3 \
--disable-doc --disable-debug --enable-pic --enable-avisynth && \
make -j$(nproc) && \
make install


Presumably, the OP could follow the same instructions, leaving out checkinstall, which is for Debian/Slackware.

[qyot27]

It's not entirely the way I wrote the guide. The guide assumes you're building FFmpeg for a distro that doesn't actually use --enable-avisynth, so it's a fully functional build of FFmpeg (albeit lacking many of the external libraries it can use). But because the FFmpeg is built static, that's the more important part as far as FFMS2 is concerned. The 'minimal' FFmpeg I'm referring to in the case of something specifically for FFMS2 (or L-SMASH-Works, same deal) is this:

cd ffmpeg && \
mkdir ffmpeg-build-for-ffms2 && \
cd ffmpeg-build-for-ffms2 && \
../../configure \
--prefix=$HOME/ffmpeg_build_for_ffms2 \
--enable-gpl \
--enable-version3 \
--disable-encoders \
--disable-muxers \
--disable-doc \
--disable-debug \
--disable-devices \
--disable-avdevice \
--enable-libdav1d \
--extra-cflags="-march=native" && \
make -j$(nproc) && \
make install
Aside from the zlib/bzip2/lzma/iconv stuff that FFmpeg automatically pulls in from the system if present, the only external library is libdav1d, and all the outputs are disabled, it's built as static, and installed to a spot in $HOME rather than the system.

At which point you'd simply change the PKG_CONFIG_PATH given to FFMS2 to $HOME/ffmpeg_build_for_ffms2/lib/pkgconfig.

[Richard1485]

Apologies! I was focusing on the "building ffms2 against an ffmpeg that is not installed to the system" part and overlooked that you had specified a minimal, static build. My bad.

EDIT: In addition to the above, I cleared the "Unknown error occurred" message by copying the .so files (symlinking would probably have done) from /use/lib/local/ to $HOME/.local/lib/avisynth ( $HOME/.avisynth did not work, probably because of needing root permissions), pointing AddAutoloadDir() to that, and specifying an absolute file-path: FFVideoSource("/home/user/file.mkv").

[forgetfool]

So I've built FFmpeg as mentioned by qyot27. Then I've built ffms2 (just did make install, as I don't have checkinstall). Even did all the stuff mentioned in the last message by Richard1485 (I only copied one file libfmms2.so, as nothing else I have is applicable I think).

On ffplay my avs script all I get is

[verbose] Initialized opengl renderer.
[NULL @ 0x7f4600000bc0] [debug] Opening 'film-test-linux.avs' for reading
[file @ 0x7f46000017c0] [debug] Setting default whitelist 'file,crypto,data'
[avisynth @ 0x7f4600000bc0] [debug] Format avisynth probed with size=2048 and score=50
[AVIOContext @ 0x7f4600009b80] [verbose] Statistics: 1680 bytes read, 0 seeks
[error] film-test-linux.avs: Unknown error occurred

- and exit (not a hang this time).

I get this even if I comment every single line of my avs script!!!

Btw, I am having vapoursynth trouble with libffms2.so in this thread: https://forum.doom9.org/showthread.php?p=1939093#post1939093

[forgetfool]

Ok, I was using the wrong ffplay. The one in /usr/bin, not the one just build in /usr/local/bin...

Now with restart of the shell I get the following output (my script is only running Version()):

[verbose] Initialized opengl renderer.
[NULL @ 0x7f5e6c000c80] [debug] Opening 'film-test-linux.avs' for reading
[file @ 0x7f5e6c001980] [debug] Setting default whitelist 'file,crypto,data'
[avs2 @ 0x7f5e6c000c80] [warning] Format avs2 detected only with low score of 1, misdetection possible!
[avs2 @ 0x7f5e6c000c80] [debug] Before avformat_find_stream_info() pos: 0 bytes read:1693 seeks:0 nb_streams:1
[avs2 @ 0x7f5e6c000c80] [warning] Could not find codec parameters for stream 0 (Video: avs2, 1 reference frame, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[avs2 @ 0x7f5e6c000c80] [debug] After avformat_find_stream_info() pos: 1693 bytes read:1693 seeks:0 frames:1
[info] Input #0, avs2, from 'film-test-linux.avs':
[info] Duration: N/A, bitrate: N/A
[info] Stream #0:0, 1, 1/1200000: Video: avs2, 1 reference frame, none, 1200k tbr, 1200k tbn, 1200k tbc
[warning] No decoder could be found for codec avs2
[fatal] Failed to open file 'film-test-linux.avs' or configure filtergraph
[AVIOContext @ 0x7f5e6c009dc0] [verbose] Statistics: 1693 bytes read, 0 seeks

[Richard1485]

Now with restart of the shell I get the following output (my script is only running Version()):

...

[avs2 @ 0x7f5e6c000c80] [warning] Format avs2 detected only with low score of 1, misdetection possible!


That's weird. AVS2 (https://forum.doom9.org/showthread.php?t=176029) is the Chinese equivalent of HEVC. Obviously, the message reads "misdetection possible", but you say that the script contains only Version()? Check that you really have commented everything else out and that you cloned the right repo.

EDIT: ffplay might be mistaking the extension for a format, but that shouldn't happen if it is built with --enable-avisynth. I didn't know that mainstream ffmpeg even knew to look for avs2, but I guess it's been added.

[forgetfool]

Ok, I started all over. (Deleted all the old stuff.)



AviSynth+
---------

git clone git://github.com/AviSynth/AviSynthPlus.git
cd AviSynthPlus
mkdir avisynth-build
cd avisynth-build
cmake ../ -G Ninja
ninja

su
ninja install



FFmpeg
------

git clone git://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
mkdir ffmpeg-build-for-ffms2
cd ffmpeg-build-for-ffms2
../configure \
--prefix=$HOME/ffmpeg_build_for_ffms2 \
--enable-gpl \
--enable-version3 \
--disable-encoders \
--disable-muxers \
--disable-doc \
--disable-debug \
--disable-devices \
--disable-avdevice \
--enable-libdav1d \
--extra-cflags="-march=native"
make -j$(nproc)

su
make install


Test 1
-------

nano test.avs
Version()

./ffplay -i test.avs

[info] ffplay version N-101739-gcad3a5d715 Copyright (c) 2003-2021 the FFmpeg developers
[info] built with gcc 10.2.0 (GCC)
[info] configuration: --prefix=/home/user1/ffmpeg_build_for_ffms2 --enable-gpl --enable-version3 --disable-encoders --disable-muxers --disable-doc --disable-debug --disable-devices --disable-avdevice --enable-libdav1d --extra-cflags='-march=native'
[info] libavutil 56. 72.100 / 56. 72.100
[info] libavcodec 58.135.100 / 58.135.100
[info] libavformat 58. 77.100 / 58. 77.100
[info] libavfilter 7.111.100 / 7.111.100
[info] libswscale 5. 10.100 / 5. 10.100
[info] libswresample 3. 10.100 / 3. 10.100
[info] libpostproc 55. 10.100 / 55. 10.100
[verbose] Initialized opengl renderer.
[NULL @ 0x7ff1a8000c80] [debug] Opening 'test.avs' for reading
[file @ 0x7ff1a8001980] [debug] Setting default whitelist 'file,crypto,data'
[avs2 @ 0x7ff1a8000c80] [warning] Format avs2 detected only with low score of 1, misdetection possible!
[avs2 @ 0x7ff1a8000c80] [debug] Before avformat_find_stream_info() pos: 0 bytes read:11 seeks:0 nb_streams:1
[avs2 @ 0x7ff1a8000c80] [warning] Could not find codec parameters for stream 0 (Video: avs2, 1 reference frame, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[avs2 @ 0x7ff1a8000c80] [debug] After avformat_find_stream_info() pos: 11 bytes read:11 seeks:0 frames:1
[info] Input #0, avs2, from 'test.avs':
[info] Duration: N/A, bitrate: N/A
[info] Stream #0:0, 1, 1/1200000: Video: avs2, 1 reference frame, none, 1200k tbr, 1200k tbn, 1200k tbc
[warning] No decoder could be found for codec avs2
[fatal] Failed to open file 'test.avs' or configure filtergraph
[AVIOContext @ 0x7ff1a8009dc0] [verbose] Statistics: 11 bytes read, 0 seeks


Any suggestion what to try?
THANKS
!

[forgetfool]

And if I follow this guide: https://github.com/AviSynth/AviSynthPlus/blob/master/distrib/docs/english/source/avisynthdoc/contributing/posix.rst, up to test installation with Version(), I get just this:

[info] ffplay version N-101739-gcad3a5d715 Copyright (c) 2003-2021 the FFmpeg developers
[info] built with gcc 10.2.0 (GCC)
[info] configuration: --enable-gpl --enable-version3 --disable-doc --disable-debug --enable-pic --enable-avisynth --cc=cc
[info] libavutil 56. 72.100 / 56. 72.100
[info] libavcodec 58.135.100 / 58.135.100
[info] libavformat 58. 77.100 / 58. 77.100
[info] libavdevice 58. 14.100 / 58. 14.100
[info] libavfilter 7.111.100 / 7.111.100
[info] libswscale 5. 10.100 / 5. 10.100
[info] libswresample 3. 10.100 / 3. 10.100
[info] libpostproc 55. 10.100 / 55. 10.100
[verbose] Initialized opengl renderer.
[NULL @ 0x7f9b14000c80] [debug] Opening 'test.avs' for reading
[file @ 0x7f9b14001980] [debug] Setting default whitelist 'file,crypto,data'
[avisynth @ 0x7f9b14000c80] [debug] Format avisynth probed with size=2048 and score=50
[AVIOContext @ 0x7f9b14009dc0] [verbose] Statistics: 11 bytes read, 0 seeks
[error] test.avs: Unknown error occurred

I don't think any AddAutoLoadDir() is needed for just Version(), right?

So I went back to previous ../configure options. I think we didn't get something right in ffmpeg configure...

[qyot27]

Ok, I started all over. (Deleted all the old stuff.)



AviSynth+
---------

git clone git://github.com/AviSynth/AviSynthPlus.git
cd AviSynthPlus
mkdir avisynth-build
cd avisynth-build
cmake ../ -G Ninja
ninja

su
ninja install



FFmpeg
------

git clone git://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
mkdir ffmpeg-build-for-ffms2
cd ffmpeg-build-for-ffms2
../configure \
--prefix=$HOME/ffmpeg_build_for_ffms2 \
--enable-gpl \
--enable-version3 \
--disable-encoders \
--disable-muxers \
--disable-doc \
--disable-debug \
--disable-devices \
--disable-avdevice \
--enable-libdav1d \
--extra-cflags="-march=native"
make -j$(nproc)

su
make install


Test 1
-------

nano test.avs
Version()

./ffplay -i test.avs

[info] ffplay version N-101739-gcad3a5d715 Copyright (c) 2003-2021 the FFmpeg developers
[info] built with gcc 10.2.0 (GCC)
[info] configuration: --prefix=/home/user1/ffmpeg_build_for_ffms2 --enable-gpl --enable-version3 --disable-encoders --disable-muxers --disable-doc --disable-debug --disable-devices --disable-avdevice --enable-libdav1d --extra-cflags='-march=native'
[info] libavutil 56. 72.100 / 56. 72.100
[info] libavcodec 58.135.100 / 58.135.100
[info] libavformat 58. 77.100 / 58. 77.100
[info] libavfilter 7.111.100 / 7.111.100
[info] libswscale 5. 10.100 / 5. 10.100
[info] libswresample 3. 10.100 / 3. 10.100
[info] libpostproc 55. 10.100 / 55. 10.100
[verbose] Initialized opengl renderer.
[NULL @ 0x7ff1a8000c80] [debug] Opening 'test.avs' for reading
[file @ 0x7ff1a8001980] [debug] Setting default whitelist 'file,crypto,data'
[avs2 @ 0x7ff1a8000c80] [warning] Format avs2 detected only with low score of 1, misdetection possible!
[avs2 @ 0x7ff1a8000c80] [debug] Before avformat_find_stream_info() pos: 0 bytes read:11 seeks:0 nb_streams:1
[avs2 @ 0x7ff1a8000c80] [warning] Could not find codec parameters for stream 0 (Video: avs2, 1 reference frame, none): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[avs2 @ 0x7ff1a8000c80] [debug] After avformat_find_stream_info() pos: 11 bytes read:11 seeks:0 frames:1
[info] Input #0, avs2, from 'test.avs':
[info] Duration: N/A, bitrate: N/A
[info] Stream #0:0, 1, 1/1200000: Video: avs2, 1 reference frame, none, 1200k tbr, 1200k tbn, 1200k tbc
[warning] No decoder could be found for codec avs2
[fatal] Failed to open file 'test.avs' or configure filtergraph
[AVIOContext @ 0x7ff1a8009dc0] [verbose] Statistics: 11 bytes read, 0 seeks


Any suggestion what to try?
THANKS
!
Of course it didn't work. Those were the minimal FFmpeg instructions intended only to build FFMS2, and AviSynth wasn't enabled. Even if it was, you wouldn't be able to do anything with it.

And if I follow this guide: https://github.com/AviSynth/AviSynth...ting/posix.rst, up to test installation with Version(), I get just this:

[info] ffplay version N-101739-gcad3a5d715 Copyright (c) 2003-2021 the FFmpeg developers
[info] built with gcc 10.2.0 (GCC)
[info] configuration: --enable-gpl --enable-version3 --disable-doc --disable-debug --enable-pic --enable-avisynth --cc=cc
[info] libavutil 56. 72.100 / 56. 72.100
[info] libavcodec 58.135.100 / 58.135.100
[info] libavformat 58. 77.100 / 58. 77.100
[info] libavdevice 58. 14.100 / 58. 14.100
[info] libavfilter 7.111.100 / 7.111.100
[info] libswscale 5. 10.100 / 5. 10.100
[info] libswresample 3. 10.100 / 3. 10.100
[info] libpostproc 55. 10.100 / 55. 10.100
[verbose] Initialized opengl renderer.
[NULL @ 0x7f9b14000c80] [debug] Opening 'test.avs' for reading
[file @ 0x7f9b14001980] [debug] Setting default whitelist 'file,crypto,data'
[avisynth @ 0x7f9b14000c80] [debug] Format avisynth probed with size=2048 and score=50
[AVIOContext @ 0x7f9b14009dc0] [verbose] Statistics: 11 bytes read, 0 seeks
[error] test.avs: Unknown error occurred

I don't think any AddAutoLoadDir() is needed for just Version(), right?

So I went back to previous ../configure options. I think we didn't get something right in ffmpeg configure...
ls -l /usr/local/lib/libavisynth*
There should be three files it lists: libavisynth.so.3.7.0, libavisynth.so.8, and libavisynth.so. For whatever reason, the libavisynth.so symlink doesn't always get installed.



Or much more easily:
1) Install AviSynth+ and FFmpeg from the Community repo.
2) Build/install minimal FFmpeg as listed above
3) Build/install FFMS2, directing PKG_CONFIG_PATH to the minimal FFmpeg and installing it to $HOME/.local:
PKG_CONFIG_PATH=$HOME/ffmpeg_build_for_ffms2/lib/pkgconfig \
CPPFLAGS="-I/usr/include/avisynth" \
./autogen.sh --prefix=$HOME/.local --libdir=$HOME/.local/lib/avisynth --enable-shared --enable-avisynth && \
make -j$(nproc)
make install
And then whenever you feel the need to update FFMS2 (every few months or something), repeat steps 2 & 3.

You may or may not need to add $HOME/.local/bin to the $PATH in your user profile so you can use ffmsindex.

Do not use root permissions when installing things to your $HOME directory. That is asking for trouble.

[forgetfool]

Ok, thank you so much, qyo27! I did what you said, and it all works now. FFVideoSource + FFAudioSource -- all good.

:thanks:

[forgetfool]

Only two years have passed, and I back with a problem. For a while I was doing what qyot27 suggested (see the message below about updating FFMS2 and repeating steps 2 & 3), everything was working fine until lately, when I tried to rebuild things.

FFMPEG built, FFMS2 didn't with the following error:


CXXLD src/core/libffms2.la
/usr/bin/ld: /home/romashka/ffmpeg_build_for_ffms2/lib/libswscale.a(swscale.o): warning: relocation against `ff_M24B' in read-only section `.text'
/usr/bin/ld: /home/romashka/ffmpeg_build_for_ffms2/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make: *** [Makefile:653: src/core/libffms2.la] Error 1

I then removed the .a file out of the way (and another one that it complained about afterwards: swscale.o) and when linking, it gave me this:


CXXLD src/core/libffms2.la
CXXLD src/index/ffmsindex
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_toupper4'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_ac3_channel_layout_tab'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_bits_dc_luminance'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_val_ac_chrominance'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_val_dc'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mpeg4audio_sample_rates'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_val_ac_luminance'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_bits_ac_luminance'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mpa_freq_tab'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_bits_dc_chrominance'
/usr/bin/ld: src/core/.libs/libffms2.so: undefined reference to `ff_mjpeg_bits_ac_chrominance'
collect2: error: ld returned 1 exit status
make: *** [Makefile:665: src/index/ffmsindex] Error 1

So I put the files back -- and it all finished fine... :-)

I then did the make install, as usually.


BUT! When I run my avs script now, I get the following:

Cannot load file '/home/romashka/.local/lib/avisynth/libffms2.so'. Reason: /home/romashka/.local/lib/avisynth/libffms2.so: undefined symbol: ff_mjpeg_val_ac_chrominance
(film.avs, line 20)
film.avs: Unknown error occurred


Can someone please help?

[forgetfool]

P.S. Line 20 is a call to FFVideoSource()

[Emulgator]

I would inspect that path '/home/romashka/.local/lib/avisynth/libffms2.so'

[forgetfool]

What do you mean by inspect? That's the path were I am building FFMS2 based on the minimal install of FFMPEG. That's how it all worked for me for a while. Perhaps, I am using a wrong version of something? Perhaps, the newest libavisynth.so.9 is the problem (before it all worked with libavisynth.so.8). Maybe I need to go back to older ffmpeg?

And the main avisynthplus still does not work causing a deadlock, as descibed earlier in this thread. Can it really be that no one is using avisynthplus on linux?

[qyot27]

Try using make clean before building FFmpeg (or even make distclean, and then re-run configure as well, if you have to, add --enable-pic to the FFmpeg configure step). The recompile with -fPIC message sometimes just happens, but other times it's because some part of the configuration changed to include a new external library that was built with fPIC but the main FFmpeg was not.

[Losko]

CXXLD src/core/libffms2.la
/usr/bin/ld: /home/romashka/ffmpeg_build_for_ffms2/lib/libswscale.a(swscale.o): warning: relocation against `ff_M24B' in read-only section `.text'
/usr/bin/ld: /home/romashka/ffmpeg_build_for_ffms2/lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make: *** [Makefile:653: src/core/libffms2.la] Error 1

I recently faced a similar "relocation..." error message (on a different OS, truth be told) when, due to a hidden setting, the linker was trying to mix up several object files created for a static build into a shared object - that doesn't make any sense.
So, as I first step I would double check your build from the basis: do you want ffmpeg shared (CHECK THIS!) ? ffms2 shared ?

[forgetfool]

Thank you, people, for your replies. I REALLY need to get this working... This is my process (from scratch, i.e. no old files and folders). Can someone spot a problem? BIG THANKS!

1) Install AviSynthPlus and FFmpeg from the Community repo (inc. DevIL !)
2) Build/install minimal FFmpeg (*)
3) Build/install FFMS2, directing PKG_CONFIG_PATH to the minimal FFmpeg and installing it to $HOME/.local (**)

(*)
git clone git://git.ffmpeg.org/ffmpeg.git
cd ffmpeg
mkdir ffmpeg-build-for-ffms2
cd ffmpeg-build-for-ffms2
../configure --prefix=$HOME/ffmpeg_build_for_ffms2 --enable-gpl --enable-version3 --disable-encoders --disable-muxers --disable-doc --enable-debug --disable-devices --disable-avdevice --enable-libdav1d --enable-pic --extra-cflags="-march=native"
make -j$(nproc)
make install

(**)
git clone git://github.com/FFMS/ffms2 ("git://" does not respond, so I am using "https://")
cd ffms2
PKG_CONFIG_PATH=$HOME/ffmpeg_build_for_ffms2/lib/pkgconfig CPPFLAGS="-I/usr/include/avisynth" ./autogen.sh --prefix=$HOME/.local --libdir=$HOME/.local/lib/avisynth --enable-shared --enable-avisynth
make -j$(nproc)
----8<----
- at this point I get the failure from above about relocation
(1) mv /home/romashka/ffmpeg_build_for_ffms2/lib/libavcodec.a /home/romashka/ffmpeg_build_for_ffms2/lib/libavcodec.a.old
(2) mv /home/romashka/ffmpeg_build_for_ffms2/lib/libswscale.a /home/romashka/ffmpeg_build_for_ffms2/lib/libswscale.a.old
(3) make -j$(nproc)
- at this point I get the undefined references failure
(4) move the files from (1) & (2) back
(5) make -j$(nproc)
- at this point it builds fine
----8<----
make install


So everything seemed to have worked. But then I run "ffplay film.avs" and get an error:
[avisynth @ 0x7f554c000c80] Cannot load file '/home/romashka/.local/lib/avisynth/libffms2.so'. Reason: /home/romashka/.local/lib/avisynth/libffms2.so: undefined symbol: ff_mjpeg_val_ac_chrominance
(film.avs, line 20)
- Line 20 is a call to FFVideoSource()

[qyot27]

Moving the FFmpeg libraries around manually outside the build process is what is breaking FFMS2, because you've removed the thing it *needs* to point to; the fact that it then builds absolutely means it's linking to the wrong library.

FFMS2 *should* be using -Wl,-Bsymbolic automatically when being pointed at a static FFmpeg, but if that's somehow broken, you have to provide it yourself to LDFLAGS.

[forgetfool]

So what you are saying is that I need it to build naturally without the things I do in between ----8<----, right? Then it will link to the right libraries and will have no missing symbols?

This is what fails:

echo " CXXLD " src/core/libffms2.la;/bin/sh ./libtool --silent --tag=CXX --mode=link g++ -std=c++11 -fvisibility=hidden -O3 -Wall -Wextra -version-info 5:0:0 -o src/core/libffms2.la -rpath /home/romashka/.local/lib/avisynth src/core/audiosource.lo src/core/ffms.lo src/core/filehandle.lo src/core/indexing.lo src/core/track.lo src/core/utils.lo src/core/videosource.lo src/core/videoutils.lo src/core/zipfile.lo src/vapoursynth/vapoursource.lo src/vapoursynth/vapoursynth.lo src/avisynth/avssources.lo src/avisynth/avisynth.lo -L/home/romashka/ffmpeg_build_for_ffms2/lib -lavformat -lbz2 -lm -lavcodec -lm -llzma -ldav1d -lz -lswscale -lswresample -lavutil -pthread -lva-drm -lva-x11 -lvdpau -lm -lva -latomic -lX11 -lz

it looks like it's missing -Wl,-Bsymbolic. I'll try to see how to enable that.

[forgetfool]

I found this during the configure of ffms2:

checking whether -Wl,-Bsymbolic is needed... no

So I am missing something? configure file almost makes no sence to me ...

[forgetfool]

So I forced these things: -Wl,-Bsymbolic in LDFLAGS

It built.

AND WORKS!

So whats missing in the line below to get it included automatically?
PKG_CONFIG_PATH=$HOME/ffmpeg_build_for_ffms2/lib/pkgconfig CPPFLAGS="-I/usr/include/avisynth" ./autogen.sh --prefix=$HOME/.local --libdir=$HOME/.local/lib/avisynth --enable-shared --enable-avisynth

[qyot27]

Nothing you can do to FFMS2 will make it add Bsymbolic automatically. The way FFmpeg was built caused the disconnect. There are probably scenarios where FFMS2 will correctly detect that Bsymbolic is needed and add it, but in other cases, it's going to always be a requirement that the user pass it to LDFLAGS.

PKG_CONFIG_PATH=$HOME/ffmpeg_build_for_ffms2/lib/pkgconfig CPPFLAGS="-I/usr/include/avisynth" LDFLAGS="-Wl,-Bsymbolic" ./autogen.sh --prefix=$HOME/.local --libdir=$HOME/.local/lib/avisynth --enable-shared --enable-avisynth

[forgetfool]

Thank you, qyot27, once again. Final input from me for now: it looks like I was simply missing --enable-shared flag when running ../configure for ffmpeg build (see above). Including the flag fixed the overall build for ffms2. For some reason the flag was not required before...

[qyot27]

When you're building a static build of FFmpeg, of course you don't want to use --enable-shared. It's not 'required' now either, because that's a completely different type of build.

[forgetfool]

Would anyone have an idea why both
1) ffplay film.avs
2) ffmpeg -i film.avs film.mkv
produce segmentation fault at the end?

Or is this question out of place here?

The stream or output file end up fine...

(gdb) bt
#0 0x00007f197409c89e in free () at /usr/lib/libc.so.6
#1 0x00007f1973c181ba in Cache::~Cache() () at /usr/lib/libvmaf.so.1
#2 0x00007f196374d06e in Cache::~Cache() () at /usr/lib/libavisynth.so
#3 0x00007f196376e58e in PClip::~PClip() () at /usr/lib/libavisynth.so
#4 0x00007f196374d0de in CacheGuard::~CacheGuard() () at /usr/lib/libavisynth.so
#5 0x00007f196374d11e in CacheGuard::~CacheGuard() () at /usr/lib/libavisynth.so
#6 0x00007f196376e58e in PClip::~PClip() () at /usr/lib/libavisynth.so
#7 0x00007f19637d6801 in () at /usr/lib/libavisynth.so
#8 0x00007f196376e58e in PClip::~PClip() () at /usr/lib/libavisynth.so
#9 0x00007f19637d6881 in () at /usr/lib/libavisynth.so
#10 0x00007f196376f8b4 in AVSValue::DESTRUCTOR() () at /usr/lib/libavisynth.so
#11 0x00007f196376f94e in AVSValue::~AVSValue() () at /usr/lib/libavisynth.so
#12 0x00007f196373ab98 in ScriptEnvironment::~ScriptEnvironment() () at /usr/lib/libavisynth.so
#13 0x00007f1963748fae in ScriptEnvironment::DeleteScriptEnvironment() () at /usr/lib/libavisynth.so
#14 0x00007f196374c9af in avs_delete_script_environment () at /usr/lib/libavisynth.so
#15 0x00007f1975a3c355 in () at /usr/lib/libavformat.so.59
#16 0x00007f1975a3c40f in () at /usr/lib/libavformat.so.59
#17 0x00007f1975a8bf85 in avformat_close_input () at /usr/lib/libavformat.so.59
#18 0x000055c525cf5241 in ()
#19 0x000055c525cf57a5 in ()
#20 0x000055c525ce55c5 in main ()

[forgetfool]

BLOODY HELL! This is impossible...

I have done everything again from complete scratch from the very start as I mention above (just to have --enable-debug for ffms2) and now everything just locks up... It's like that initial deadlock if you use original ffms2, except that it definitely uses my the manually built libffms2.so (if don't do AddAutoloadDir("/home/romashka/.local/lib/avisynth") it complains about not finding FFVideoSource).

I have done everything without --enable-debug, from scratch (just to be sure)... ffplay film.avs still locks up

ffplay film.avs
ffplay version n5.0.1 Copyright (c) 2003-2022 the FFmpeg developers
built with gcc 12.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100
nan : 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0

:confused::mad::angry:

[forgetfool]

It's even crazier:

LOCKS UP (all proper files/func names) -
AddAutoloadDir("/home/romashka/.local/lib/avisynth")
a = FFAudioSource("audio.mp3")
v = FFVideoSource("video.AVI")
AudioDub(v, a)

LOCKS UP (no audio involved) -
AddAutoloadDir("/home/romashka/.local/lib/avisynth")
FFVideoSource("video.AVI")

RAISES AN ERROR (as expected, no such video) -
AddAutoloadDir("/home/romashka/.local/lib/avisynth")
FFVideoSource("no-such-video.AVI")

LOCKS UP (????????!!!!!)
AddAutoloadDir("/home/romashka/.local/lib/avisynth")
a = FFAudioSource("audio.mp3")
v = FFVideoSource("no-such-video.AVI")
AudioDub(v, a)

RAISES AN ERROR (as expected, wrong func name) -
AddAutoloadDir("/home/romashka/.local/lib/avisynth")
FFVideoSourceblahblahblah("no-such-video.AVI")

LOCKS UP (????????!!!!!)
AddAutoloadDir("/home/romashka/.local/lib/avisynth")
a = FFAudioSource("audio.mp3")
v = FFVideoSourceblahblahblah("no-such-video.AVI")
AudioDub(v, a)



LSMASHSource(), DirectShowSource(), FFmpegSource() all lock up (when using just them with a valid file)

[qyot27]

Arch doesn't put the subdirectories in ~/.local on the system search paths; you have to do that yourself. (https://bbs.archlinux.org/viewtopic.php?id=249477)

To reiterate again: build the minimal FFmpeg as static, and configure FFMS2 with LDFLAGS="-Wl,-Bsymbolic".

[forgetfool]

Ok, I got it working cleaning EVERYTHING out first. This is my process.

for minimal FFMPEG:
1) ../configure --prefix=$HOME/ffmpeg_build_for_ffms2 --enable-gpl --enable-version3 --disable-encoders --disable-muxers --disable-doc --enable-debug --disable-devices --disable-avdevice --enable-libdav1d --extra-cflags="-march=native"
2) make -j$(nproc)
3) make install

This puts all the built stuff into $HOME/ffmpeg_build_for_ffms2


for FFMS2:
1) PKG_CONFIG_PATH=$HOME/ffmpeg_build_for_ffms2/lib/pkgconfig CPPFLAGS="-I/usr/include/avisynth" ./autogen.sh --prefix=$HOME/.local --libdir=$HOME/.local/lib/avisynth --enable-shared --enable-avisynth
2) make -j$(nproc)
3) make install

Currently 2) fails with the message
"lib/libavcodec.a(vc1dsp_mmx.o): relocation R_X86_64_PC32 against symbol `ff_pw_9' can not be used when making a shared object; recompile with -fPIC"

Everything goes smoothly if I manually enable the required flags in configure (src_core_libffms2_la_LDFLAGS="-Wl,-Bsymbolic"). They should be enabled automatically, I think, from the code below, but they are not:

.....
if ac_fn_c_try_link "$LINENO"
then :
eval no_bsymbolic=yes
else $as_nop
eval no_bsymbolic=no
fi
rm -f core conftest.err conftest.$ac_objext conftest.beam \
conftest$ac_exeext conftest.$ac_ext

if test "$no_bsymbolic" = "no"; then
LDFLAGS="$LDFLAGS -Wl,-Bsymbolic"
.....


So I re-run 2), then run 3) and it puts libffms2.so into $HOME/.local/lib/avisynth, and that's the one that is being used (I verified that).

(Perhaps, I MUST CLEAN EVERYTHING, otherwise it links to old/global things and that's why I get a deadlock, like with the main package FFMS2...)



"/usr/bin/ffplay -v trace film.avs" (i.e. using the official FFMPEG package) works fine, however, it produces segfault after the rendering is finished.

Initialized opengl renderer.
ffplay version n5.0.1 Copyright (c) 2003-2022 the FFmpeg developers
built with gcc 12.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100
Initialized opengl renderer.
[NULL @ 0x7fae94000c80] Opening '/home/romashka/Downloads/video-test/film-test-linux.avs' for reading
[file @ 0x7fae940015c0] Setting default whitelist 'file,crypto,data'
[avisynth @ 0x7fae94000c80] Format avisynth probed with size=2048 and score=50
[avisynth @ 0x7fae94000c80] avs_is_field_based: 0 sq= 0B f=0/0
[avisynth @ 0x7fae94000c80] avs_is_parity_known: 0
[avisynth @ 0x7fae94000c80] Before avformat_find_stream_info() pos: 0 bytes read:1541 seeks:0 nb_streams:2
[avisynth @ 0x7fae94000c80] All info found 0KB sq= 0B f=0/0
[avisynth @ 0x7fae94000c80] stream 0: start_time: 0 duration: 41
[avisynth @ 0x7fae94000c80] stream 1: start_time: 0 duration: 242.86
[avisynth @ 0x7fae94000c80] format: start_time: 0 duration: 242.86 (estimate from stream) bitrate=0 kb/s
[avisynth @ 0x7fae94000c80] After avformat_find_stream_info() pos: 0 bytes read:1541 seeks:0 frames:3.....
...
... all the frames rendering...
...
Segmentation Fault



So what you meant by "STATIC build" is what I already have via main FFMPEG package (/usr/bin/ffmpeg & friends). But, I think, it is not static. It's just "complete" so to speak, not "minimal", like the guy I am building for FFMS2 to work. The idea was to build a minimal FFMPEG, just so that we can build FFMS2 manually. Can you please explain in more detail why we needed that, i.e. why the FFMS2 that comes with a distro would not work (i.e. causes deadlock)? Can it be made to work? What's needed for that?


Now back to segfault. I guess the problem is with the different versions of libs:

package ("complete") FFMPEG has:
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100

manually built ("minimal") FFMPEG has:
libavutil 57. 27.100 / 57. 27.100
libavcodec 59. 35.100 / 59. 35.100
libavformat 59. 25.100 / 59. 25.100
libavfilter 8. 41.100 / 8. 41.100
libswscale 6. 6.100 / 6. 6.100
libswresample 4. 6.100 / 4. 6.100
libpostproc 56. 5.100 / 56. 5.100


Can you please take time to read the above. I really want to understand what's going on. Thank you, and I really appreciate you spending time on this!