Cross-Platform D2V Source for VapourSynth

[xekon]

Thank you JoeyMonco, your posts steered me in the correct direction, after a couple google searches I found that you have to install autotools using pacman
now it gets stuck running configure:

$ ./configure
checking for VapourSynth... no
configure: error: Package requirements (vapoursynth) were not met:

No package 'vapoursynth' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables VapourSynth_CFLAGS
and VapourSynth_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.

[JoeyMonco]

And the location to the PKG_CONFIG_PATH as the error states.

[xekon]

what do I need to set VapourSynth_CFLAGS and VapourSynth_LIBS to? seems that would be the easier route, no .pc file needed.

export VapourSynth_CFLAGS=what goes here?
export VapourSynth_LIBS=and what goes here?

[jackoneill]

what do I need to set VapourSynth_CFLAGS and VapourSynth_LIBS to? seems that would be the easier route, no .pc file needed.

export VapourSynth_CFLAGS=what goes here?
export VapourSynth_LIBS=and what goes here?

For the CFLAGS you need the folder where VapourSynth.h is. For the LIBS you just need a harmless string to make configure happy (it can't be empty).


VapourSynth_CFLAGS="-I/some/folder"
VapourSynth_LIBS="-L."

[xekon]

thank you so much jackoneill, i got your other post showing me the workaround as well.
but for the sake of learning I would still like to try building this, I made those changes and have successfully ran ./configure now

now when I tried to run 'make' i was getting TONS of 'undefined reference to `std::__cxx11::basic_string' messages.
I resolved those by opening all the .cpp .hpp files in the d2vsource folder and adding the following to the top of each file:
#define _GLIBCXX_USE_CXX11_ABI 0


Here is my current install routine:
1. msys2-x86_64-20161025.exe http://msys2.github.io/
install to c:\msys64

2. win-builds-1.5.0.exe http://win-builds.org/doku.php/download_and_installation_from_windows
select "Cygwin or MSYS" and "x86_64", on the next page click process in the top right corner.

3. download d2vsource-master and vapoursynth-master, extract and place into C:\msys64\
you should have the following directory structure:
C:\msys64\d2vsource-master\autogen.sh
C:\msys64\vapoursynth-master\include\VapourSynth.h

4. open the MSYS2 MinGW 64-bit shell, enter the following to install autotools:

pacman -S make pkgconfig autoconf automake autogen libtool mingw-w64-x86_64-gcc
pacman -Syuu


5. update your "PKG_CONFIG_PATH", to view its current value enter:
echo ${PKG_CONFIG_PATH}

update it to include win-builds files, also set vapoursynth flags/libs:

export PKG_CONFIG_PATH="/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig:/opt/windows_64/lib64/pkgconfig"
export VapourSynth_CFLAGS="-I/vapoursynth-master/include"
export VapourSynth_LIBS="-L."


6. navigate to the d2vsource folder and build the project:

cd /d2vsource-master
./autogen.sh
./configure
make
make install

[Pat357]

pacman -S mingw-w64-x86_64-gcc
should've been be installed with GCC, but maybe you have another version hanging around ?

If the above doesn't work, try
pacman -S --force mingw-w64-x86_64-gcc

[xekon]

Thank you Pat357, I tried the force command as you showed, and it reinstalled that package, but I am still getting the same errors, is there a way to "force" when running make? something like: "make --force mingw64"?
looking at my shell prompt I can see its in MINGW64 mode, it shows it on the end of my command prompt in purple, just before the directory im currently in.

I updated my current complete install routine in my previous post, have uninstalled and reinstalled a few times so far: https://forum.doom9.org/showpost.php?p=1795225&postcount=155

here is the error message I am getting when running 'make'

$ make
CXX src/core/compat.lo
CXX src/core/d2v.lo
CXX src/core/decode.lo
CXX src/vs/applyrff.lo
CXX src/vs/d2vsource.lo
CXX src/vs/directrender.lo
CXX src/vs/vapoursynth.lo
CXXLD libd2vsource.la
src/core/.libs/compat.o:compat.cpp:(.text+0xde): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x2c2): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x3c5): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x491): undefined reference to `operator delete[](void*, unsigned long long)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x4e1): undefined reference to `operator delete(void*, unsigned long long)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x11f1): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x1b56): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x1be1): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x1bf7): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x1c0d): undefined reference to `std::__throw_out_of_range_fmt(char const*, ...)'
src/core/.libs/d2v.o:d2v.cpp:(.text+0x1c23): more undefined references to `std::__throw_out_of_range_fmt(char const*, ...)' follow
src/core/.libs/d2v.o:d2v.cpp:(.text+0x1cf4): undefined reference to `operator delete[](void*, unsigned long long)'
src/core/.libs/decode.o:decode.cpp:(.text+0x3cc): undefined reference to `operator delete(void*, unsigned long long)'
src/core/.libs/decode.o:decode.cpp:(.text+0x3f4): undefined reference to `operator delete(void*, unsigned long long)'
src/core/.libs/decode.o:decode.cpp:(.text+0x1034): undefined reference to `operator delete(void*, unsigned long long)'
src/vs/.libs/applyrff.o:applyrff.cpp:(.text+0x76b): undefined reference to `operator delete(void*, unsigned long long)'
src/vs/.libs/applyrff.o:applyrff.cpp:(.text+0x58): undefined reference to `operator delete(void*, unsigned long long)'
src/vs/.libs/directrender.o:directrender.cpp:(.text+0x24): more undefined references to `operator delete(void*, unsigned long long)' follow
collect2.exe: error: ld returned 1 exit status
make: *** [Makefile:502: libd2vsource.la] Error 1

also here is the output from checking gcc version gcc -v:
xekon@xekon-PC MINGW64 /d2vsource-master
$ gcc -v
Using built-in specs.
COLLECT_GCC=C:\msys64\mingw64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.3.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../gcc-6.3.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32
--target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap
--with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,objc,obj-c++,fortran,ada --enable-shared --enable-static --enable-libatomic
--enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug
--disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry
--disable-nls --disable-werror --disable-symvers --with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64
--with-isl=/mingw64 --with-pkgversion='Rev1, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
Thread model: posix
gcc version 6.3.0 (Rev1, Built by MSYS2 project)

[jackoneill]

Does it help to know that those missing symbols are part of the standard C++ library (libstdc++)? I have no idea how to fix the problem.

[Pat357]

Since some important bugs have been fixed in d2vsource since v1.0 was released,
I've compiled both ffmpeg and d2vsource from git (6021968)

See https://github.com/dwbuiten/d2vsource/commits/master for changes since d2vsource v1.0

Commits on Dec 24, 2016

@dwbuiten

Merge pull request #31 from dubhater/master
dwbuiten committed on GitHub on 24 Dec 2016

Commits on Dec 9, 2016

@dubhater

applyrff: Make it actually work

dubhater committed on 9 Dec 2016
Commits on Aug 13, 2016

@dubhater

d2vsource: Avoid double free

dubhater committed on 13 Aug 2016
Commits on Aug 9, 2016

@dubhater

Remove outdated comment
dubhater committed on 9 Aug 2016

Commits on Jul 25, 2016

@dwbuiten

Merge pull request #29 from dubhater/master

dwbuiten committed on GitHub on 25 Jul 2016
Commits on Jun 21, 2016

@dubhater

d2vsource: Fix inverted _ColorRange frame property

dubhater committed on 21 Jun 2016


Here are the latest windows binaries (x86 & x64) compiled under Mingw64 using GCC v6.3.0 :

http://www33.zippyshare.com/v/4GGpY6Yb/file.html

Let me know if you have problems with this version.
All credits go to dwbuiten & dubhater for all the coding.

[Pat357]

here is the error message I am getting when running 'make'

I agree that your problem has to do with libstdc++.a.
Can you find this file in /ming64 ? If no were's it on your system ?

Open a MSYS2 console and do pacman -Syu to update everything, just to be sure.

I've seen similar errors after this lib got updated and you use other libs which were compiled before the update i.e 2 different versions can give this error.
Make sure you recompile ffmpeg and ALL it's libs to make sure they all use the latest libstdc++.a (you can temporally rename libstdc++.dll.a)
How do you configure ffmpeg ?

further set LIBS="-lstdc++" right before configure (for d2vsource) like this :
./autogen.sh
[[ -f Makefile || -f config.log ]] && make distclean
LDFLAGS+=" -static-libgcc -static-libstdc++" CXXFLAGS+=" -static-libgcc" \
LIBS="-lstdc++" ./configure ## (yes on the same line)
make
make install

Try this and let me know.
You can also try my compile ;-)

[Daemon404]

Binaries for 1.1 are in the OP.

Somehow I'm not dead.

[KingLir]

Binaries for 1.1 are in the OP.


Thanks! Can't build on macOS, getting this after autogen and configure:

$ make
CXX src/core/compat.lo
CXX src/core/d2v.lo
CXX src/core/decode.lo
src/core/decode.cpp:450:45: warning: 'codec' is deprecated
[-Wdeprecated-declarations]
if (dctx->fctx->streams[i]->codec->codec_type == AVMEDIA...
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavformat/avformat.h:893:21: note:
'codec' has been explicitly marked deprecated here
AVCodecContext *codec;
^
src/core/decode.cpp:491:13: warning: 'avcodec_decode_video2' is deprecated
[-Wdeprecated-declarations]
avcodec_decode_video2(dctx->avctx, out, &av_ret, &dctx->inpkt);
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:4812:5: note:
'avcodec_decode_video2' has been explicitly marked deprecated here
int avcodec_decode_video2(AVCodecContext *avctx, AVFrame *picture,
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:1578:16: warning:
'convergence_duration' is deprecated [-Wdeprecated-declarations]
typedef struct AVPacket {
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:1630:13: note:
'convergence_duration' has been explicitly marked deprecated here
int64_t convergence_duration;
^
src/core/decode.cpp:510:25: warning: 'avcodec_decode_video2' is deprecated
[-Wdeprecated-declarations]
int r = avcodec_decode_video2(dctx->avctx, out, &av_ret...
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:4812:5: note:
'avcodec_decode_video2' has been explicitly marked deprecated here
int avcodec_decode_video2(AVCodecContext *avctx, AVFrame *picture,
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:1578:16: warning:
'convergence_duration' is deprecated [-Wdeprecated-declarations]
typedef struct AVPacket {
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:1630:13: note:
'convergence_duration' has been explicitly marked deprecated here
int64_t convergence_duration;
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:1578:16: warning:
'convergence_duration' is deprecated [-Wdeprecated-declarations]
typedef struct AVPacket {
^
/usr/local/Cellar/ffmpeg/3.2.4/include/libavcodec/avcodec.h:1630:13: note:
'convergence_duration' has been explicitly marked deprecated here
int64_t convergence_duration;
^
6 warnings generated.
CXX src/vs/applyrff.lo
src/vs/applyrff.cpp:192:36: error: expected expression
data->fields.push_back({ i, Progressive });
^
src/vs/applyrff.cpp:193:36: error: expected expression
data->fields.push_back({ i, Progressive });
^
src/vs/applyrff.cpp:196:40: error: expected expression
data->fields.push_back({ i, Progressive });
^
src/vs/applyrff.cpp:197:40: error: expected expression
data->fields.push_back({ i, Progressive });
^
src/vs/applyrff.cpp:200:44: error: expected expression
data->fields.push_back({ i, Progressive });
^
src/vs/applyrff.cpp:201:44: error: expected expression
data->fields.push_back({ i, Progressive });
^
src/vs/applyrff.cpp:207:36: error: expected expression
data->fields.push_back({ i, tff ? Top : Bottom });
^
src/vs/applyrff.cpp:208:36: error: expected expression
data->fields.push_back({ i, tff ? Bottom : Top });
^
src/vs/applyrff.cpp:211:40: error: expected expression
data->fields.push_back({ i, tff ? Top : Bottom });
^
9 errors generated.

[Cary Knoop]

New version works great!

I have to remind myself to take care when filenames contain spaces.

[jackoneill]

Thanks! Can't build on macOS, getting this after autogen and configure:
src/vs/applyrff.cpp:211:40: error: expected expression
data->fields.push_back({ i, tff ? Top : Bottom });
^
9 errors generated.

Oops. Try this:

make clean && ./configure CXXFLAGS='-std=c++11' && make

[KingLir]

Oops. Try this:

make clean && ./configure CXXFLAGS='-std=c++11' && make


Thanks! Works.

[sl1pkn07]

this is ok with the latest ffmpeg 4?


diff --git a/src/core/d2v.hpp b/src/core/d2v.hpp
index f87b1a0..ecdae78 100644
--- a/src/core/d2v.hpp
+++ b/src/core/d2v.hpp
@@ -68,7 +68,7 @@ static const int idct_algo_conv[8] = {
FF_IDCT_AUTO,
FF_IDCT_AUTO,
FF_IDCT_AUTO,
- FF_IDCT_XVIDMMX,
+ FF_IDCT_XVID,
FF_IDCT_SIMPLEMMX
};

diff --git a/src/core/decode.cpp b/src/core/decode.cpp
index b563697..11a0fef 100644
--- a/src/core/decode.cpp
+++ b/src/core/decode.cpp
@@ -242,12 +242,6 @@ decodecontext *decodeinit(d2vcontext *dctx, int threads, string& err)
/* Set the thread count. */
ret->avctx->thread_count = threads;

- /*
- * Enable EMU_EDGE so that we can use buffers that are
- * not padded by 32 pixels.
- */
- ret->avctx->flags |= CODEC_FLAG_EMU_EDGE;
-
/* Use refcounted frames. */
ret->avctx->refcounted_frames = 1;

[sl1pkn07]

done in upstream

[lansing]

Can this plugin works with D2VWitch to index video like this so I can skip a step?

core.d2v.Source(r"my_video.ts")

Also I have problem opening d2v made by dgindex when the video path in the d2v is non English because the dgindex d2v is in ANSI encoding instead of utf-8.

[jackoneill]

Can this plugin works with D2VWitch to index video like this so I can skip a step?

core.d2v.Source(r"my_video.ts")


That would be nice. It's on my to do list.


Also I have problem opening d2v made by dgindex when the video path in the d2v is non English because the dgindex d2v is in ANSI encoding instead of utf-8.

That's unlikely to be fixed in any way. Does DGIndex have a feature you need that D2V Witch doesn't have?

[lansing]

That's unlikely to be fixed in any way. Does DGIndex have a feature you need that D2V Witch doesn't have?

Yeah I know this is a dgindex problem that have been existed since the beginning. I just wanna see if it would be easy enough to fix it on this end.

[ChaosKing]

You don't have to use the full path in d2v. Just uncheck "Use full paths" in options in dgindex and only the file name will be stored.

[lansing]

You don't have to use the full path in d2v. Just uncheck "Use full paths" in options in dgindex and only the file name will be stored.

The problem is still there if my file name is non English.

[lansing]

I'm trying to stress my new Ryzen 3900x with this filter, but I was never able to get cpu utilization over 25%. And setting threads count to 10 is 200fps faster than leaving it at auto, is there something wrong?

[Selur]

Trying to build d2vwitch using:
### d2vwitch
if echo "$args" | grep -q -i -w -E 'all|d2vwitch'
then
mkdir build
cd build

top="$PWD"
export PATH="$top:$PATH"
export PKG_CONFIG_PATH="$top/libs/lib/pkgconfig"

git clone --depth 1 https://github.com/dubhater/D2VWitch
git clone --depth 1 https://github.com/FFmpeg/FFmpeg
git clone --depth 1 https://github.com/vapoursynth/vapoursynth

build_nasm

cd FFmpeg
./configure --prefix="$top/libs" \
--enable-gpl \
--enable-version3 \
--disable-encoders \
--disable-muxers \
--disable-outdevs \
--disable-programs \
--disable-doc \
--disable-debug \
--disable-xlib \
--disable-sdl2 \
--extra-cflags="-ffunction-sections -fdata-sections"
make $MAKEFLAGS
make install

cd ../D2VWitch
autoreconf -if
vapoursynth_CFLAGS="-I../vapoursynth/include" \
vapoursynth_LIBS=" " \
LDFLAGS="-Wl,--gc-sections" \

./configure
make $MAKEFLAGS
strip d2vwitch
cp -f d2vwitch ../..
cd ..

cat <<EOL >../d2vwitch-sources.txt
https://github.com/dubhater/D2VWitch
$(git -C D2VWitch rev-parse HEAD)

https://github.com/vapoursynth/vapoursynth
$(git -C vapoursynth rev-parse HEAD)

https://github.com/FFmpeg/FFmpeg
$(git -C FFmpeg rev-parse HEAD)
EOL

cd ..
rm -rf build
fi
see: https://github.com/Selur/HybridDeploy

problem is, the build stops at:
checking for vapoursynth... no
configure: error: Package requirements (vapoursynth) were not met:

No package 'vapoursynth' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables vapoursynth_CFLAGS
and vapoursynth_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.


Any idea what am I missing? This did work before,..

Cu Selur

[jackoneill]

I think you need to put something that is not whitespace in vapoursynth_LIBS. „-L.” should be harmless.

[Selur]

Thanks for the suggestion, but using "-L." sadly didn't change much:
+ cd ../D2VWitch
+ autoreconf -if
configure.ac:17: installing './compile'
configure.ac:10: installing './config.guess'
configure.ac:10: installing './config.sub'
configure.ac:6: installing './install-sh'
configure.ac:6: installing './missing'
Makefile.am: installing './depcomp'
+ vapoursynth_CFLAGS=-I../vapoursynth/include
+ vapoursynth_LIBS=-L.
+ LDFLAGS=-Wl,--gc-sections
+ ./configure
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for grep that handles long lines and -e... /bin/grep
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for _LARGEFILE_SOURCE value needed for large files... no
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for vapoursynth... no
configure: error: Package requirements (vapoursynth) were not met:

No package 'vapoursynth' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables vapoursynth_CFLAGS
and vapoursynth_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
full log: https://rentry.co/wwf6b
it stops with the same error. :(

Cu Selur

[jackoneill]

cd ../D2VWitch
autoreconf -if
vapoursynth_CFLAGS="-I../vapoursynth/include" \
vapoursynth_LIBS=" " \
LDFLAGS="-Wl,--gc-sections" \

./configure


Perhaps you need a backslash on the empty line as well. Or just put the assignments and the call to configure on the same line. (And use vapoursynth_LIBS="-L.")

[Selur]

cd ../D2VWitch
autoreconf -if
vapoursynth_CFLAGS="-I../vapoursynth/include" vapoursynth_LIBS="-L." LDFLAGS="-Wl,--gc-sections" ./configure
make $MAKEFLAGS

get's me further but still fails, see: https://paste.ee/p/7KJ4O

[jackoneill]

I guess it's time to update that code to match the changes in ffmpeg.

[Jukus]

https://github.com/dwbuiten/d2vsource/issues/48