may24
24th March 2022, 10:50
Hi all
... it's been some time ...
Now I realize that my x265.exe was quite a bit out of date and I decided to build a more actual one.
Unfortunately - well it was quite a bit expected - things didn't work "out of the box". And the included documentation turned out quite a "bit scrappy" ...
Currently I'm on Ubuntu 21.10 (64Bit) and try to build a x265.exe (also 64bit).
First things first ... if not already present:
apt-get install git cmake cmake-curses-gui build-essential
And here appears the first pit. The Multicoreware documentation suggests "yasm" for assembly of coded primitives. But it seems that the current code doesn't support yasm anymore. Autodetect doesn't find it and even when explicitly specified, it gets ignored.
So installed "nasm" instead - which gets detected fine.
Now clone the repository:
git clone https://bitbucket.org/multicoreware/x265_git.git
I'm only interested in the 10-Bit Encoder, so I'll further focus on that.
You may want to follow the path down to ''x265_git/build/msys'' and simply run "multilib.sh" ... that won't work. Ok, that would'd been too easy ;)
A quick examination of the bash script shows multiple issues here ...
...
cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF
...
First, the source path is wrong: it should be "../../source"
Second: When you run cmake you'll realize: There is no "MSYS Makefiles" target ! Only "Unix Makefiles" ... plus, no hint to any Win target at all.
ok, beat it ... let's create our own build.cmake
Again I took the suggestions from the multicoreware webside (https://bitbucket.org/multicoreware/x265_git/wiki/CrossCompile), plus a little fine-tuning:
#!cmake
# this one is important
SET(CMAKE_SYSTEM_NAME Windows)
# specify the cross compiler
SET(CMAKE_C_COMPILER x86_64-w64-mingw32-gcc)
SET(CMAKE_CXX_COMPILER x86_64-w64-mingw32-g++)
SET(CMAKE_RC_COMPILER x86_64-w64-mingw32-windres)
SET(CMAKE_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
Ok, let's cmake + make:
cmake -DCMAKE_TOOLCHAIN_FILE=build.cmake ../../source -DHIGH_BIT_DEPTH=ON -DENABLE_SHARED=OFF
make
(Again, I'm just interested in the 10-bit version of the encoder)
Well, it builds and produced a x265.exe of aprox 3MB. Copying the binary to my Win10 box ... run in a cmd ...
x265 [info]: HEVC encoder version 3.5+36-9b59d4554
x265 [info]: build info [Windows][GCC 10.0.0][32 bit][noasm] 10bit
WHAT is this ! :mad: :mad: :mad: :mad: :mad:
Why only 32 bit ?!?! why NOASM ?!?!
... and yes I've installed all cmake + x86_64-w64-mingw32 stuff ... that is not the problem !
So after diggin' for aaaaaaaages, I found out that:
1. For whatever reason cmake doesn't detect my Arch correctly ... or just ignores it. IDK but wtf !!
uname -a
Linux deimos 5.13.0-37-generic #42-Ubuntu SMP Tue Mar 15 14:34:06 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
2. Although NASM get's detected it doesn't mean assembly is enabled! Don't know why exactly ... but there're some strange Warnings about "find_package":
CMake Warning (dev) at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
The package name passed to `find_package_handle_standard_args` (nasm) does
not match the name of the calling package (Nasm). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/FindNasm.cmake:23 (find_package_handle_standard_args)
CMakeLists.txt:359 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
Google says: You can ignore it ... sounds legit
And a quick check of CMakeCache.txt shows: "NASM_EXECUTABLE:FILEPATH=/usr/bin/nasm"
However, I found out that you'll need to manually enable the assembly to make it work. Also you may (probably depending on your OS) force the compiler to build for your achitecture.
So finally I end up with this:
build.cmake
#!cmake
# this one is important
SET(CMAKE_SYSTEM_NAME Windows)
# specify the cross compiler
SET(CMAKE_SYSTEM_PROCESSOR amd64)
SET(CMAKE_C_COMPILER x86_64-w64-mingw32-gcc)
SET(CMAKE_CXX_COMPILER x86_64-w64-mingw32-g++)
SET(CMAKE_RC_COMPILER x86_64-w64-mingw32-windres)
SET(CMAKE_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
cmake -DCMAKE_TOOLCHAIN_FILE=build.cmake ../../source -DHIGH_BIT_DEPTH=ON -DENABLE_SHARED=OFF -DENABLE_ASSEMBLY:BOOL=ON
The result is an aprox. 11Mb binary that shows:
x265 [info]: HEVC encoder version 3.5+36-9b59d4554
x265 [info]: build info [Windows][GCC 10.0.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
Quite a bit in size ... so let's compress it:
upx -9 -v -k --ultra-brute x265.exe
The result is a 993kb executable. Nice !
... and btw. this build runs aprox. 4 times faster then the 32bit-noasm version.
I hope I could help all of you who struggle to compile their x265.exe from source to avoid some pit-falls ... and, just ignore the 10k+ warnings that will flood your screen ;)
cheers
... it's been some time ...
Now I realize that my x265.exe was quite a bit out of date and I decided to build a more actual one.
Unfortunately - well it was quite a bit expected - things didn't work "out of the box". And the included documentation turned out quite a "bit scrappy" ...
Currently I'm on Ubuntu 21.10 (64Bit) and try to build a x265.exe (also 64bit).
First things first ... if not already present:
apt-get install git cmake cmake-curses-gui build-essential
And here appears the first pit. The Multicoreware documentation suggests "yasm" for assembly of coded primitives. But it seems that the current code doesn't support yasm anymore. Autodetect doesn't find it and even when explicitly specified, it gets ignored.
So installed "nasm" instead - which gets detected fine.
Now clone the repository:
git clone https://bitbucket.org/multicoreware/x265_git.git
I'm only interested in the 10-Bit Encoder, so I'll further focus on that.
You may want to follow the path down to ''x265_git/build/msys'' and simply run "multilib.sh" ... that won't work. Ok, that would'd been too easy ;)
A quick examination of the bash script shows multiple issues here ...
...
cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF
...
First, the source path is wrong: it should be "../../source"
Second: When you run cmake you'll realize: There is no "MSYS Makefiles" target ! Only "Unix Makefiles" ... plus, no hint to any Win target at all.
ok, beat it ... let's create our own build.cmake
Again I took the suggestions from the multicoreware webside (https://bitbucket.org/multicoreware/x265_git/wiki/CrossCompile), plus a little fine-tuning:
#!cmake
# this one is important
SET(CMAKE_SYSTEM_NAME Windows)
# specify the cross compiler
SET(CMAKE_C_COMPILER x86_64-w64-mingw32-gcc)
SET(CMAKE_CXX_COMPILER x86_64-w64-mingw32-g++)
SET(CMAKE_RC_COMPILER x86_64-w64-mingw32-windres)
SET(CMAKE_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
Ok, let's cmake + make:
cmake -DCMAKE_TOOLCHAIN_FILE=build.cmake ../../source -DHIGH_BIT_DEPTH=ON -DENABLE_SHARED=OFF
make
(Again, I'm just interested in the 10-bit version of the encoder)
Well, it builds and produced a x265.exe of aprox 3MB. Copying the binary to my Win10 box ... run in a cmd ...
x265 [info]: HEVC encoder version 3.5+36-9b59d4554
x265 [info]: build info [Windows][GCC 10.0.0][32 bit][noasm] 10bit
WHAT is this ! :mad: :mad: :mad: :mad: :mad:
Why only 32 bit ?!?! why NOASM ?!?!
... and yes I've installed all cmake + x86_64-w64-mingw32 stuff ... that is not the problem !
So after diggin' for aaaaaaaages, I found out that:
1. For whatever reason cmake doesn't detect my Arch correctly ... or just ignores it. IDK but wtf !!
uname -a
Linux deimos 5.13.0-37-generic #42-Ubuntu SMP Tue Mar 15 14:34:06 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
2. Although NASM get's detected it doesn't mean assembly is enabled! Don't know why exactly ... but there're some strange Warnings about "find_package":
CMake Warning (dev) at /usr/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:273 (message):
The package name passed to `find_package_handle_standard_args` (nasm) does
not match the name of the calling package (Nasm). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/FindNasm.cmake:23 (find_package_handle_standard_args)
CMakeLists.txt:359 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
Google says: You can ignore it ... sounds legit
And a quick check of CMakeCache.txt shows: "NASM_EXECUTABLE:FILEPATH=/usr/bin/nasm"
However, I found out that you'll need to manually enable the assembly to make it work. Also you may (probably depending on your OS) force the compiler to build for your achitecture.
So finally I end up with this:
build.cmake
#!cmake
# this one is important
SET(CMAKE_SYSTEM_NAME Windows)
# specify the cross compiler
SET(CMAKE_SYSTEM_PROCESSOR amd64)
SET(CMAKE_C_COMPILER x86_64-w64-mingw32-gcc)
SET(CMAKE_CXX_COMPILER x86_64-w64-mingw32-g++)
SET(CMAKE_RC_COMPILER x86_64-w64-mingw32-windres)
SET(CMAKE_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_C_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
SET(CMAKE_SHARED_LIBRARY_LINK_CXX_FLAGS "-static-libgcc -static-libstdc++ -static -O3 -s")
cmake -DCMAKE_TOOLCHAIN_FILE=build.cmake ../../source -DHIGH_BIT_DEPTH=ON -DENABLE_SHARED=OFF -DENABLE_ASSEMBLY:BOOL=ON
The result is an aprox. 11Mb binary that shows:
x265 [info]: HEVC encoder version 3.5+36-9b59d4554
x265 [info]: build info [Windows][GCC 10.0.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
Quite a bit in size ... so let's compress it:
upx -9 -v -k --ultra-brute x265.exe
The result is a 993kb executable. Nice !
... and btw. this build runs aprox. 4 times faster then the 32bit-noasm version.
I hope I could help all of you who struggle to compile their x265.exe from source to avoid some pit-falls ... and, just ignore the 10k+ warnings that will flood your screen ;)
cheers