x265 HEVC Encoder

[pingfr]

@littlepox: yes, the source is actually an already HEVC compressed stream 720p clip grabbed off youtube.

Are you implying that if I get a Blu-Ray 1080p raw uncompressed source to encode piped through avisynth then I might see an increased CPU usage/load?

[littlepox]

@littlepox: yes, the source is actually an already HEVC compressed stream 720p clip grabbed off youtube.

Are you implying that if I get a Blu-Ray 1080p raw uncompressed source to encode piped through avisynth then I might see an increased CPU usage/load?

Yes. under 1080p, there are more than 2x works to do per frame encode. with 720p, you probably have two choices:

1. run multiple instances, this is what we do when we are with >12C24T.
2. modify your settings to play-around with some parameters that could be bottleneck, unable to do multi-threading. As far as I can see, these settings shall help you out with little sacrifice or even better quality under 720p (remember x265 is tuned for 4K, so you do not need to follow exactly even for quality purposes):

--ctu 32 --lookahead-slices 4 --me-range 25

[pingfr]

@littlepox: I think I'll stay with the --ctu 64 --lookahead-slices 1 and --me-range 57 from the default's veryslow profile for now.

I'll just find a 1080p source material to encode tonight or something.

Will return and post any new findings as soon as I have anything new to share with you guys.

[pradeeprama]

@Ma

Thanks for the explanation, now it seems we can just use the v1.9 you compiled without the commit, or manually add --pools "N,N" in the CLI. The rest shall be simple as you said to use separated logic on Win/Linux.



Thanks for the extensive testing @littlepox & Ma.

In conclusion, it looks like using --pools +,+ with the x265+1.9 build spawns threads on both nodes, but the second node isn't seeing a lot of utilization. If you use --pools 24,24, again threads are spawned in both nodes and you explicitly see much higher utilization.

If this is correct, then the issue is one of the Windows scheduler not being intelligent enough to schedule things correctly. I will take a look at the Windows API again to see if there are some hints we can provide to make the scheduling better.

If this is not the problem, then I am lost in the messages :-(.

Also, @littlepox, if you're adding --pools 24,24 to your command line explicitly, there is no need to go for the binary that Ma generated - x265's default 1.9 binary should work just fine for you.

[littlepox]

Thanks for the extensive testing @littlepox & Ma.

In conclusion, it looks like using --pools +,+ with the x265+1.9 build spawns threads on both nodes, but the second node isn't seeing a lot of utilization. If you use --pools 24,24, again threads are spawned in both nodes and you explicitly see much higher utilization.

If this is correct, then the issue is one of the Windows scheduler not being intelligent enough to schedule things correctly. I will take a look at the Windows API again to see if there are some hints we can provide to make the scheduling better.

If this is not the problem, then I am lost in the messages :-(.

Also, @littlepox, if you're adding --pools 24,24 to your command line explicitly, there is no need to go for the binary that Ma generated - x265's default 1.9 binary should work just fine for you.

Exactly as you said. Also, we like to use MA's build because we do NOT need to add anything, it can automatically work for all hardware.

This is what we are expecting for improvements: x265 should always choose the way to maximize utility with other things equal; please find some way to cope with the windows API, if not, separation of the logic may be a solution.

[Ma]

In conclusion, it looks like using --pools +,+ with the x265+1.9 build spawns threads on both nodes, but the second node isn't seeing a lot of utilization. If you use --pools 24,24, again threads are spawned in both nodes and you explicitly see much higher utilization.

If this is correct, then the issue is one of the Windows scheduler not being intelligent enough to schedule things correctly. I will take a look at the Windows API again to see if there are some hints we can provide to make the scheduling better.

My understanding is: '--pools +,+' and no pools options works the same -- once fast on both nodes, once slow on one node (but more often slow); '--pools 24,24' options works fast on both nodes every time.

My feelings are: if Windows starts on both nodes it is fast to the end of encoding, if Windows starts on one node it is slow to the end of encoding -- maybe Windows doesn't want to move thread from one node to another if the thread is at 100% workload.

[Ma]

without --pools 24,24:
x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1
BUT numa node 1 is not utilized:
http://img.2222.moe/images/2016/02/11/wrong.png

At the time when this screenshot was taken was there any heavy process running (beside x265)? It looks like there are 6 to 10 threads on numa node 1 (beautifully distributed only at physical cores).

[littlepox]

At the time when this screenshot was taken was there any heavy process running (beside x265)? It looks like there are 6 to 10 threads on numa node 1 (beautifully distributed only at physical cores).

No, not any.

[Ma]

No, not any.

Thanks for info. It could be the case that "it's not a bug, it's a feature". x265 code looks OK, Windows schedules threads on both numa nodes but node 0 gets more threads (for example 42 threads to node 0 and 6 threads to node 1). x265 in that case works slow.
-------------
I think that the best solution is to leave threadpool code as is and only to change logic of allocating threads on numa nodes. First attempt in attachment, VS 2015 build of stable version 1.9+3 with new logic (only for Windows OS):
www.msystem.waw.pl/x265/x265-vs2015-stable-1.9+3.7z

[pradeeprama]

Thanks for info. It could be the case that "it's not a bug, it's a feature". x265 code looks OK, Windows schedules threads on both numa nodes but node 0 gets more threads (for example 42 threads to node 0 and 6 threads to node 1). x265 in that case works slow.
-------------
I think that the best solution is to leave threadpool code as is and only to change logic of allocating threads on numa nodes. First attempt in attachment, VS 2015 build of stable version 1.9+3 with new logic (only for Windows OS):
www.msystem.waw.pl/x265/x265-vs2015-stable-1.9+3.7z

Thanks for the patch Ma. I see that you're trying to resort --pools +,+ to effectively mean --pools 24,24 (or whatever the # threads per socket is) on Windows. For this use-case, I'm sure it'll work better but I'm not sure if this is universally applicable across other videos/machines. We will do more testing on a windows multi-socket system and report back with what we see. I am also a little uncomfortable keeping one meaning of --pools +,+ on linux and another on windows.

[LigH]

What exactly is the changed condition to enable 2-pass CRF? If I read the patches correctly: Use a VBV maximum bitrate in the 1st pass, so you can either let x265 calculate the optimal RF with VBV constraints to achieve a target size (in VBR mode), or use a specified RF with VBV constraints regardless of a target size (in VBV-CRF mode)?
__

New build with changed 2-pass VBV-CRF conditions:

x265 1.9+15-425b583f25db (https://www.mediafire.com/download/8f06cr79fe2besb/x265_1.9+15-425b583f25db.7z) (GCC 5.3.0)

[Ma]

I see that you're trying to resort --pools +,+ to effectively mean --pools 24,24 (or whatever the # threads per socket is) on Windows.

Yes, for '--pools +,+' and '--pools *' and no pools options it simulates '--pools 24,24' option. I think it is simple and more effective than current code and better than backout commit 10983.

I hope that you find better fix.

[qyot27]

I apologize for not following up on this much earlier, but this is a basic rundown of the issues when cross-compiling x265-yuuki currently:

input-filters.diff doesn't cleanly apply because the amount of fuzz got too large.

CMake has to be run twice to even detect that -DENABLE_LAVF has been turned on and get the locations for the libs. This is due to the position of the ENABLE_LAVF block in CMakeLists.txt
coming before the ENABLE_CLI block, but requiring the CLI is enabled. If the ENABLE_LAVF block is moved to a position after the CLI is turned on, or if the CLI requirement is removed
from the ENABLE_LAVF block, then it can detect the libs on the first try (although it seemed to cause other issues during building or linking).

There are problems linking against FFmpeg's private libs, especially when all of them are built static. zlib, bzip2, lzma, and iconv are all autodetected and used if present when FFmpeg is configured,
and these cause errors on link with the lavf module because the libs haven't been properly added to the linker list by cmake. To say nothing of when more expansive configurations with more
external libs are present, or for other options that are enabled by default (like what's brought in if the user doesn't explicitly use --disable-network). If the user explicitly disables these four libraries
and the network support when building FFmpeg, then x265 can link to them just fine, but otherwise it requires the user to manually append the list of linker libs to the failed link command, and this
then breaks the ability of ninja to do a proper install, even though the .exe is okay.

FFmpeg API usage needs to be updated. PIX_FMT_* needs to be changed to AV_PIX_FMT_* to allow compilation to proceed successfully, and even though it's just a warning, uses of deprecated
functions should be updated to non-deprecated ones (which x264 recently fixed for the lavf module there):

[78/88] Building CXX object CMakeFiles/cli.dir/input/lavf.cpp.obj
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp: In member function 'bool x265::LavfInput::readPicture(x265_picture&, x265::InputFileInfo*)':
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:108:13: warning: 'void av_free_packet(AVPacket*)' is deprecated [-Wdeprecated-declarations]
av_free_packet(&pkt);
^
In file included from /usr/i686-w64-mingw32/include/libavformat/avformat.h:318:0,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.h:12,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:29:
/usr/i686-w64-mingw32/include/libavcodec/avcodec.h:4040:6: note: declared here
void av_free_packet(AVPacket *pkt);
^
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:108:13: warning: 'void av_free_packet(AVPacket*)' is deprecated [-Wdeprecated-declarations]
av_free_packet(&pkt);
^
In file included from /usr/i686-w64-mingw32/include/libavformat/avformat.h:318:0,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.h:12,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:29:
/usr/i686-w64-mingw32/include/libavcodec/avcodec.h:4040:6: note: declared here
void av_free_packet(AVPacket *pkt);
^
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:108:32: warning: 'void av_free_packet(AVPacket*)' is deprecated [-Wdeprecated-declarations]
av_free_packet(&pkt);
^
In file included from /usr/i686-w64-mingw32/include/libavformat/avformat.h:318:0,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.h:12,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:29:
/usr/i686-w64-mingw32/include/libavcodec/avcodec.h:4040:6: note: declared here
void av_free_packet(AVPacket *pkt);
^
[88/88] Linking CXX executable x265.exe
[8bit:$]

The ENABLE_LAVF block in input.cpp should really probably be changed to accept any file format lavf can. AviSynth was the main reason to argue it before, but really, only supporting *.mp4 and
*.mkv as input is too limiting (other containers and secondary extensions like *.m2ts, *.ts, *.mpg, *.ogv, *.webm, *.vob and several others are also found commonly enough to be nice to open).
The easiest solution would be to simply make the block into an else and the return statement, so anything that's not y4m or a pipe gets handled by lavf (and presumably lavf would error out in the
case that it can't open the file, although this probably needs more error-proofing in input/lavf.cpp unless it already does emit a clean error and not a crash).


The zimg filter also fails to build because of an error in zimgfilter.h:

[86/88] Building CXX object CMakeFiles/cli.dir/filters/filters.cpp.obj
FAILED: /usr/bin/i686-w64-mingw32-g++ -DENABLE_LAVF -DENABLE_LSMASH -DENABLE_MKV -DENABLE_ZIMG -DEXPORT_C_API=1 -DHAVE_INT_TYPES_H=1 -DHIGH_BIT_DEPTH=0 -DX265_ARCH_X86=1 -DX265_DEPTH=8 -DX265_NS=x265 -D_WIN32_WINNT=_WIN32_WINNT_WINXP -D_WIN32_WINNT_WIN7=0x0601 -D__STDC_LIMIT_MACROS=1 -mfpmath=sse -march=pentium3 -msse -mtune=pentium3 -O3 -DNDEBUG -I/home/qyot27/x265-build/x265-yuuki/source/. -I/home/qyot27/x265-build/x265-yuuki/source/common -I/home/qyot27/x265-build/x265-yuuki/source/encoder -I. -Wall -Wextra -Wshadow -std=gnu++98 -march=i686 -Wno-array-bounds -ffast-math -mstackrealign -fno-exceptions -MMD -MT CMakeFiles/cli.dir/filters/filters.cpp.obj -MF CMakeFiles/cli.dir/filters/filters.cpp.obj.d -o CMakeFiles/cli.dir/filters/filters.cpp.obj -c /home/qyot27/x265-build/x265-yuuki/source/filters/filters.cpp
In file included from /home/qyot27/x265-build/x265-yuuki/source/filters/filters.cpp:27:0:
/home/qyot27/x265-build/x265-yuuki/source/filters/zimgfilter.h:56:5: error: 'zimg_resize_context' does not name a type
zimg_resize_context* resizeCtx[3];
^
/home/qyot27/x265-build/x265-yuuki/source/filters/zimgfilter.h:57:5: error: 'zimg_depth_context' does not name a type
zimg_depth_context* depthCtx;
^
[86/88] Building CXX object CMakeFiles/cli.dir/x265.cpp.obj
ninja: build stopped: subcommand failed.
This is with a git build of zimg, so 2.0.4 or something. With the older 1.1.1 release (which I tested only because I thought maybe it might have been expecting that), there's even more compiler
noise thrown and the above errors still apply.


For reference, this is the set of instructions I'm following:
FFmpeg:
git clone git://source.ffmpeg.org/ffmpeg.git && \
cd ffmpeg && \
./configure --prefix=/usr/i686-w64-mingw32 --cross-prefix=i686-w64-mingw32- \
--enable-gpl --enable-version3 --disable-w32threads --enable-avresample \
--disable-encoders --disable-muxers --disable-doc --disable-debug \
--disable-devices --disable-avdevice --enable-avisynth --cpu=pentium3 \
--extra-cflags="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
--target-os=mingw32 --arch=x86 && \
make -j$(nproc) && \
sudo checkinstall --pkgname=ffmpeg-minimal-mingw --pkgversion="$(git \
rev-list --count HEAD)-g$(git rev-parse --short HEAD)-$(date --rfc-3339=date | \
sed 's/-//g')-git" --backup=no --deldoc=yes --delspec=yes --deldesc=yes \
--strip=yes --fstrans=no --default

Add --disable-network --disable-zlib --disable-bz2 --disable-lzma to keep FFmpeg from
calling in private libs, otherwise, see pkg-config note at the end.

mkdir -p ~/x265-build/12bit ~/x265-build/10bit ~/x265-build/8bit && \
cd ~/x265-build && \
hg qclone https://bitbucket.org/msg7086/x265-yuuki && \
cd x265-yuuki && \
hg qpush -a && \

Condense the ifdef ENABLE_LAVF block in source/input/input.cpp to:
#ifdef ENABLE_LAVF
else
return new LavfInput(info);
#endif

Correct uses of PIX_FMT_* to AV_PIX_FMT_*
sed -i 's/case PIX_FMT/case AV_PIX_FMT/g' source/input/lavf.cpp

cd ../12bit && \
cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="$HOME/x265_build/x265-12bit" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DHIGH_BIT_DEPTH:bool=on -DMAIN12:bool=on -DENABLE_SHARED:bool=off \
-DENABLE_ASSEMBLY:bool=off -DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off \
[-DWINXP_SUPPORT:bool=on] && \
ninja && \
sudo checkinstall --pkgname=x265-main12-mingw --pkgversion="$(grep X265_VERSION \
build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
--rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
--deldesc=yes --strip=yes --fstrans=no --default cp libx265.a /usr/i686-w64-mingw32/lib/libx265_main12.a && \
mv *.deb ~/mingw_debs

# Build 10-bit:
cd ../10bit && \
cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="$HOME/x265_build/x265-10bit" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DHIGH_BIT_DEPTH:bool=on -DENABLE_SHARED:bool=off -DENABLE_ASSEMBLY:bool=off \
-DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off [-DWINXP_SUPPORT:bool=on] && \
ninja && \
sudo checkinstall --pkgname=x265-main10-mingw --pkgversion="$(grep X265_VERSION \
build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
--rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
--deldesc=yes --strip=yes --fstrans=no --default cp libx265.a /usr/i686-w64-mingw32/lib/libx265_main10.a && \
mv *.deb ~/mingw_debs

# Only the .a files from 12-bit and 10-bit are installed to the system,
# to reduce the chances of conflicts.

# Build 8-bit:
cd ../8bit && \
cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="/usr/i686-w64-mingw32" \
-DCMAKE_PREFIX_PATH="/usr/i686-w64-mingw32" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DENABLE_SHARED:bool=off -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT:bool=on -DLINKED_12BIT:bool=on \
-DEXTRA_LIB="/usr/i686-w64-mingw32/lib/libx265_main10.a;/usr/i686-w64-mingw32/lib/libx265_main12.a" \
-DENABLE_LSMASH:bool=on -DENABLE_MKV:bool=on -DENABLE_LAVF:bool=on [-DWINXP_SUPPORT:bool=on] && \

cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="/usr/i686-w64-mingw32" \
-DCMAKE_PREFIX_PATH="/usr/i686-w64-mingw32" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DENABLE_SHARED:bool=off -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT:bool=on -DLINKED_12BIT:bool=on \
-DEXTRA_LIB="/usr/i686-w64-mingw32/lib/libx265_main10.a;/usr/i686-w64-mingw32/lib/libx265_main12.a" \
-DENABLE_LSMASH:bool=on -DENABLE_MKV:bool=on -DENABLE_LAVF:bool=on [-DWINXP_SUPPORT:bool=on] && \
ninja && \
sed -i 's/lx265/lx265 -lx265_main10 -lx265_main12/' x265.pc && \
sudo checkinstall --pkgname=x265-mingw --pkgversion="$(grep X265_VERSION \
build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
--rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
--deldesc=yes --strip=yes --fstrans=no --default ninja install && \
mv *.deb ~/mingw_debs

CMake step needs to be run twice so FFmpeg is detected, but it only gets detected
if it's installed to the system mingw-w64 area. Then, if the zlib/bz2/lzma/iconv libs
and network support are still enabled, it fails to link. This can be fixed by appending the output of:
PKG_CONFIG_PATH=/usr/i686-w64-mingw32/lib/pkgconfig pkg-config --cflags --libs --static libavcodec libavformat libswscale

to the failed link command*. After this, the install step doesn't work, but x265.exe does.

*which, if following the FFmpeg instructs at the beginning, should be something like:
-I/usr/i686-w64-mingw32/include -lavcodec -lws2_32 -liconv -lm -llzma -lbz2 -lz -lpsapi -ladvapi32 -lshell32 -lswresample -lavutil -lavformat -pthread -lswscale -L/usr/i686-w64-mingw32/lib

sudo apt-get purge ffmpeg-minimal-mingw
(this needs to be purged to prevent it interfering with future ffmpeg or mpv builds)

[MeteorRain]

I apologize for not following up on this much earlier, but this is a basic rundown of the issues when cross-compiling x265-yuuki currently:

Thank you for all the efforts.

Please note that I have moved my repo to Github (https://github.com/msg7086/x265-Yuuki-Asuna) due to some personal reason.

And if you can provide some fixes, feel free to send a PR for me.

The zimg referenced is a pretty old version, mainly due to when I built this patch, it was still in 1.x . If you somehow port it to 2.x, please send a PR. I don't have much time to look into it for the moment.

My builds on Linux (and was successful) are compiled against:
zimg 1.1.1
ffmpeg 2.8.5-1+b1 (debian 9 sys pkg)
l-smash rev1384

[MeteorRain]

I apologize for not following up on this much earlier, but this is a basic rundown of the issues when cross-compiling x265-yuuki currently:

input-filters.diff doesn't cleanly apply because the amount of fuzz got too large.

Using HG is always a hassle for me. Moving to git and I'm feeling much better when dealing with patches.

CMake has to be run twice to even detect that -DENABLE_LAVF has been turned on and get the locations for the libs.

Thanks, this has been fixed and pushed.

There are problems linking against FFmpeg's private libs, especially when all of them are built static. zlib, bzip2, lzma, and iconv are all autodetected and used if present when FFmpeg is configured,
and these cause errors on link with the lavf module because the libs haven't been properly added to the linker list by cmake. To say nothing of when more expansive configurations with more
external libs are present, or for other options that are enabled by default (like what's brought in if the user doesn't explicitly use --disable-network). If the user explicitly disables these four libraries
and the network support when building FFmpeg, then x265 can link to them just fine, but otherwise it requires the user to manually append the list of linker libs to the failed link command, and this
then breaks the ability of ninja to do a proper install, even though the .exe is okay.

I guess, with the latest patch, cmake should be able to detect your pkgconfig files and automatically include these dependencies into consideration. If not, we'll see if someone will send a PR to fix it. I don't have much experience on cmake, sorry.

FFmpeg API usage needs to be updated. PIX_FMT_* needs to be changed to AV_PIX_FMT_* to allow compilation to proceed successfully, and even though it's just a warning, uses of deprecated
functions should be updated to non-deprecated ones (which x264 recently fixed for the lavf module there):

These have been fixed with latest patch. BTW I couldn't find the x264 fixes so I modified it according to your description.

The ENABLE_LAVF block in input.cpp should really probably be changed to accept any file format lavf can.

The LAVF part is ported poorly. Besides, I planned to have VFR support but didn't actually finished that. So I cannot guarantee that it's working well with various of types. I'd probably extend the list a bit, but I'd like to keep it minimal until someone has actually reviewed the code.

The zimg filter also fails to build because of an error in zimgfilter.h:

This is with a git build of zimg, so 2.0.4 or something. With the older 1.1.1 release (which I tested only because I thought maybe it might have been expecting that), there's even more compiler
noise thrown and the above errors still apply.

This is on my todo-list, but is also open for PR.

[qyot27]

Using HG is always a hassle for me. Moving to git and I'm feeling much better when dealing with patches.
I agree, I have practically zero working knowledge of Mercurial (enough to clone and look at the log, and I managed to figure out qrefresh/qpush when dealing with the patchsets). I actually was more or less ready to send a pull request on Github, but it looks like I'll need to review which changes can go through now.

I guess, with the latest patch, cmake should be able to detect your pkgconfig files and automatically include these dependencies into consideration. If not, we'll see if someone will send a PR to fix it. I don't have much experience on cmake, sorry.
The solution I'd ultimately came up with was to spin the case of a static FFmpeg off into its own option so that the existing ENABLE_LAVF continues to work as it always has with shared libs.

I'll test to see if the fix I'd implemented is irrelevant now.

These have been fixed with latest patch. BTW I couldn't find the x264 fixes so I modified it according to your description.
Huh, I thought I'd seen x264 stop complaining about using deprecated functions (av_free_packet, etc.) when building the lavf input. The pix_fmt IDs were separate from that.

[qyot27]

I've opened a pull request with two patches to address these points:
I guess, with the latest patch, cmake should be able to detect your pkgconfig files and automatically include these dependencies into consideration. If not, we'll see if someone will send a PR to fix it. I don't have much experience on cmake, sorry.
It's not so much that the pkgconfig files weren't detected, it was that it wasn't adding the content of Libs.private to the linker list. It was populating the right *_STATIC_* variables in CMakeCache.txt, though, so I was able to get around the problem by calling the variables directly.

The LAVF part is ported poorly. Besides, I planned to have VFR support but didn't actually finished that. So I cannot guarantee that it's working well with various of types. I'd probably extend the list a bit, but I'd like to keep it minimal until someone has actually reviewed the code.
Even though I'd simply prefer a wildcard, I went ahead and extended the list manually with the most common (or significant) extensions I could think of. The only one I forgot to include was *.webm, but considering those are just a subset of Matroska, it's not like the user can't just rename them *.mkv.

[MeteorRain]

I've opened a pull request with two patches to address these points:

Thanks, and that has been merged rebased. :thanks:

[n808]

Trying to encode a standard test sequence (BasketballDrill) using 2-pass and the latest 1.9 build ( as well as earlier builds), the 2nd pass creates a bitrate just a fraction of the requested..

My parameters are the same as I am using for x264:
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 1
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 2

Stats for the 2 passes:
1:encoded 100 frames in 2.94s (33.99 fps), 891.82 kb/s, Avg QP:36.8
2:encoded 100 frames in 3.46s (28.86 fps), 109.70 kb/s, Avg QP:51.00

There must be something basic I am missing..?

[benwaggoner]

Trying to encode a standard test sequence (BasketballDrill) using 2-pass and the latest 1.9 build ( as well as earlier builds), the 2nd pass creates a bitrate just a fraction of the requested..

My parameters are the same as I am using for x264:
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 1
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 2

Stats for the 2 passes:
1:encoded 100 frames in 2.94s (33.99 fps), 891.82 kb/s, Avg QP:36.8
2:encoded 100 frames in 3.46s (28.86 fps), 109.70 kb/s, Avg QP:51.00

There must be something basic I am missing..?
Encoding just 100 frames is just 2 seconds, way less than a GOP, and just 4x the slow's lookahead of 25 frames. How does the actual output look? You can output to bits_1p.hevc and bits_2p.hevc to compare.

I would expect the 2nd pass rate control to be a whole lot better than the 1st for such a short clip. If there's even a second of black leader, that'd throw everything off.

[n808]

It's a commonly used test sequence for standards development, including HEVC, so there's no black leader..

x264 encodes the same 100 frames with the exact same parameters like this:
1:encoded 100 frames, 237.53 fps, 885.64 kb/s
2:encoded 100 frames, 73.05 fps, 1045.64 kb/s

Something is not working right. x265 seems to clamp to QP to max 51. I may check if libx265 in ffmpeg works any better.

[LigH]

It would probably be useful to see the complete console output and maybe even a CSV log (level 1 or 2) of this second pass. There must be a reason why the target bitrate is apparently not relevant for x265 in the 2nd pass.
_

BTW:

For a more efficient command line, you may like to convert raw YUV to Y4M; this may be possible with y4mtools (https://github.com/concalma/y4mtools), MJPGTools (https://jadecolour.wordpress.com/2015/04/07/file-conversion-between-y4m-and-raw-yuv-format/) (Linux), AviSynth with RawSource26 (http://avisynth.nl/index.php/RawSource26) and avs2pipemod (http://avisynth.nl/index.php/Avs2pipemod), or ffmpeg (-f yuv4mpegpipe).

[n808]

I re-created the same results with a compressed version (x264 crf 10) available here (https://www.dropbox.com/s/6f8ssh22viycs2s/BasketballDrill_832x480_50_crf10.mkv?dl=0). if anyone else wants to try.

I converted this to Y4M using ffmpeg. Then ran the same two commands (adjusted to remove size and fps). The output in the 2nd pass is at the bottom. It's likely that "Error: 2pass curve failed to converge" may be causing the problem, but I don't know how to set parameters to avoid it. If I try higher bitrates, e.g 2000kbps, the error above is gone, but it still misses the mark completely (1795 kbps 1st pass / 789 kbps 2nd pass). Similarly, 2nd pass for for 4000 and 8000 kbps is much much worse than 1st pass, with a lower bitrate that requested.

P.S. I am using raw YUV due to some tools in my toolchain do not yet support Y4M.

====
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 frames 0 - 99 of 100
raw [info]: output file: bit
x265 [info]: HEVC encoder version 1.9+32-c2228fb8151d
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 1000.00 kbit/s, expected: 1140.97 kbit/s, avg QP: 68.7127
x265 [warning]: try increasing target bitrate
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 4 / 1 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp strong-intra-smoothing deblock sao stats-read
[1.0%] 1/100 frames, 5.75 fps, 738.00 kb/s, eta 0:00:17
[10.0%] 10/100 frames, 22.52 fps, 166.76 kb/s, eta 0:00:03
[19.0%] 19/100 frames, 26.32 fps, 137.94 kb/s, eta 0:00:03
[27.0%] 27/100 frames, 27.69 fps, 129.30 kb/s, eta 0:00:02
[35.0%] 35/100 frames, 27.91 fps, 125.79 kb/s, eta 0:00:02
[43.0%] 43/100 frames, 28.33 fps, 125.13 kb/s, eta 0:00:02
[51.0%] 51/100 frames, 28.70 fps, 124.24 kb/s, eta 0:00:01
[59.0%] 59/100 frames, 28.99 fps, 122.76 kb/s, eta 0:00:01
[67.0%] 67/100 frames, 29.13 fps, 122.07 kb/s, eta 0:00:01
[75.0%] 75/100 frames, 29.41 fps, 121.13 kb/s, eta 0:00:00
[84.0%] 84/100 frames, 29.72 fps, 118.84 kb/s, eta 0:00:00
[92.0%] 92/100 frames, 29.89 fps, 117.43 kb/s, eta 0:00:00

x265 [info]: frame I: 1, Avg QP:51.00 kb/s: 308.80
x265 [info]: frame P: 24, Avg QP:51.00 kb/s: 183.78
x265 [info]: frame B: 75, Avg QP:51.00 kb/s: 83.48
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 3.29s (30.40 fps), 109.80 kb/s, Avg QP:51.00

[LigH]

x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 1000.00 kbit/s, expected: 1140.97 kbit/s, avg QP: 68.7127
x265 [warning]: try increasing target bitrate

True, a strong hint, but directions are not yet clear to me. Apparently x265 believes that QP 51 is sufficient to reach the target bitrate, which is nonsense and results in a bitrate way below par. I will try to recreate your test, but don't have the same hardware (sometimes bugs are CPU instruction dependend, who knows...).

BTW: Is this BasketballDrill_832x480_50.yuv downladable somewhere? It's not in Derf's collection (https://media.xiph.org/video/derf/), unfortunately.
__

Trying similar parameters with a smaller file on an AMD Phenom-II X4, everything works well.

I believe a common reason for 2-pass issues was that a statistics file created in a 1st pas was not read correctly or misinterpreted... For now I have no better idea.
__

I can confirm this behaviour, using a CRF10 AVC-in-MKV source of BasketballDrill (https://www.dropbox.com/s/6f8ssh22viycs2s/BasketballDrill_832x480_50_crf10.mkv?dl=1) (uploaded a copy to my archive (https://www.mediafire.com/download/l6arqxn7kac8ar7/BasketballDrill_832x480_50_crf10.mkv)).

Furthermore, for different other sources, 2-pass VBR misses the target bitrate, the result has about 75-80%.

I will notify the x265 Developers mailing list about this case.
__

I was able to narrow it down between a few archived builds:

This issue was introduced between versions 1.8+167-e951ab673b1c and 1.8+201-769081eb5f4c.

[LigH]

Hum, funny stuff.

I received BasketballDrill_832x480_50.yuv again from staff of the Uni Hannover. It has originally 501 frames. Running them, the bitrate calculation converges.

So I have to wonder if the reason behind this issue is indeed that you encode only 100 frames with a keyframe distance of 100 frames.

Indeed: Adding "--frames 100 --keyint 100" to the command line provokes the miscalculation.

[n808]

@LigH: Thanks for the help! However, I still cannot get it to work properly, and I am little confused by a couple of your answers.

When I use all 501 frames, it somewhat improves, but 2nd pass is still much worse than 1st pass:
1: encoded 501 frames in 14.57s (34.38 fps), 1029.45 kb/s, Avg QP:35.94
2: encoded 501 frames in 31.30s (16.00 fps), 711.98 kb/s, Avg QP:38.16

When I encode the shortened 100 frames version (Y4M) with neither --frames, nor --keyint, the resulting bitrate is no better than with those parameters. So it seems 2 pass does not work on short clips at all. I tried other clips (100 frames) and the problem is the same.

>> Furthermore, for different other sources, 2-pass VBR misses the target bitrate, the result has about 75-80%.

OK. That's what I am seeing as well, for longer sequences and other bitrates. See 1st paragraph. It seems 1-pass is the only option for now.

>> Trying similar parameters with a smaller file on an AMD Phenom-II X4, everything works well.

Is this also the case for the command line parameters and clip in question (posted above)? How about the 75-80% 2nd pass issue in general?

(Yeah, these test sequences are hard to track down without the right connections...)

[LigH]

Remember, I found that this issue was introduced during a specific period of development. Now we have to give the developers some time to discover which change exactly caused it, so they can later fix it... But I hope I could help good enough to find the reason quickly now.

I believe the extreme result of being unable to converge always happens when you have only one GOP; when (what I guess may be the reason) the statistics of the first GOP are accidentally skipped, then everything is skipped because that was only one. The default keyframe interval is already more than 100 frames, so encoding only 100 frames will provoke it already without explicitly limiting the keyframe interval to only 100 frames, which could have been e.g. 250 or 300 frames without this parameter.

[n808]

Remember, I found that this issue was introduced during a specific period of development. Now we have to give the developers some time to discover which change exactly caused it, so they can later fix it...
Sure - that's expected of course. I was only trying to clarify what was still confusing to me, so thanks for explaining.

Here's what I think is the conclusion:

There is a bug where 2-pass will fail completely: encoding less than one GOP
There is a bug where 2-pass works very poorly, reaching only 75-80% of requested bitrate. I am not sure if it's possible to get 2-pass rate control to work in current releases.

[Ma]

x265-1.8+167:
i:\speed\2p>ffmpeg -i b.mkv -v warning -f yuv4mpegpipe - | x265-18+167 --y4m - -p slow --bitrate 1000 --keyint 100 -o w2.hevc --
pass 2
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: w2.hevc
x265 [info]: HEVC encoder version 1.8+167-e951ab673b1c
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect rd=4 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 signhide tmvp
x265 [info]: tools: strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I: 1, Avg QP:27.14 kb/s: 9306.40
x265 [info]: frame P: 24, Avg QP:30.98 kb/s: 2060.87
x265 [info]: frame B: 75, Avg QP:36.53 kb/s: 467.95
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 11.99s (8.34 fps), 938.64 kb/s, Avg QP:35.10


x265-1.8+168:
i:\speed\2p>ffmpeg -i b.mkv -v warning -f yuv4mpegpipe - | x265-18+168 --y4m - -p slow --bitrate 1000 --keyint 100 -o w2.hevc --
pass 2
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: w2.hevc
x265 [info]: HEVC encoder version 1.8+168-73cc1cd4c7fc
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 1000.00 kbit/s, expected: 1133.83 kbit/s, avg QP: 68.7129
x265 [warning]: try increasing target bitrate
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect rd=4 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 signhide tmvp
x265 [info]: tools: strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I: 1, Avg QP:51.00 kb/s: 358.00
x265 [info]: frame P: 24, Avg QP:51.00 kb/s: 194.60
x265 [info]: frame B: 75, Avg QP:51.00 kb/s: 88.01
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 5.98s (16.71 fps), 116.29 kb/s, Avg QP:51.00


So, the prime suspect is 73cc1cd (https://bitbucket.org/multicoreware/x265/commits/73cc1cd4c7fc0920a6b126fefebeea6269749498)

[LigH]

No surprise, Sherlock! :cool: Of course, the most important change in the rate control.

[Ma]

Finally there is a patch (https://patches.videolan.org/patch/12409/raw/) for 2-pass encoding.

x265-1.9+39+patch:i:\speed\2p>ffmpeg -i b.mkv -v warning -f yuv4mpegpipe - | x265-19+39p --y4m - -p slow --bitrate 1000 --keyint 100 -o w2.hevc --
pass 2
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: w2.hevc
x265 [info]: HEVC encoder version 1.9+39-45c0dbd43dec
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 4 / 1 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I: 1, Avg QP:27.62 kb/s: 9299.60
x265 [info]: frame P: 24, Avg QP:31.61 kb/s: 2019.28
x265 [info]: frame B: 75, Avg QP:37.20 kb/s: 474.77
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 10.88s (9.19 fps), 933.70 kb/s, Avg QP:35.76

It is working.

[LigH]

But not yet pushed ... I'll wait for a new release until it's confirmed and commited to the repository.

[n808]

Great news - Thanks!

[pingfr]

There are at least two interesting patches which yet have to be merged into the trunk as we speak:

https://patches.videolan.org/patch/12409/
https://patches.videolan.org/patch/12349/

I guess it's just a matter of time.

What I would love to see is a working patch to use avs as source and not only y4m or yuv so we don't have to rely on ffmpeg or avs4x265 anymore. ;)

[qyot27]

What I would love to see is a working patch to use avs as source and not only y4m or yuv so we don't have to rely on ffmpeg or avs4x265 anymore. ;)
See the LAVF patches in x265-yuuki (https://github.com/msg7086/x265-Yuuki-Asuna/commits/Yuuki); AviSynth is included in that now. From a philosophical standpoint, through LAVF is the cleanest way to do it as well, and you also get the benefit of not just AviSynth that way.

The only slight disadvantage is that the interleaved workaround for >8bit output used by Dither and f3kdb is probably not as easily able to fix in the LAVF input source as it is for x264's avs input (of which there is a patch to reconstitute the input video back to its proper bit depth and pretend its been at a higher bit depth all along). Kind of a minor issue compared to supporting it at all, though. In ffmpeg you can do it if you pipe rawvideo to a second instance of ffmpeg and set the input pix_fmt and resolution params correctly, but in a single instance I haven't found a solution yet (maybe something in libavfilter can do it, but that'd require x265 to also include an avfilter interface when using LAVF input).

[LazyNcoder]

Hi guys,
can anyone tell me what's the latest state of hardware accelerated x265 encoding?
Is there anyway to use OpenCL or any other kind of GPU encoding method in x265 encoding?

Thanks

[Darkangle]

I want to update the version of x265 in Staxrip to the latest version. Anyone know how to do that?

[sneaker_ger]

can anyone tell me what's the latest state of hardware accelerated x265 encoding?
Is there anyway to use OpenCL or any other kind of GPU encoding method in x265 encoding?
No, x265 is still a pure software encoder.

There are other encoders which use non-CPU hardware. See StaxRip or Hybrid GUIs, for example. (These may be faster but less effective than x265, i.e. x265 may have better quality at same filesize or lower filesize at same quality if you are willing to invest the time)

I want to update the version of x265 in Staxrip to the latest version. Anyone know how to do that?
Try replacing the integrated x265.exe with the multi-bitdepth version you can find here (http://forum.doom9.org/showpost.php?p=1757289&postcount=3262) (Win64\x265_ml.exe).

[Darkangle]

I think I didn't do it right. Could you be more specific? There are 2 folders in x265 64-Bit 8-Bit & 64-Bit 10-Bit. Each folder contains libx265.dll and x265.exe. So how can I manually update it?

[LigH]

You obviously don't use the latest release (https://github.com/stax76/staxrip/releases); version StaxRip_x64_1.3.1.7_beta (https://github.com/stax76/staxrip/releases/download/StaxRip_x64_1.3.1.7_beta/StaxRip_x64_1.3.1.7_beta.7z) uses only the multi-library EXE, not different specific bitdepth versions and separate DLLs anymore. Please clean up your installation.

In addition: Please wait with updating x265. The fix for the 2-pass VBR mode was not yet released to the official sources, only announced.

[sneaker_ger]

Either that or get yourself the single-bitdepth versions you need from e.g. https://encoder.pw.

[LigH]

I am confused by the naming. If Darkangle had different separate bitdepth versions, the DLLs would have different filenames still, they are bound to e.g. either libx265_main.dll for 8 bit or libx265_main10.dll for 10 bit precision in current versions.

[sneaker_ger]

Hmm, yeah. Should probably clean up and upgrade to the latest version like you said.

[Darkangle]

@LigH: thank you for the link.
Btw, is there anyway to make the the huge block less obvious when using ctu64 in x265? (Does it involve the settings of psy-rd, rdoq=2, psy-rdoq ?)

[LigH]

x265 1.9+54-291beccb6760 (https://www.mediafire.com/download/oze9eo3tclly9lk/x265_1.9+54-291beccb6760.7z) with fixed 2-pass VBR rate control.

Size 64 CTUs are subject to further optimizations when time permits, IIRC; an earlier attempt to interpolate them turned out to have side effects. Psy-RDO parameters may help. Using a convenient bitrate or quality target should help as well, in general.

[pingfr]

@LigH: Could you be more specific on the fore mentionned "side effects" please?

[LigH]

I refer to the analysis by Mandarinka (http://forum.doom9.org/showthread.php?p=1746309#post1746309), which caused (together with previous discussions about a loss of detail) disabling intra prediction for 64x64 CUs (http://forum.doom9.org/showthread.php?p=1746982#post1746982).

[Blowis]

Hi @Ligh download link x265 1.9+54-291beccb6760 dead: This download is currently unavailable.

[LigH]

Not my fault, I believe, I would expect a soon repair (if not, I may have to re-upload) ... but there is a copy attached in the VideoHelp forum (http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds?p=2435148&viewfull=1#post2435148).

[pradeeprama]

Yes, for '--pools +,+' and '--pools *' and no pools options it simulates '--pools 24,24' option. I think it is simple and more effective than current code and better than backout commit 10983.

I hope that you find better fix.

We ran several tests on a dual-socket E5-2670 v3 machine running windows server encoding 1080p anime clips, and some 4K animation videos. Across the board we saw that performance when we don't specify the pools option on the command line (or equivalently specify --pools +,+) gave nearly better or close performance to specifying --pools 16,16. We did, as expected, notice that when a video was run for the first time, the fps is low because you end up incurring too many page faults but when you do repeated runs and access the video from the RAM, fps stabilizes.

Ma/littlepox: I suspect that the issue is something to do with either the warm-up issue as your first run always had no --pools option, or something with your server setup.