x264.exe just stops

[fabje]

Hello,

I have been using x264-r2377.exe for a very long time and it works fine. But I have a problem with the two versions after that so x264-r2389 and x264-r2409.

Just at a random time it stops encoding and I don't know why, my system specs.
i7 3930K (even at default speeds I have the problem), 16GB DDR3

My avisynth file:
LoadCPlugin("D:\AviSynth\plugins\Yadif\yadif.dll")
SetMTMode(5,3)
DirectShowSource("D:\source.ts", audio=false, convertfps=true, fps=25)
Crop(0,0, -Width % 8,-Height % 8)

SetMTMode(2)
Yadif()
Crop(2,2,-2,-2)
LanczosResize(1280,720)
Trim(69,135469)
Avisynth never uses more then 1.2GB or memory.

[wiggaz]

I have the same problem even without filtering.
The x264-64bit-2377 works fine but 2389 and now 2409 just freeze at some random point.

This is an example of script I used:
LoadPlugin("C:\Program Files (x86)\Haali\avss.dll")
dss2("D:\ENCODING\video.mkv", fps=23.976).AssumeFPS(24000,1001)


I've tried on a Xeon E5 (exacore) and on a Xeon E3 (quadcore).

Any help?
Thanks in advance.

[LoRd_MuldeR]

Try with something else than Avisynth input. If it works then, you know the problem is most likely on the Avisynth side. If it still freezes, it would indicate some problem in x264.

[fabje]

Try with something else than Avisynth input. If it works then, you know the problem is most likely on the Avisynth side. If it still freezes, it would indicate some problem in x264.
It worked for month's wouldn't it be strange then if the problem is in Avisynth? When nothing changed only the x264 version?

[MasterNobody]

As I understand you use builds from http://download.videolan.org/pub/x264/binaries/. Then I would guess the problem can be change in buildbot enviroment because according to logs r2377 was build with thread: win32
and r2389/r2409 were build with thread: posix
And I would guess it used mingw's pthread (which probably still not very stable) and not pthreads-win32. Try with this clean r2409 builds: x264_r2409.zip (http://www.sendspace.com/file/pd4bln) (mirror (http://rghost.ru/53133650))

[fabje]

As I understand you use builds from http://download.videolan.org/pub/x264/binaries/. Then I would guess the problem can be change in buildbot enviroment because according to logs r2377 was build with thread: win32
and r2389/r2409 were build with thread: posix
And I would guess it used mingw's pthread (which probably still not very stable) and not pthreads-win32. Try with this clean r2409 builds: x264_r2409.zip (http://www.sendspace.com/file/pd4bln) (mirror (http://rghost.ru/53133650))
Thanks, that could be the problem just to be clear I need to use this one, I'm using 32bit: x264-win32-pthread

[MasterNobody]

You can use any one from them. It have 32/64 bit and win32/pthreads-win32 (not mingw's pthread) builds inside archive.

[fabje]

You can use any one from them. It have 32/64 bit and win32/pthreads-win32 (not mingw's pthread) builds inside archive.
Looks like it solves the problem, only downside it is a bit slower :( Any idea?

[LoRd_MuldeR]

It worked for month's wouldn't it be strange then if the problem is in Avisynth? When nothing changed only the x264 version?

If there is a bug in your Avisynth or one of your Avisynth plug-ins, this can result in "undefined behavior". And that means you might get a crash, you might get a deadlock or you might not notice any problem at all - it's just impossible to know. In that case, switching your x264 version is like rolling the dice again. So ruling out Avisynth-related issues should be the first thing to do, if you have strange x264 problems.

And I would guess it used mingw's pthread (which probably still not very stable) and not pthreads-win32. Try with this clean r2409 builds: x264_r2409.zip (http://www.sendspace.com/file/pd4bln) (mirror (http://rghost.ru/53133650))

Yes, in my experience x264 doesn't like the winpthreads that ships with recent MinGW builds. x264 was running EXTREMELY slow. Building with "--enable-win32thread" resolved that issue for me.

[CarlEdman]

It seems the usual suspects are already on the trail, but let me add my experience in case it helps:

I've been using x264 and avisynth to reencode by DVDs/Blu-Rays for dog's years usually with good result. But over recent months (month?), I have had the encode process just hang a bunch of times. Usually killing the program and restarting fixes the problem, but I have finally run across an encode which exhibited the problem five times in a row, each time at a different frame within the first 10%. Perhaps significantly, I had previously encoded the file without problems when I accidentally left the '--tune animation' option off.

My avisynth script is:
SetMTMode(5,6)
SetMemoryMax(1024)
DGSource("FILE.dgi", deinterlace=1, use_pf = true)
ColorMatrix(hints = true, interlaced=false)
KillAudio()
autocrop(threshold=30,wMultOf=2, hMultOf=2,samples=11, mode=0)
SetMTMode(2)
super = MSuper(planar=true)
bv1 = MAnalyse(super, isb = true, delta= 1, blksize=16, overlap=8)
fv1 = MAnalyse(super, isb = false, delta = 1, blksize=16, overlap=8)
bv2 = MAnalyse(super, isb = true, delta = 2, blksize=16, overlap=8)
fv2 = MAnalyse(super, isb = false, delta = 2, blksize=16, overlap=8)
bv3 = MAnalyse(super, isb = true, delta = 3, blksize=16, overlap=8)
fv3 = MAnalyse(super, isb = false, delta = 3, blksize=16, overlap=8)
MDegrain3(super,thSAD=400,planar=true,bv1,fv1,bv2,fv2,bv3,fv3)
Distributor()
My invocation is:
avs2pipemod -y4mp tmp_kpsf9.avs | x264 --demuxer y4m --tune animation --preset veryslow --crf 19.0 --fps 24000/1001 --sar 1:1 --non-deterministic --no-fast-pskip --no-dct-decimate --profile high --level 4.0 --frames 131985 --output "FILE.out.264"
The x264 version is:
% x264 --version
x264 0.142.2389 956c8d8
(libswscale 2.1.2)
(libavformat 55.10.3)
built on Jan 22 2014, gcc: 4.8.2
configuration: --bit-depth=8 --chroma-format=all
x264 license: GPL version 2 or later
libswscale/libavformat license: LGPL version 2.1 or later
I'm happy to share the input files that help reproduce the problem, but I have to warn that the input 264 is 16 GBytes (and the input dgi almost 3 Mbytes).

[fabje]

Well my problems are solved, it never hangs anymore :)

@CarlEdman have you tried the x264 versions that MasterNobody posted here? That solved my problems.

[CarlEdman]

Well my problems are solved, it never hangs anymore :)

@CarlEdman have you tried the x264 versions that MasterNobody posted here? That solved my problems.

It is in the pipeline as soon as my conversion scripts get around to that file again. Will be sure to post results.

[CarlEdman]

Ok, I tried the x264r2409 x64 pthread version from MasterNobody's archive (or at least its mirror--the original archive was tagged as malware infected by Google Chrome). That worked like a charm.

[jpsdr]

Building with "--enable-win32thread" resolved that issue for me.

So, your advice for building a windows binary is to use this configuration option instead of default ?
The option is the same for x86 or x64 build ?

[Groucho2004]

So, your advice for building a windows binary is to use this configuration option instead of default ?
I've been using "--enable-win32thread" since it was introduced to build my binaries, never had a problem. It's even a tiny bit faster than pthreads in my testing scenarios.

The option is the same for x86 or x64 build ?
Yes.

[jpsdr]

Argh...! I've made a build with --enable-win32thread, and my first 5 encodes went fine, but 6th just stopped... Well more exaclty crashed and made pc reboot. I'm running now the exact same command line with a debug build i've made, but i'm affraid it will result the same way when i had this problem before : debug build worked fine, but normal fprofiled build not... I'll restart the encode, but if it faill again, i'll try to rebuild with the standard threads.

[LoRd_MuldeR]

Argh...! I've made a build with --enable-win32thread, and my first 5 encodes went fine, but 6th just stopped... Well more exaclty crashed and made pc reboot.

On modern operating systems, a normal user-space process, like x264, cannot make the whole system crash - neither deliberately nor by accident (bug). If the user-space process encounters a critical problem, the operating system will just terminate it. So if you encounter system crashes or sudden reboots during x264 encoding, it usually indicates a more serious problem with your machine - such as overheating, hardware faults or driver issues.

[jpsdr]

Allready gone by that when i've got previously this issue, done several stressing tests, memory check, etc... All clear. OS is XP64 SP2. And i've reboot issue only with x264. I'm waiting for debug build under dbg to finish (or crash), before re-trying again with normal build, and re-try with build with standard thread if the "re-trying" fail. Result in several hours.

[Groucho2004]

Allready gone by that when i've got previously this issue, done several stressing tests, memory check, etc... All clear. OS is XP64 SP2. And i've reboot issue only with x264. I'm waiting for debug build under dbg to finish (or crash), before re-trying again with normal build, and re-try with build with standard thread if the "re-trying" fail. Result in several hours.
A few thoughts/comments:
# x264 can stress a system more than any of the stress test tools (Prime95, Linx, whatever)
# A debug build may not stress the system as much as a release build so if the debug works it doesn't mean the issue is resolved
# Again, as already pointed out - your crash points to either a hardware or a driver issue! Disable OpenCL support to exclude driver issues.

[jpsdr]

I disable OpenCL in my build (--disable-opencl), because i don't want it, and i don't have what is needed to compile a build with OpenCL. It was Prime95 that i've used, doing 3 different tests, each one during 48h, and also memory checked with memtest and memtest+.
Now, i'm redoing the encode of my 6 files with a new build made not using win32thread, result in 3 days.
After, i'll re-test with the win32thread build i've kept.

[jpsdr]

Encode of the 6 files finished properly after 3 days of continuous encode with posx build.
I've restarted the file which crashed occured with the winthread build, an no crash this time.
At least, it's not reproducible like the last time i had the issue, so, it's not x264 related. I'll left it as "unknow" cause.

[Richard1485]

I am glad I saw this thread because I experienced the same issue after updating from r2358 to r2409. (The same problem occurs with r2389.) I have just rolled back to r2358, and I am having no trouble with exactly the same input. At least it's not my computer!

I'll try MasterNobody's builds, but may I ask if the builds here (http://download.videolan.org/pub/x264/binaries/win32/) are going to be updated to avoid this problem in future?

[MasterNobody]

I'll try MasterNobody's builds, but may I ask if the builds here (http://download.videolan.org/pub/x264/binaries/win32/) are going to be updated to avoid this problem in future?
Yes. Next builds (when there will be new commits) at videolan shouldn't have this problem (they will switch to x264's win32threads)

[Richard1485]

Thanks. I've just downloaded your archive. What is the difference between the 32-bit one with -pthread and the one without? Is -pthread simply a way of making the encoder run more quickly?

[LoRd_MuldeR]

Thanks. I've just downloaded your archive. What is the difference between the 32-bit one with -pthread and the one without? Is -pthread simply a way of making the encoder run more quickly?

PThreads means POSIX Threads (http://en.wikipedia.org/wiki/POSIX_Threads). It's the standard interface for creating and managing threads on Unix-like systems, such as Unix, BSD and Linux.

Like most cross-platform applications, x264 uses PThreads to implement multi-threading. While the Windows platform does not natively support PThreads, an implementation of PThreads for Windows exists, thanks to the "Pthreads-w32" project. So x264 can be built with the Pthreads-w32 library to enable multi-threading on the Windows platform. And initially this was the only way to enable x264 multi-threading on Windows. However, for quite some time now, x264 additionally supports using the native Win32 API for multi-threading. This way, x264 for Windows can now be built with either PThreads (Pthreads-w32) or with "native" Win32 threading. In theory it shouldn't make much of a difference, because Pthreads-w32 is more or less a wrapper that implements a POSIX compatible threading interface on top of the Win32 API. Unfortunately, there seem to be some kind of compatibility issues between x264 and the Pthreads-w32 library that ships with recent MinGW builds.

[MasterNobody]

Unfortunately, there seem to be some kind of compatibility issues between x264 and the Pthreads-w32 library that ships with recent MinGW builds.
The problems are not with Pthreads-w32 but with separate implementation of POSIX threads in mingw-w64 which is internally called winpthreads (and used instead of Pthreads-w32). So here is table for better understanding:
Pthreads-w32 v2.9.1 - OK
x264's win32threads - OK
mingw-w64's winpthreads - problems (at least with current version)

Jeff B
My builds with pthread prefix was build with Pthreads-w32 and should be also OK. I build both Pthreads-w32 and win32threads because I could and it is better for testing to have alternatives. I am not sure which of them really faster.

[Richard1485]

Thank you both for the explanations. I used -win32-pthread to encode last night, and everything went fine.

[Lyris]

I've started getting this too, but only it seems after downloading a new build of x264.
No crash, the encode just freezes after a while.

[kabelbrand]

FYI: videolans r2431 x264 builds seem to use win32 threads again:
http://buildbot.videolan.org/builders/x264-bin-win32-x86/builds/13
http://download.videolan.org/pub/videolan/x264/binaries/win32/

[filler56789]

Could someone please DEFINE what is a "recent build" of MinGW? I intend to compile x264 with GCC 4.8.1 from the middle of last year, and I hope it's NOT "recent enough" :)

[LoRd_MuldeR]

Could someone please DEFINE what is a "recent build" of MinGW? I intend to compile x264 with GCC 4.8.1 from the middle of last year, and I hope it's NOT "recent enough" :)

It's not that easy to answer, because there is not "the" MinGW. There exists at least the original MinGW project and the MinGW-w64 fork. The latter seems to be more popular these days (and it's not just for 64-Bit compilation). Furthermore, different people provide different builds of MinGW, built with different configuration options and/or different libraries (or library versions) included.

Now, problems regarding x264 and Pthreads seem to exist with the "winpthreads" library, which is now part of MinGW-w64 in order to support the C++11 threading features. There seems to be no such problem with the "pthread-w32" library, which was traditionally used with MinGW to support PThreads (before "winpthreads"). Unfortunately, I couldn't get "pthread-w32" to work with recent MinGW-Builds (which is based on the MinGW-w32 codebase).

Anyway, you can always compile x264 with "native" Win32 threading to workaround the "winpthreads" issue...

[foxyshadis]

Could someone please DEFINE what is a "recent build" of MinGW? I intend to compile x264 with GCC 4.8.1 from the middle of last year, and I hope it's NOT "recent enough" :)

Well, the w64 fork has nightly builds of 4.9.2 from last night. I'd say that's as recent a build as you'll find. (w64 is my preferred, although getting ffmpeg to build with anything is just painful, even my Fedora box.)

[filler56789]

^ Thanks for the answers ^.^;

Well, the w64 fork has nightly builds of 4.9.2 from last night. I'd say that's as recent a build as you'll find.
Furthermore, different people provide different builds of MinGW, built with different configuration options and/or different libraries (or library versions) included.
FYI: videolans r2431 x264 builds seem to use win32 threads again:

OK, so I downloaded r2431 from VideoLan, and x264 --version says it was compiled with GCC 4.8.2 O_o

In this way, I conclude that I'll keep using my "outdated" MSYS-w64 environment
(i.e., w/ GCC 4.8.1) until «I-don't know-when» ^.^;

EDIT:

Pthreads-w32 v2.9.1 - OK
x264's win32threads - OK
mingw-w64's winpthreads - problems (at least with current version)

and according to http://xhmikosr.1f0.de/tools/msys/MSYS_MinGW-w64_GCC_483_x86-x64_Full_components.txt:

GCC version 4.8.3 Stable (r11 built by nevcairiel)
gcc 4.8.3
gmp 5.1.3
mpfr 3.1.2
mpc 1.0.2
cloog 0.18.0
isl 0.11.1
binutils 2.24
mingw-w64 3.1.0
pthreads-w32 2.9.1
zlib 1.2.8
bzip2 1.0.6

Hmmm, then it should be safe to upgrade to GCC 4.8.3 ^_^

[nhakobian]

There are 3 builds of mingw-w64 on Windows that I am familiar with:

XhmikosR's builds (linked above)
Nevcairiel's builds (http://files.1f0.de/mingw/)
Komisar's builds (http://komisar.gin.by/mingw/index.html)

Nevcairiel's builds are used to build the ffmpeg used in LavFilters, so I'm fairly confident that his build has to be stable. I've personally used Nevcairiel and Komisar's builds in the past and they have both worked perfectly. I've also used the cross-compile builds found in the ubuntu repositories and they work very well too.

[jpsdr]

Is it possible that the issue solved here (https://github.com/DarkShikari/x264-devel/commit/5e58ce7a8b39ab66c7d6420b85a8e09dd08dfaaf) may have been one possible cause ?

[nevcairiel]

There are 3 builds of mingw-w64 on Windows that I am familiar with:

XhmikosR's builds (linked above)
Nevcairiel's builds (http://files.1f0.de/mingw/)

Those two are the same, I just build plain mingw-w64 gcc packages, and XhmikosR takes them and re-packages them with a full MSYS environment on top, but its my compiler build.
For building x264 or ffmpeg, its probably easier to get XhmikosR's package, since it includes everything you need, not only the compiler.

Regarding pthreads-w32 and mingw winpthreads, my mingw-w64/gcc packages always contain pthreads-w32, and never winpthreads, since winpthreads is still severly flawed in my experience.

[qyot27]

Even in the event that an environment might ship with winpthreads vs. pthreads-w32, it's not like you can't just install pthreads-w32 alongside it and then point x264 at the right place. Just for experiments' sake, I tried doing that a few days ago under my msys2 environment, and it worked fine (not that it matters in the long run, I always cross-compile everything in the FFmpeg-libass-mpv complex; msys2 is mostly only there so my Windows partition has a properly updated userland). Compile and install pthreads-w32 to its own prefix and then use --extra-cflags/ldflags when configuring x264. And if you're building them as static, make sure to pass -DPTW32_STATIC_LIB to --extra-cflags too.