r1376 deadlock

[moviefan]

Hi guys,

I'm experiencing deadlocks in the latest revision (r1376). They seem to appear randomly when encoding the same video. CPU load remains at 50% (dual core) and there's no further progress. Has anyone noticed this issue?

I used a build by myself (GCC 3.4.5 -march=i686) as well as an ICC build by techouse).

Regards
moviefan

[LoRd_MuldeR]

As always, you need to provide a sample file (unprocessed source) and the exact command-line to reproduce the problem...

[Dark Shikari]

I haven't gotten any deadlocks here; like last time (with the ultra-rare crash), in order to debug a deadlock, I'll need the following done:

1. Use a debug build of x264.
2. Run x264 in gdb.
3. When it deadlocks, force a backtrace on all threads and post it.

I'm not actually familiar enough with gdb to be entirely sure how to do this, so if someone else is more experienced I would welcome a more detailed set of instructions.

I could also try running a huge number of encodes and seeing if any of them die, but often deadlocks tend to be very system-dependent. The exact settings and the type of source would help though.

[kemuri-_9]

it should be something along the lines of

C:\x264debug>gdb --args x264 [opts]
run
*deadlock occurs*
ctrl-c
thread apply all bt
continue

use gdb from within cmd, not within sh/bash,
as gdb needs to intercept ctrl-c as a interrupt signal rather than have sh/bash intercept it and kill gdb
(I interpreted this as windows based from the gcc version)

[moviefan]

OK, that's gonna take some time as last it happened once after about 80000 frames at 1.4fps. But I'm gonna try to provide the information you are requesting. Before I do this: Could the Avisynth 2.6 build be causing the issue? So would you recommend to switch back to 2.5.8? Or do you expect x264 to be the cause of the deadlock? Or is there no statement possible?

[Dark Shikari]

OK, that's gonna take some time as last it happened once after about 80000 frames at 1.4fps. But I'm gonna try to provide the information you are requesting. Before I do this: Could the Avisynth 2.6 build be causing the issue? So would you recommend to switch back to 2.5.8? Or do you expect x264 to be the cause of the deadlock? Or is there no statement possible?It could be either. We can't know until we find where it actually deadlocked.

[aegisofrime]

I'm not sure if this information is useful, but I have probably encoded 10-20 TGMC'ed videos, since the release of r1376, each 1 to 2 hours long and have not encountered a deadlock. It's probably Avisynth rather than x264. I'm using the x264.nl build.

I personally don't see any new performance or functionality enhancements with Avisynth 2.6, so why not stick with 2.5.8 which is more mature and presumably more stable?

[moviefan]

For me it's valuable information. I have installed Avisynth 2.5.8 and I'm currently encoding with it. If the problem is version 2.6 it is more time efficient to try 2.5.8 first because if it works I will have completed one encoding process on the way of realizing that it had not been an x264 issue. I will report when it has finished.

[Manao]

No, it won't prove anything. Deadlocks are more often than not not reproducible. Can you post your avisynth script, and tell whether or not you were using any multithreading features in avisynth ? And, as Dark Shikari requested, what were the encoding settings ?

[moviefan]

I used LSFMod and GradFun2DBMod with SetMTMode(2,0). Commandline was
--bframes 3 --ref 6 --b-pyramid strict --level 4.0 --vbv-maxrate 24000 --vbv-bufsize 24000
--keyint 24 --min-keyint 2 --sar 1:1 --aud --nal-hrd --subme 10 --trellis 2 --threads auto
--thread-input --partitions all --psy-rd 1.0:0.2 --qpfile qpfile.txt --slow-firstpass

The qpfile contains the list of IDR keyframes at which chapter marks are supposed to be.

[aegisofrime]

I used LSFMod and GradFun2DBMod with SetMTMode(2,0). Commandline was
--bframes 3 --ref 6 --b-pyramid strict --level 4.0 --vbv-maxrate 24000 --vbv-bufsize 24000
--keyint 24 --min-keyint 2 --sar 1:1 --aud --nal-hrd --subme 10 --trellis 2 --threads auto
--thread-input --partitions all --psy-rd 1.0:0.2 --qpfile qpfile.txt --slow-firstpass

The qpfile contains the list of IDR keyframes at which chapter marks are supposed to be.

I LSFmod. In my experience I have encountered deadlocks when using filters like SeeSaw with Distributor(), but LSFMod has always been playing ball just nice for me. Not too familiar with GradFun2DBMod though :(

[aegisofrime]

I used LSFMod and GradFun2DBMod with SetMTMode(2,0). Commandline was
--bframes 3 --ref 6 --b-pyramid strict --level 4.0 --vbv-maxrate 24000 --vbv-bufsize 24000
--keyint 24 --min-keyint 2 --sar 1:1 --aud --nal-hrd --subme 10 --trellis 2 --threads auto
--thread-input --partitions all --psy-rd 1.0:0.2 --qpfile qpfile.txt --slow-firstpass

The qpfile contains the list of IDR keyframes at which chapter marks are supposed to be.

I don't think LSFmod is the cause of the problem. In my experience I have encountered deadlocks when using filters like SeeSaw with Distributor(), but LSFMod has always been playing ball just nice for me. Not too familiar with GradFun2DBMod though :(

[moviefan]

It really seems that it was an issue with Avisynth 2.6 and multithreading. I have successfully encoded about 180000 frames with Avisynth 2.5.8 without multithreading with techouse's ICC build of revision 1376.