Segmentation Fault with mencoder and x264

[evade]

I get a segmentation fault near the beginning of the second pass of anything I try to encode in x264 (less than one minute in).

The first pass completes flawlessly and plays correctly. I have been using the latest svn/cvs sources of MPlayer and x264.

I have tried -vc ffmpeg12 instead of libmpeg2 and I have also tried avs2yuv (piping works on my system now, thanks Akupenguin)

I have also tried this on both Gentoo and Debian Sarge.

My command line generally looks something like this:

mencoder
-vc ffmpeg12 \
-ofps 24000/1001 \
-mc 0 \
-noskip \
-oac copy \
-ovc x264 \
-x264encopts qp_constant=22:subq=3:pass=1 \
-o ../dvdrip/DrStrangelove.pass1.avi \
$target

mencoder \
-vc ffmpeg12 \
-ofps 24000/1001 \
-mc 0 \
-noskip \
-oac copy \
-ovc x264 \
-x264encopts bitrate=1000:pass=2 \
-o ../dvdrip/DrStrangelove.pass2.avi \
$target

Does anyone have any ideas? Let me know if I can provide any specifc output.

[akupenguin]

I need:
a gdb trace: http://ffmpeg.sourceforge.net/ffmpeg-bugreport.html. (x264 compiles with debug info by default, but you should disable mmx for this.)
the multipass stats file (divx2pass.log)
maybe the source video (just enough of it to reproduce the crash, and not until after I've seen the above).

[evade]

Do you mean compile x264 without mmx or disable it at runtime. how?


Anyway I have run the gdb trace but will gladly run it again without mmx if need be.

http://mencoder.szczuka.com/gdb-output.txt
http://mencoder.szczuka.com/divx2pass.log.bz2

[akupenguin]

It should be fixed now.

[evade]

I'm afraid it doesn't appear to be. Here is the output from my last attempt with rev 114:

http://mencoder.szczuka.com/DRSTRANGLOVE_commandline.txt
http://mencoder.szczuka.com/DRSTRANGLOVE_divx2pass.log.bz2
http://mencoder.szczuka.com/DRSTRANGLOVE_gdb_output.txt


I am also trying another source which previously crashed. I will have the results of that later.

[akupenguin]

Another try (115). If it still doesn't work, I'll need the first ~900 frames of the vob (e.g. cut with dd).

[FooFighter007]

Is there a special Makefile out there for amd64 users. I run a gentoo linux box and the currernt Makefile from SVN doesn't seem to work.

Regards,

Foo

[evade]

Still crashing.

http://mencoder.szczuka.com/DRSTRANGLOVE_115_divx2pass.log.bz2
http://mencoder.szczuka.com/DRSTRANGLOVE_115_gdb-output.txt
http://mencoder.szczuka.com/PartOf.vob

[evade]

And another crash:

http://mencoder.szczuka.com/I_Robot_commandline.txt
http://mencoder.szczuka.com/I_Robot_divx2pass.log.bz2
http://mencoder.szczuka.com/I_Robot_gdb-output.txt

[akupenguin]

Originally posted by evade
http://mencoder.szczuka.com/PartOf.vob Not big enough, contains only 344 frames, wheras the crash was on 856. Your second sample would be better, since it crashed on frame 59.

[evade]

Sorry, try this:

Source 1:
http://mencoder.szczuka.com/BiggerPartOf.vob

Source 2:
http://mencoder.szczuka.com/PartOfIRobot.m2v

[akupenguin]

I can't reproduce the crash with either 113 or 115 on either clip, but I am getting some warnings from valgrind. No promises yet.

[evade]

These encodes were done on seperate Debian Sarge boxes, both P4's.

I am running the I Robot encode on a different box. A gentoo amd64 box in box, in a 32-bit chroot.

It is on pass 2 and has has progressed beyond the the crash. It is Currently on frame 9200.

I am also running the Dr Strangelove encode on a P4 gentoo box (on which this encode has crashed before). It is still on the first pass....

I will keep you posted.

[evade]

The Dr Strangelove encode on the P4 Gentoo box crashed at the same place (frame 856). The divx2pass.logs were identical.

The successful (to this point, 70% through second pass) I Robot encode on the amd64, which used 3DNOW! in addition to MMX MMXEXT SSE and SSE2 generated a divx2pass.log file with a different md5sum than the failed encode on the P4.

here it is:

http://mencoder.szczuka.com/Working_I_Robot_3DNOW_divx2pass.log.bz2

[akupenguin]

Nope, my valgrind errors went away in 115. I'm on a AMD Barton 2500.

To disable MMX: remove "-DARCH_X86 -DHAVE_MMXEXT -DHAVE_SSE2" from the Makefile.

(BTW, while the CPU detection code knows about 3DNOW, there isn't any asm to take advantage of it.)

[evade]

I still get a segmentation fault on Dr Stranglove with MMX disabled.
Would you like any output.
should I run the first pass again with mmx diabled?

[akupenguin]

You don't need to rerun the 1st pass.

Yes, I want the same backtrace without MMX.

[evade]

here you go.

http://mencoder.szczuka.com/DRSTRANGLOVE_MMX_Disabled_gdb_output.txt


BTW

I have now tried this particular encode on 4 boxes:
2 Debian P4's
1 Gentoo P4
and 1 Gentoo amd64 (in 32-bit mode)

The encode crashes on all the P4's and works on the AMD.