x264 crashing on quad core system

[pyrates]

I got a core 2 quad cpu with 3 gigs of ram and so far with builds 658 and 656 of x264, I'm getting this crash with it:

C:\>x264.exe --pass 2 --bitrate 7550 --stats "L:\Encoding\movie\1080.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --direct-8x8 -1 --filter -2,-1 --subme 7 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --non-deterministic --cqmfile "E:\1\site\Appz\Video\h.264 stuff\eqm_avc_hr.cfg" --progress --no-psnr --output "H:\1080p.264" "L:\Encoding\movie\1080p.avs"
avis [info]: 1920x816 @ 23.98 fps (196432 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 SSSE3
x264 [warning]: VBV maxrate specified, but no bufsize.
mb type: 7 mes: 72287/196432 (36.8%), 2.04 fps, eta 16:56:30
mv: l1r0 (-132,328)
limit: 288
mb_xy: 22,27
completed: 504
Assertion failed: 0, file encoder/analyse.c, line 2714

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

The problem is it's the same error message every time, but at different points in the movie. First time was around 13%, another was around 26%, and now 36.8%. I'm trying it with build 654, but don't know why this is happening. Anyone know what this error means?

Here is the avisynth script I use:

# SetMTMode(2,0)
# Modify the path so that it finds your decomb dll file #
LoadPlugin("E:\1\site\Appz\Video\DeComb\decomb521.dll")

# Modify the path so that it finds your MPEG2DEC dll file #
LoadPlugin("E:\1\site\Appz\Video\DGMPGDec\DGDecode.dll")


LoadPlugin("E:\MKVMagic\exe\filter\Convolution3D.dll")
LoadPlugin("E:\MKVMagic\exe\filter\deen.dll")
LoadPlugin("E:\MKVMagic\exe\filter\MSharpen.dll")
LoadPlugin("E:\MKVMagic\exe\filter\tdeint.dll")
LoadPlugin("E:\MKVMagic\exe\filter\UnFilter.dll")
LoadPlugin("E:\MKVMagic\exe\filter\undot.dll")

import("E:\1\site\Appz\Video\MvBob\mvbob.avs")
import("E:\1\site\Appz\Video\AVS Scripts\convert60ito24p.avsi")
Import("E:\MKVMagic\exe\filter\HybridFuPP.avsi")

# VC-1 Source
DirectShowSource("L:\Encoding\movie\FEATURE_1_FEATURE_2.rebuilt.EVO",video=true,audio=false,fps=24000/1001,framecount=196432,seek=false,seekzero=false)

ConvertToYV12()

#Crop(left, top, -right, -bottom)
Crop(0, 132, -0, -132)

# Assume Top Field
#AssumeTFF().SeparateFields()
# Telecide(order=1)
# Assume Bottom Field
#AssumeBFF().SeparateFields()
#Telecide(order=0)

# Reduce down to 23.976 fps
# Decimate()

# Undot()
# Convolution3d(preset="movieHQ")

# HybridFuPP(preset="High")

#assumefps(23976,1000)
assumefps(24000,1001)

And yes I can play the file the directshowsource references just fine.

[akupenguin]

--non-deterministic is the trigger. As for why it's at different times, see the option name. Same reason as I why I haven't found the bug yet.

What the error message means in detail: Despite all the limitations to search range I put in place, motion estimation managed to return a motion vector pointing into a not-yet-encoded region of the reference frame. Rather than silently produce a video that decodes to something other than x264 thought it was encoding (i.e. containing artifacts), I opt to abort. And the reason for using assert rather than fixing the motion vector is that at that point x264 is in an inconsistent state and I don't know why, so anything I do would at best hide the problem.

[pyrates]

--non-deterministic is the trigger. As for why it's at different times, see the option name. Same reason as I why I haven't found the bug yet.

What the error message means in detail: Despite all the limitations to search range I put in place, motion estimation managed to return a motion vector pointing into a not-yet-encoded region of the reference frame. Rather than silently produce a video that decodes to something other than x264 thought it was encoding (i.e. containing artifacts), I opt to abort. And the reason for using assert rather than fixing the motion vector is that at that point x264 is in an inconsistent state and I don't know why, so anything I do would at best hide the problem.

So all I need to do is get rid of that option and it should be fixed? I'm also trying to set the number of threads to 4 instead of 6 that auto would set it to. See if that can fix it.

[smekoslav]

My x264.exe keeps crashing during 2nd pass of encoding (VC1 directshow source as well), always at different point of the video.
My encoder commandline:
--pass 2 --bitrate 8423 --stats "Q:\PE2.stats" --ref 3 --bframes 3 --b-pyramid --weightb --direct auto --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --merange 12 --threads auto --thread-input --cqmfile "W:\HDTV Encoding\eqm_avc_hr.cfg" --progress --no-psnr --no-ssim --output "Q:\PE2.mkv" "Q:\PE2.avs"

My avs:
directshowsource("q:\evovideo.mkv",audio=false,fps=23.976,seekzero=false,seek=true)
crop(2,142,-2,-144)
limitedsharpenfaster(strength=220)
Spline36Resize(1920,800)

[akupenguin]

smekoslav: That's not much of a bug report. See, pyrates' post contains an actual error message, and I know what that error means. Your post simply says "it doesn't work". How could anyone possibly help you?

How to Report Bugs Effectively (http://www.chiark.greenend.org.uk/~sgtatham/bugs.html) (for programs in general).
The FFmpeg bug reporting guidelines (http://ffmpeg.mplayerhq.hu/bugreports.html) (also applies to x264).
Don't bother answering until you've read those in full.

[smekoslav]

Well sorry, but that's the thing, there's no error message or other info in the megui log. During 2nd pass of VC-1 video encoding x264.exe simply crashes/closes (all i get is that winXP send report message saying that it encountered an error and needs to close) and all that megui log says for 2nd pass is this:
Log for job job6-2
avis [info]: 1920x800 @ 23.98 fps (143766 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!
x264 [warning]: VBV maxrate specified, but no bufsize.
The current job contains errors. Skipping chained jobs

This error only occurs when encoding WVC1 video that I got from EVO files on HDDVD (used haali gdsmux to convert EVO->MKV and then used that MKV as a directshowsource).
Some people said it could be the WMV decoder making the process unstable, but nothing concrete.
The first link doesn't work and I've done everything the second ones says except console output (how can I obtain one if x264.exe closes/crashes?)
Sorry again, but that's the best I can do atm.

[akupenguin]

oops, I fixed the first link.
The main point of the ffmpeg link is about gdb. If a program crashes, you build a debug version and run it in a debugger.

[pyrates]

So all I need to do is get rid of that option and it should be fixed? I'm also trying to set the number of threads to 4 instead of 6 that auto would set it to. See if that can fix it.

Well that didn't fix it. So I'm just removing that option that was suggested. Let's hope this fixes it now.

By the way, I read what --non-deterministic does. From what akupenguin described it as, I noticed that some hd rips that I downloaded, some scenes it looks like the frame is corrupted, is this what this option is supposed to prevent from happening?

[akupenguin]

No, what I described was the error message, not the option. --non-deterministic (erroneously) causes the situation that the assert checks for. The check runs regardless, and the fact that x264 threading normally doesn't crash means that normal encodes aren't corrupted in that manner.

[pyrates]

No, what I described was the error message, not the option. --non-deterministic (erroneously) causes the situation that the assert checks for. The check runs regardless, and the fact that x264 threading normally doesn't crash means that normal encodes aren't corrupted in that manner.

Ah thank you then. I'll be sure to keep that off then.