x264 cli bugreports

[Asmodian]

I assumed clamping the bitrate was more surprising than the VBV settings, after all users understand average bitrate better than VBV, but I can see it either way.

[Dark Shikari]

VBV is a compatibility setting -- we generally try to avoid changing those (more than anything else), since an unexpected change to a compatibility setting could be the difference between a video that does play and one that doesn't.

[Dark Shikari]

That would require changing how b_level_idc works; currently it doesn't enforce any restrictions in libx264. Patches welcome if you want to create an API function called param_apply_level or similar.

[Stereodude]

I have a BD source (1920x1080 p24000/1001 with 176281 frames) that will crash every version of x264 (8-bit x64) after v2273 on my i7-4770k (Haswell) in Windows 7 x64 SP1 within ~200 frames of the end on the 1st pass. DGDecNV doesn't indicate any problem with the original h264 file. I've extracted it twice to ensure no problems with the source file and they were bit identical. I can play the .avs file in VirtualDub through the "problem" area just fine. I've tried x64 versions 2334, 2310, 2309, and 2273 accessing the .avs directly (64-bit AVIsynth 2.5.8). I tried piping from 32-bit AVIsynth 2.6.0a4 into 2334 x64 using avs2pipemod and it crashes at the same spot. I lastly tried 2334 x86 fed from 32-bit AVIsynth 2.6.0a4 which also crashed.

I've encoded shorter movies without issue on this system with the same flow. I underclocked the system below stock frequencies and it made no difference. I tried encoding just the last 5000 frames of the movie and that works fine. :confused:

Here's my command line:
x264 --bitrate 21710 --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 --qpfile movie.chp -o NUL movie.avs

my AVS file:
LoadPlugin("DGDecodeNV.dll")
DGSource("Video_2.dgi")
ConvertToYV12().AssumeFPS(24000,1001)

example of the last output from x264:
E:\TempBD\movie>x264.exe --bitrate 21710 --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 --qpfile movie.chp -o NUL movie.avs
avs [info]: 1920x1080p 1:1 @ 24000/1001 fps (cfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x264 [info]: profile Main, level 4.1
[99.9%] 176110/176281 frames, 57.88 fps, 21080.47 kb/s, eta 0:00:02

Then I get a popup in Windows that x264 has crashed.
http://img90.imageshack.us/img90/5453/5sb.png

Details from v2334 x86 / 32-bit AVS crash:
Problem signature:
Problem Event Name: APPCRASH
Application Name: x264x86.exe
Application Version: 0.133.2334.0
Application Timestamp: 519b2e49
Fault Module Name: x264x86.exe
Fault Module Version: 0.133.2334.0
Fault Module Timestamp: 519b2e49
Exception Code: c0000005
Exception Offset: 00046d4b
OS Version: 6.1.7601.2.1.0.768.3
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789


Summary:
v2273 x64 / 64-bit AVS - works
v2309 x64 / 64-bit AVS - crashes
v2310 x64 / 64-bit AVS - crashes
v2334 x64 / 64-bit AVS - crashes
v2334 x64 / piped from 32-bit AVS - crashes
v2334 x86 / 32-bit AVS - crashes

Any ideas?

[Dark Shikari]

Do you have a backtrace, or at least which function it crashed in?

[Stereodude]

Do you have a backtrace, or at least which function it crashed in?Not sure, how/where do I generate or get that data?

[LoRd_MuldeR]

Not sure, how/where do I generate or get that data?

Use a debugger! Usually you'd use GDB for binaries compiled with MinGW/GCC. It's contained in the MinGW package:
http://sourceforge.net/projects/mingwbuilds/files/host-windows/releases/4.8.1/32-bit/threads-posix/sjlj/x32-4.8.1-release-posix-sjlj-rev1.7z/download

gdb.exe --args x264.exe <your options here>

Then type "run" and wait until the crash occurs. Once the break was triggered, type "bt" in order to print the callstack....

(Note, however, that you might need a debug build of x264 in order to obtain a meaningful callstack)

[Stereodude]

Use a debugger! Usually you'd use GDB for binaries compiled with MinGW/GCC. It's contained in the MinGW package:
http://sourceforge.net/projects/mingwbuilds/files/host-windows/releases/4.8.1/32-bit/threads-posix/sjlj/x32-4.8.1-release-posix-sjlj-rev1.7z/download

gdb.exe --args x264.exe <your options here>

Then type "run" and wait until the crash occurs. Once the break was triggered, type "bt" in order to print the callstack....

(Note, however, that you might need a debug build of x264 in order to obtain a meaningful callstack)
Okay thanks. I'm not a software programmer by trade, but I work on embedded systems debugging other people's C code peeking into registers and how the code interacts with the hardware to figure out why it doesn't do what they want, so I'm used to hardware debuggers that connect to a microcontroller via a JTAG/Nexus interface, not debugging software on a Windows platform.

I wasn't sure if there was some x264 command line option that would generate the data Dark Shikari was after or not. I will give your instructions a shot and see what I get.

It also seems I have a variety of .dmp files from x264 from the crashes. Are those at all useful?

Edit: No dice... (unless I took your example too literally)

E:\TempBD>C:\Temp\mingw32\bin\gdb.exe --args x264.exe --bitrate 21710 --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 --qpfile movie.chp -o NUL movie.avs
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-w64-mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from E:\TempBD\x264.exe...(no debugging symbols found)...done.
(gdb) run
Starting program: E:\TempBD\x264.exe --bitrate 21710 --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim bt709 --transfer bt709 --colormatrix bt709 --sar 1:1 --pass 1 --qpfile movie.chp -o NUL movie.avs
Error creating process E:\TempBD\x264.exe, (error 50).
(gdb)

Googling for GDB error 50 didn't turn up anything useful (at least to me).

Edit2: I figured it out. Your instructions are for debugging the x86 x264 binary, not the x64 one I originally tried. It's running now.

[Stereodude]

Well, this doesn't seem very useful, but maybe with a .map file it makes more sense?

E:\TempBD>C:\Temp\mingw32\bin\gdb.exe --args x264x86.exe --bitrate 21710 --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 --qpfile movie.chp -o NUL moviex86.avs
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-w64-mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from E:\TempBD\x264x86.exe...(no debugging symbols found)...done
.
(gdb) run
Starting program: E:\TempBD\x264x86.exe --bitrate 21710 --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim bt709 --transfer bt709 --colormatrix bt709 --sar 1:1 --pass 1 --qpfile movie.chp -o NUL moviex86.avs
[New Thread 5100912.0x4dd648]
[New Thread 5100912.0x4dcde4]
avs [info]: 1920x1080p 1:1 @ 24000/1001 fps (cfr)
[New Thread 5100912.0x4dd3a8]
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
[New Thread 5100912.0x4dd6a4]
[New Thread 5100912.0x4dca08]
x264 [info]: profile Main, level 4.1
[New Thread 5100912.0x4dd728]
[New Thread 5100912.0x4dd7e0]
[New Thread 5100912.0x4dcf64]
[New Thread 5100912.0x4dd614]
[New Thread 5100912.0x4dd5e8]
[New Thread 5100912.0x4d52c8]
[New Thread 5100912.0x4dd484]
[New Thread 5100912.0x4dd6c4]
[New Thread 5100912.0x4dd68c]
[New Thread 5100912.0x4dd558]
[New Thread 5100912.0x4dd6a0]
[New Thread 5100912.0x4dd688]
[New Thread 5100912.0x4dd7a4]
[New Thread 5100912.0x4dd52c]
[New Thread 5100912.0x4dd798]
[New Thread 5100912.0x4dd6e0]
[New Thread 5100912.0x4dd6d4]
[New Thread 5100912.0x4dd408]
[New Thread 5100912.0x4dd48c]
[99.9%] 176090/176281 frames, 57.32 fps, 21083.31 kb/s, eta 0:00:03
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 5100912.0x4dd7e0]
0x00446d4b in ?? ()
(gdb) bt
#0 0x00446d4b in ?? ()
#1 0x0000000c in ?? ()
#2 0x00000006 in ?? ()
#3 0x00000000 in ?? ()
(gdb)

So... Where can I get a debug build?

[Dark Shikari]

I see your machine supports AVX2; could you update to the latest build (r2345 or so?) to make sure it wasn't that denoise_dct alignment issue I just fixed?

Beyond that, a debug build would definitely narrow it down; JEEB or someone else can probably provide one.

[Stereodude]

I see your machine supports AVX2; could you update to the latest build (r2345 or so?) to make sure it wasn't that denoise_dct alignment issue I just fixed?Ok, I tried r2345 x64 (clean, from Komisar) and it works fine / no crash. I didn't realize that the r2334 posted on x264.nl wasn't the latest version. :thanks:

[LoRd_MuldeR]

I figured it out. Your instructions are for debugging the x86 x264 binary, not the x64 one I originally tried. It's running now.

I was pointing to 32-Bit MinGW (containing a 32-Bit GDB), but for debugging a 64-Bit binary you probably have to grab the GDB from the 64-Bit MinGW package.

Well, this doesn't seem very useful, but maybe with a .map file it makes more sense?
So... Where can I get a debug build?

Building x264 from the sources isn't very difficult, as long as you don't want all the extra libraries. Especially if software development isn't completely new for you.

It seems you already solved your problem this time, but next time you may need a debug build, just grab MinGW (http://sourceforge.net/projects/mingwbuilds/files/host-windows/releases/) + MSYS (http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/) and then in the MSYS terminal do the following:

git clone git://git.videolan.org/x264.git x264-src
cd x264-src
./configure --enable-win32thread --enable-debug
make

Note that the "--enable-win32thread" is needed with the latest MinGW builds, as the PThreads it includes performs very poor with x264 for whatever reason, like ~0.5 fps.

(Your mileage may vary)

I didn't realize that the r2334 posted on x264.nl wasn't the latest version. :thanks:

In the absence of Sharktooth, I have updated the sticky (http://forum.doom9.org/showthread.php?p=610495#post610495) a bit, hopefully this helps to avoid confusion...

[Groucho2004]

Is that intended?
I'm sure that whoever built the binaries disabled GPAC (and FFMS) on purpose. Also, this is not really a bug report.

[Chainmax]

I'm currently trying to use JEEB's 32bit r2245 x264 build to encode a fairly resource-intensive script. For the third time, the encode seems to stop at about 66%. The process does not get hung and it does not crash, but the progress indicator just stops increasing. What could be the cause of this? I am using AVISynth v 2.60a4 and SEt's v2.6MT.

[turbojet]

I'm currently trying to use JEEB's 32bit r2245 x264 build to encode a fairly resource-intensive script. For the third time, the encode seems to stop at about 66%. The process does not get hung and it does not crash, but the progress indicator just stops increasing. What could be the cause of this? I am using AVISynth v 2.60a4 and SEt's v2.6MT.

Are you using SetMTMode?

[Chainmax]

Yes. Should I try eliminating all related lines, or just the Distributor() one?

[Groucho2004]

Yes. Should I try eliminating all related lines, or just the Distributor() one?
- Post your script
- Don't use Distibutor() in the script
- How much memory does Avisynth use with your script?
- Post your x264 command line

[Chainmax]

Yeah, my bad. Here's the script:

SetMemoryMax(900)

SetMTMode(5,4)

Plugin loading

MPEG2Source("x:\wherever\somevideo.d2v")

SetMTMode(2)

TComb(fthreshL=6,fthreshC=7,othreshL=7,othreshC=8)

source=last
out1=source.Checkmate()
out2=source.checkmate(tthr2=0)
out2.mergechroma(out1)

FFT3DFilter(sigma=10,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16,interlaced=true)

Interp = nnedi3()
Deinted=TDeint(type=1,edeint=Interp,emask=TMM())
TFM(d2v="x:\wherever\somevideo.d2v",mode=6,PP=7,slow=2,mChroma=true,Clip2=Deinted)
vinverse()
TDecimate(mode=1)

RemoveGrain(mode=5)

p=last
a=p.Trim(0,32578)
b=p.Trim(32579,32654).ConvertToYUY2().SpatialSoften(11,0,700).SpatialSoften(11,0,700).ConvertToYV12().ChubbyRain2().FFT3DFilter(sigma=16,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
c=p.Trim(32655,0)
a+b+c

q=last
d=q.Trim(0,34603).Crop(6,2,704,478,align=true)
e=q.Trim(34604,0).ConvertToRGB().Crop(7,2,703,478,align=true).Spline36Resize(704,478).ConvertToYV12()
d+e

source=last
preNR=source.dfttest(sigma=1.25,tbsize=3).FFT3DFilter(sigma=10,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)
preNR_super=preNR.MSuper(pel=2, sharp=1)
source_super=source.MSuper(pel=2, sharp=1)
backward_vec3 = MAnalyse(preNR_super,isb = true, delta = 3, overlap=4)
backward_vec2 = MAnalyse(preNR_super,isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(preNR_super,isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(preNR_super,isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(preNR_super,isb = false, delta = 2, overlap=4)
forward_vec3 = MAnalyse(preNR_super,isb = false, delta = 3, overlap=4)
source.MDegrain3(source_super,backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=350)

RemoveGrain(mode=5)

Spline36Resize(704,480)
aWarpSharp2(depth=16,chroma=4)

dull=last
sharp=last.LimitedSharpenFaster(SMode=4,Strength=150)
Soothe(sharp,dull,25)

Tweak(sat=1.1)

GradFunkMirror(1.51)

Task Manager reports around 1.5GB being used by x264. The command line is not too involved, but it is overkill:

x264-10bit --preset placebo --tune animation --bframes 16 --crf 18 --keyint 240 --no-dct-decimate
--no-fast-pskip --output d:\result.mp4 d:\wherever\sauce.avs

About the Distributor() line, what little I read about it was that it was not necessary to include on the script, and removing it might solve issues every now and then. What is its utility then? Are there an cases where it's worth including it n the script?

[Groucho2004]

Woah. With that script, Avisynth MT and the placebo preset it's no surprise that it locks up.
The script alone with Avisynth MT probably needs already 2 G which leaves nothing for the encoder.

I'd start by tuning the script so it runs with reasonable speed and not too much memory. Use MDegrain2 instead of 3. Play with SetMemoryMax().

Use AVSMeter to monitor the memory usage and speed.

The Distributor() call is only necessary for some piping tools and ffdshow. x264 invokes the call itself.

Edit:
Your problem is most likely related to Avisynth(MT) so maybe this should moved to the appropriate thread.

[cretindesalpes]

Indeed your script is very heavy (cascade of temporal filters). Does the 66 % correspond to a Trim in your script (frame 32579, 34604...)? You probably have a new filter chain suddenly kicking in and generally the AVS-MT cache doesn't like that, outputing a frame every minute or so if you’re lucky.

[MrVideo]

Win7-64
x264 x86 Win32 versions: 2851, 2893, 2901
Downloaded from: x264.nl

Problem: when using 2893, or 2901, with AVISynth, x264 complains that it "failed to load avisynth." 2851 works without any issues.

When running the --version option, I notice that 2851 has the --bit-depth=8 configuration option, while 2893 and 2901 do not. Is that the reason why neither version can find avisynth, even though it is there?

Can the next x264 Win32 binary release get fixed so that it will again work with AVISynth?

Update: I've discovered another location for a 2901 download: LigH's x264 (https://www.mediafire.com/?bxvu1vvld31k1)
It too doesn't contain the --bit-depth=8 configuration option, but it works.

Any particular reason why the x264.nl versions fail and the LigH's version works?

[MasterNobody]

Do official builds (http://download.videolan.org/pub/x264/binaries/) from videolan works? For me both win32 and win64 builds work without problems with AviSynth demuxer. Also check which version of AviSynth you have (original or AviSynth+ and which revision) and that you use have both 32-bit and 64-bit AviSynth installed i.e. you not trying to run win64 x264 build with win32 AviSynth or vice versa. Try to run both win32 and win64 builds.

[MrVideo]

My mistake, in that I said the builds were from x264.nl. Those three were from the videolan repository. So no, they did not work.

AVISynth is 32bit 2.6 version. The version also required for BDRB. I can't test the Win64 build at the moment, as I have a compile currently running.

Update: The videolan 2901 x64 version had the same result.

[MasterNobody]

I can't replicate it here with:
Windows 7 64-bit
x264-r2901-7d0ff22.exe CRC32: A262BDF6 MD5: CC70693AD3E613ECBEEE178B578BCDFE
AviSynth_260.exe CRC32: D510D65E MD5: 7299618CCDFA57A2302381FD104234D6

Make sure that you run 32-bit x264 which have size of 20803584 bytes and not 64-bit x264 with same name but size of 22505472 bytes. Also I would recommend switch to AviSynth+ (as only developed/maintained fork) instead of AviSynth 2.6

[MrVideo]

Thanks. Making me recheck the files determined that all of the name changing (the long x264 binary name to x264.exe) and then renaming the x264.exe to x264-2901-x86.exe and x264-2901-x64.exe resulted in the x264-2901-x86.exe file actually being the x64 version. I do not know how I managed to screw this up when transferring the files from the storage area on the NAS. Unfortunately, running the programs with --help or --version results in no information about the binary being 32bit or 64bit. It didn't even dawn on me that the two files were the same size. I've since run "file" on each one (I have Cygwin installed) and see the result. That is now fixed.

As for the 2893 version, I somehow managed to download the x64 version and not the x86 version.

As for running AviSynth+, I can't do that. AFAIK, BDRB will not work with the + release. Jdobbs' thread only recommends installing the 2.60 version.