Current Patches, Where to get them, How they affect speed/output

[LoRd_MuldeR]

Is it possible to get any extra information from x264 crashing? I've been getting very random unhandled win32 exception crashes with techouse's unpatched x264 ICC build r1400 (QxSSSE3 used).

Use a debug build, run it inside a debugger (most likely GDB) and make a backtrace as soon as it crashes...

[kemuri-_9]

Use a debug build, run it inside a debugger (most likely GDB) and make a backtrace as soon as it crashes...

the ICL builds use the MSVC debug format which is a different debug format from that of MinGW's that can not be used within GDB.
you'll need MSVC to work with debug MSVC builds.

[Boulder]

I just had a crash encoding Shrek..I wrote the last frame number down and now I'll see whether the crash occurs at the same point. If that happens, I guess I'll have to start working on the problem. I do not know how to create a debug build (or any build to be exact) so I'll have to ask techouse then. I take that the free version of MSVC is enough?

[kemuri-_9]

I just had a crash encoding Shrek..I wrote the last frame number down and now I'll see whether the crash occurs at the same point. If that happens, I guess I'll have to start working on the problem. I do not know how to create a debug build (or any build to be exact) so I'll have to ask techouse then. I take that the free version of MSVC is enough?

not familiar enough with the various feature sets m$ likes to fling around with the free versions so i imagine it would be enough, but only using it will tell.

[masterkivat]

Sorry for being annoying, but could someone patch the newest build (r1416*) with the new AQ Experimental? I don't care being x86 or x64 :)

[julius666]

Sorry for being annoying, but could someone patch the newest build (r1416*) with the new AQ Experimental? I don't care being x86 or x64 :)

http://forum.doom9.org/showpost.php?p=1369594&postcount=153

[masterkivat]

http://forum.doom9.org/showpost.php?p=1369594&postcount=153

:thanks:
*Gonna pay more atention at the main thread of the AQ* :p

[VFR maniac]

x264_rev1416M_release1.rar (http://www.megaupload.com/?d=HI7K4F4H)

gcc: 4.4.3-dwarf2
pthreads: 2.9.0.0 (autostatic)
ffmpeg: rev21565 (--enable-pthreads)
libswscale: rev30472
ffms2: rev275
gpac: 2010-01-23 UTC
patch:
x264_hrd_pd_interlace.21_r1416.diff (http://www.esnips.com/doc/a4a9a4d4-d189-4f7c-85a2-a6fa3c27065e/x264_hrd_pd_interlace.21_r1416)
x264_tcfile_io_v5_r1416.diff (http://www.esnips.com/doc/ddbe32d3-0f20-4a00-aaa9-9294e19fd6af/x264_tcfile_io_v5_r1416)
x264_thread_pool_v2.6_r1416.diff (http://www.esnips.com/doc/1942dbf8-1de3-423e-9fcd-fae570b92e70/x264_thread_pool_v2.6_r1416)
x264_fgo_r1416.diff (http://www.esnips.com/doc/e7194de8-1303-47de-8eb1-d83905b41148/x264_fgo_r1416)
x264_AQ_experiments_v4.diff (http://komisar.gin.by/test/x264_AQ_experiments_v4.diff)
$ ./configure --extra-cflags="-I/usr/local/include -march=core2 -fno-strict-aliasing" --extra-ldflags="-L/usr/local/lib"
Platform: X86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[bob0r]

On my Intel Q6600 -march=core2 is slower than now using it.
Tested with GCC 4.4.3 32bit.

[Boulder]

not familiar enough with the various feature sets m$ likes to fling around with the free versions so i imagine it would be enough, but only using it will tell.I got a crash once again, not at the same frame though. The pop-up window told me that I could try debugging it via Visual Studio's just-in-time debugger so I'm going to download the Express Edition of MSVC++ 2008 and see if I can make it work.

[Dark Shikari]

I got a crash once again, not at the same frame though. The pop-up window told me that I could try debugging it via Visual Studio's just-in-time debugger so I'm going to download the Express Edition of MSVC++ 2008 and see if I can make it work.That's rather useless without debug symbols, and even more useless given that MSVC doesn't use the same debug symbol format as GCC...

[kemuri-_9]

That's rather useless without debug symbols, and even more useless given that MSVC doesn't use the same debug symbol format as GCC...

yes, it's a pita world and even more pita since people are all 'oh compiling with SSSE3 will make it go light-speed'
when all it does is prevent me and my AMD brethren from being able to use the build for negligible speedup. (huzzah placebo effect?)

either way here's an ICL -QxSSSE3 build of r1416 with the PDB (http://kemuri9.net/dev/x264/x264_r1416_icl_ssse3_debug.zip) so you can get debug info out of it with msvc.
skipped out on the ffms/lavf input so i wouldn't have to include ffms2.dll.
do note: i can't run this thing, so i will not be able to help you use it.

[Boulder]

yes, it's a pita world and even more pita since people are all 'oh compiling with SSSE3 will make it go light-speed'
when all it does is prevent me and my AMD brethren from being able to use the build for negligible speedup. (huzzah placebo effect?)

either way here's an ICL -QxSSSE3 build of r1416 with the PDB (http://kemuri9.net/dev/x264/x264_r1416_icl_ssse3_debug.zip) so you can get debug info out of it with msvc.
skipped out on the ffms/lavf input so i wouldn't have to include ffms2.dll.
do note: i can't run this thing, so i will not be able to help you use it.I don't know about a placebo effect, I did some testing recently and found out that the ICL builds were 2-3% faster compared to the vanilla build from x264.nl. It's not a lot but still ;)

Thank you, I'll try your build and see what happens. I'll be on a working trip next week so it might take a while.

[kemuri-_9]

I don't know about a placebo effect, I did some testing recently and found out that the ICL builds were 2-3% faster compared to the vanilla build from x264.nl. It's not a lot but still ;)

yes, the ICL builds are faster, but that's because it's ICL instead of GCC.
ICL w/ QxSSSE3 over ICL w/ SSE default is placebo effect.
it's the same situation for GCC w/ -march=core2 over GCC w/ march=i686.

[Boulder]

I wonder why practically no one uses the Qax then, isn't that supposed to create a multi-platform compatible build?

[tormento]

A 1416_x64 ICL build, please ;)

[burfadel]

Intel compiler made programmes are made specfically to run slower on AMD. It was taken to court and AMD I won recently I believe.

[kemuri-_9]

A 1416_x64 ICL build, please ;)

x264_r1416_set.zip (http://kemuri9.net/dev/x264/x264_r1416_set.zip)

Intel compiler made programmes are made specfically to run slower on AMD. It was taken to court and AMD I won recently I believe.

from my AMD laptop (single core):

>x264_gcc --preset ultrafast -o NUL "E:\random\op.mpg" --frames 1000
encoded 1000 frames, 40.30 fps, 3253.98 kb/s

>x264_icl --preset ultrafast -o NUL "E:\random\op.mpg" --frames 1000
encoded 1000 frames, 41.40 fps, 3253.98 kb/s

so even if the intel compiler still biases against AMD, it's still faster than GCC.

[aegisofrime]

Funny, I did a test myself of ICL vs GCC and GCC came out 2 or 3 FPS faster. Then again my preset was "Slower" Maybe I will try it again with "Ultrafast".

I heard about the ICL being biased against AMD, and in fact it's one of the things the FCC (or some other American government agency) is forcing Intel to clean up its act on.

[rack04]

x264_x86_r1416M (http://www.mediafire.com/?ng0wfohnzjd)

Toolchain:
gcc-core-4.4.3 (x86.core2.Komisar) (http://komisar.gin.by/mingw/index.html)
Patch:
x264_AQ_experiments_v4 (http://komisar.gin.by/test/x264_AQ_experiments_v4.diff)
Build:
./configure --extra-cflags="-march=core2"
Platform: X86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
make fprofiled VIDS="/c/x264/parkrun.1280x720.yuv"
LAVF/FFMS input:
ffmpeg rev21593 and libswscale rev30480 (http://www.mediafire.com/?hm4zf0jymmd)
ffms2 r275 (http://www.mediafire.com/?uhdmzvt3wry)

[outlaw.78]

Here is a AMD optimized 64bit version

http://www.mediafire.com/file/dmwgwotg04n/x264-0.84.1416-amdfam10-x86_64.7z

gcc 4.5.0 20100129 (experimental)
ffmpeg svn 21594
ffms2 svn 275
pthreads 2.9.0.0 shared
x264 0.84.1416 x86_64,amdfam10,fprofiled

[tormento]

x264_r1416_set.zip (http://kemuri9.net/dev/x264/x264_r1416_set.zip)
Strange enough, your version works perfectly when muxing the video to mkv but fails when tsmuxing to m2ts. The player sees the audio track only. Other x64 compiles (komisar) work ok.

[rack04]

x264_x86__core2_r1416M (http://www.mediafire.com/?rzgn3uelmzh)
x264_x86_generic_r1416M (http://www.mediafire.com/?k34zytnonjz)

Toolchain:
gcc-core-4.4.3 (x86.core2.Komisar) (http://komisar.gin.by/mingw/index.html)
Patch:
nal_hrd_patch_v1.22_fix (http://www.mediafire.com/file/imdmvtyngyz/nal-hrd1.22_fix.diff)
x264_AQ_experiments_v4 (http://komisar.gin.by/test/x264_AQ_experiments_v4.diff)
Build:
./configure --extra-cflags="-march=core2" (core2)
./configure --extra-cflags="-march=i686" (generic)
Platform: X86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
make fprofiled
LAVF/FFMS input:
ffmpeg rev21634 and libswscale rev30501 (http://www.mediafire.com/?4ut2m1zzzzz)
ffms2 r278 (http://www.mediafire.com/?m2zjzywmzmd)

[Boulder]

either way here's an ICL -QxSSSE3 build of r1416 with the PDB (http://kemuri9.net/dev/x264/x264_r1416_icl_ssse3_debug.zip) so you can get debug info out of it with msvc.
skipped out on the ffms/lavf input so i wouldn't have to include ffms2.dll.
do note: i can't run this thing, so i will not be able to help you use it.

I got this one to crash and caught the exception in MSVC2008. It shows "stack frame" as avisynth.dll, is it possible that the crash is actually due to Avisynth? How should I proceed next, I saved a minidump but is there more to do? The address at breakpoint is 00AD2BE0 and the command shown on the line is "mov ax,word ptr [ecx]". In the output it says "0xC0000005 : Access violation reading location 0x000677c0."

[rabbitiswise]

Intel compiler made programmes are made specfically to run slower on AMD. It was taken to court and AMD I won recently I believe.
Of course Intel's compiler will schedule instructions for Intel CPUs, but the specific complaint has to do with auto CPU dispatch (eg. Qax) and library functions, such as standard math functions like pow(), sin(), etc. The generated code and library functions check flags to see if it's an SSE, SSE2, etc. capable CPU, and will use specific routines depending on those flags. However, when SSE/SSE2 is determined for the flags var, it only looks for "GenuineIntel" CPUs, which means it rejects "AuthenticAMD".

But you can hack it by doing something early in startup like:

#ifdef __INTEL_COMPILER
extern "C" void __intel_cpu_indicator_init(void);
extern "C" unsigned long __intel_cpu_indicator;

{
__intel_cpu_indicator_init();
if (__intel_cpu_indicator < 0x80)
__intel_cpu_indicator = IsSSE2 ? 0x200 : (IsSSE ? 0x80 : __intel_cpu_indicator);
}
#endif

That will fool the Intel Compiler's auto CPU dispatch and replacement math libs into thinking an appropriate non-Intel/AMD CPU is SSE/SSE2 capable. I'm not sure what bit is set for SSE3, SSSE3, SSE4.1 or SSE4.2, if flags are set for them at all (I guess they are used for auto CPU dispatch, but I haven't seen math lib functions that check for them).

--
Stubie

[kemuri-_9]

I got this one to crash and caught the exception in MSVC2008. It shows "stack frame" as avisynth.dll, is it possible that the crash is actually due to Avisynth? How should I proceed next, I saved a minidump but is there more to do? The address at breakpoint is 00AD2BE0 and the command shown on the line is "mov ax,word ptr [ecx]". In the output it says "0xC0000005 : Access violation reading location 0x000677c0."

according to that, it does sound seem like avisynth is what is crashing.
you would need an avisynth with debug symbols and get the crash to occur again to get a backtrace into avisynth....
but chances are likely its due to a certain plugin in particular or a general ENOMEM case.
so debugging this issue just jumped up in difficulty.

[aegisofrime]

Of course Intel's compiler will schedule instructions for Intel CPUs, but the specific complaint has to do with auto CPU dispatch (eg. Qax) and library functions, such as standard math functions like pow(), sin(), etc. The generated code and library functions check flags to see if it's an SSE, SSE2, etc. capable CPU, and will use specific routines depending on those flags. However, when SSE/SSE2 is determined for the flags var, it only looks for "GenuineIntel" CPUs, which means it rejects "AuthenticAMD".

But you can hack it by doing something early in startup like:

#ifdef __INTEL_COMPILER
extern "C" void __intel_cpu_indicator_init(void);
extern "C" unsigned long __intel_cpu_indicator;

{
__intel_cpu_indicator_init();
if (__intel_cpu_indicator < 0x80)
__intel_cpu_indicator = IsSSE2 ? 0x200 : (IsSSE ? 0x80 : __intel_cpu_indicator);
}
#endif

That will fool the Intel Compiler's auto CPU dispatch and replacement math libs into thinking an appropriate non-Intel/AMD CPU is SSE/SSE2 capable. I'm not sure what bit is set for SSE3, SSSE3, SSE4.1 or SSE4.2, if flags are set for them at all (I guess they are used for auto CPU dispatch, but I haven't seen math lib functions that check for them).

--
Stubie

Interesting. I assume that this code should in inserted into the x264 source before compiling? It will be interesting to test a vanilla ICL version and one with this hack :)

[Boulder]

according to that, it does sound seem like avisynth is what is crashing.
you would need an avisynth with debug symbols and get the crash to occur again to get a backtrace into avisynth....
but chances are likely its due to a certain plugin in particular or a general ENOMEM case.
so debugging this issue just jumped up in difficulty.

OK, I think I need to start thinking about what I've changed recently regarding plugins and functions. Fortunately there have been quite few changes. Thanks for your help :)

[rabbitiswise]

Interesting. I assume that this code should in inserted into the x264 source before compiling? It will be interesting to test a vanilla ICL version and one with this hack :)
The code should be inserted into early startup code, eg. in main(). For a dll, somewhere in DllMain() would work. Since math libs are statically linked (usually), each module/exe/dll will have its own __intel_cpu_indicator, and so each one needs the hack applied.

If /Qax (auto CPU dispatch) is not used, then this hack will make no performance difference for compiler generated code. If library functions like sin(), cos(), pow(), log(), etc. are rarely used, then this hack will make little difference for that.

The other thing that the Intel Compiler does, is that for some values of /Qx (not all, but some), code will be generated in the same obj file as main() that also checks for appropriate CPU level support, and will abort/exit if a GenuineIntel CPU is not found. The way around that is to have main() in a source file of its own, that just calls another main type function in a different source file. The source file with main() can be compiled with no /Qx option at all, so no CPU level/capabilities are checked there at all, while all the other source files can be compiled with an appropriate /Qx option.

--
Stubie

[Fr4nz]

Hello,

I want to report a problem with x264 v1416 which affects some users when using "patched" MeGUI.

The problem is described here:
http://forum.doom9.org/showpost.php?p=1371283&postcount=757

The author of the patched MeGUI, Zathor, says that MeGUI is not the culprit, so probably we have a bug inside 1416. In fact, reverting back to v1400 solves the problem. Note also that ALL x264 1416 builds (Rack's, Jeeb's, etc.) give the problem.

[Dark Shikari]

Hello,

I want to report a problem with x264 v1416 which affects some users when using "patched" MeGUI.

The problem is described here:
http://forum.doom9.org/showpost.php?p=1371283&postcount=757

The author of the patched MeGUI, Zathor, says that MeGUI is not the culprit, so probably we have a bug inside 1416. In fact, reverting back to v1400 solves the problem. Note also that ALL x264 1416 builds (Rack's, Jeeb's, etc.) give the problem.I can't debug with no information.

I must have either:

1) Enough information to replicate the problem (exact source and settings).

2) A backtrace from GDB.

If I can't get either, I can't do anything.

[Fr4nz]

I can't debug with no information.

I must have either:

1) Enough information to replicate the problem (exact source and settings).

2) A backtrace from GDB.

If I can't get either, I can't do anything.

1) ANY source; settings are reported in the previous post;

2) What I'd need in order to do it?

[outlaw.78]

Here is a AMD optimized 64bit version

x264 x64 AMD (http://www.mediafire.com/?2kwmmorhgyn)

gcc 4.5.0 20100206 (experimental)
ffmpeg svn 21678
ffms2 svn 281
pthreads 2.9.0.0 shared
x264 0.84.1416 x86_64,amdfam10,fprofiled

[RainyDog]

Here is a AMD optimized 64bit version

x264 x64 AMD (http://www.mediafire.com/?2kwmmorhgyn)

gcc 4.5.0 20100206 (experimental)
ffmpeg svn 21678
ffms2 svn 281
pthreads 2.9.0.0 shared
x264 0.84.1416 x86_64,amdfam10,fprofiled

Thanks very much for this Outlaw, much appreciated.

[outlaw.78]

Thanks very much for this Outlaw, much appreciated.

np anytime :)

[Fr4nz]

I've an update about the problem reported here (http://forum.doom9.org/showpost.php?p=1372206&postcount=2980)some days ago: official unpatched x264 1416 builds from x264.nl seem to work flawlessly. On the contrary, Komisar NAL-HRD patched 1416 builds still freezes, as I've reported in an update here (http://forum.doom9.org/showpost.php?p=1374109&postcount=794) today, so it seems that his builds are bugged or have problems with MeGUI.

Komisar, please, if you're watching this forum could you check what is wrong with your build?

[komisar]

Fr4nz, I use NAL-HRD patch from this post (VFR maniac): http://forum.doom9.org/showthread.php?p=1366965#post1366965
But i dont use vbv-restrictions and i can not check this freezes. Do the VFR-maniac build freezes?

[Fr4nz]

Fr4nz, I use NAL-HRD patch from this post (VFR maniac): http://forum.doom9.org/showthread.php?p=1366965#post1366965
But i dont use vbv-restrictions and i can not check this freezes. Do the VFR-maniac build freezes?

Hi Komisar, maybe this can help: from avs2yuv log, this is the error that is causing the freeze:


This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.
Assertation failed!
Program: E:\dvd\avs2yuv\x264_x64.exe
File: encoder/set.c, Line 679
Expression: dpb_output_delay < pow( 2, sps->vui.nal_hrd_parameters.i_dpb_output_delay_length )
E:\work\Up.2009.1080p.avs:
1920x1080, 2500000/104271 fps, 138282 frames
Output error: wrote only 2676405 of 3110400 bytes

Fatal Error: The encoder has encountered an unexpected error!

So, it seems that something is broken in the NAL-HRD patch...

[komisar]

Fr4nz, yes... This is because HAL-HRD patch... My next build will be with another patch (http://forum.doom9.org/showthread.php?t=152127) (if by then it will not apply officially)

P.S. Does the VFR-maniac build with this error too?

[Fr4nz]

Fr4nz, yes... This is because HAL-HRD patch... My next build will be with another patch (http://forum.doom9.org/showthread.php?t=152127) (if by then it will not apply officially)

P.S. Does the VFR-maniac build with this error too?

Yes it does.

So, do you plan to wait for incoming revisions in order to release a new build? :-\

[komisar]

heh, time to port patches for the new x264 revision... :)

new revision incomming....

[jpsdr]

Hello.
Can someone please make a version with new x264 and autoVAQ4 and nal_hrd patch ?

Thanks.

[nurbs]

komisar already did. Follow the link in his signature.

[rack04]

Toolchain:
cross-mingw.gcc443.core2.20100124
gpac-cvs-20090805.core2
coreutils-5.97-2-msys-1.0.11-ext
pkg-config_0.23-3
glib_2.22.3-1
yasm-0.8.0.2289
msys-1.0.11
Patch:
x264_NAL-HRD.1.22.r1442 (http://komisar.gin.by/x.patch/last.used/x264_NAL-HRD.1.22.r1442.diff)
x264_AQ_experiments_v4 (http://komisar.gin.by/test/x264_AQ_experiments_v4.diff)
x86 Build:

x264_x86_r1442M (http://www.multiupload.com/FLJ09RG0TD)

./configure --extra-cflags="-march=core2"
Platform: X86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
make fprofiled VIDS=parkrun.1280x720.yuv
LAVF/FFMS:
ffmpeg r21847 and libswscale r30592 (http://www.multiupload.com/FUTBXNU986)
ffms2 r290 (http://www.multiupload.com/2HJ04UIN40)

[outlaw.78]

x264 64bit (http://www.mediafire.com/?znmmnkiohic)

gcc 4.5.0 20100211 (experimental)
ffmpeg svn 21850
ffms2 svn 290
pthreads 2.9.0.0 shared
x264 0.85.1442 x86_64,amdfam10,fprofiled

[RainyDog]

x264 64bit (http://www.mediafire.com/?znmmnkiohic)

gcc 4.5.0 20100211 (experimental)
ffmpeg svn 21850
ffms2 svn 290
pthreads 2.9.0.0 shared
x264 0.85.1442 x86_64,amdfam10,fprofiled

Cheers again Outlaw :) These builds are perfect for my Phenom II.

[aegisofrime]

x264 64bit (http://www.mediafire.com/?znmmnkiohic)

gcc 4.5.0 20100211 (experimental)
ffmpeg svn 21850
ffms2 svn 290
pthreads 2.9.0.0 shared
x264 0.85.1442 x86_64,amdfam10,fprofiled

Thanks again for the AMD builds. I hope I'm not asking too much, could you do a AQ patched version as well? :thanks:

[outlaw.78]

http://komisar.gin.by/test/x264_AQ_experiments_v4.diff

this patch by komisar you want?

[aegisofrime]

http://komisar.gin.by/test/x264_AQ_experiments_v4.diff

this patch by komisar you want?

Yes that's the one. If it's not too much trouble for you :D

[outlaw.78]

hmmmm i am getting an error trying to apply this patch to latest source code, using git it says patch does not apply...

any help?