Issue introduced in libx264 r1077

[gruntster]

Since r1077 (http://git.videolan.org/?p=x264.git;a=commit;h=e46f64824a718ca146724d6cfa104aaba16eb169) of libx264, Avidemux crashes when encoding if b_transform_8x8 (--8x8dct) is turned off. Unfortunately the issue doesn't occur in debug builds or when run through gdb. I have only be able to test the issue on Windows 32-bit so not sure if it occurs on other OSes or CPUs.

The only error I can capture is logged by Avidemux's error handler:


*********** EXCEPTION **************
Registers:
EAX: 000010C2 EBX: 000001AD ECX: 10000000 EDX: 00000000 ESI: 00000000
EDI: 00000002 ESP: 0021C260 EBP: 000001AD EIP: 6F7CF14D EFlags: 00010246

Exception Code: EXCEPTION_INT_DIVIDE_BY_ZERO (C0000094)
Exception Flags: 00000000

Origin:
C:\Program Files\Avidemux 2.4\libx264-66.dll(x264_macroblock_analyse+0x4F14D) [0x6F7CF14D]
*********** EXCEPTION **************


Anyone have any ideas what's causing this? Perhaps another Windows alignment issue?

[J_Darnley]

Are you only getting this with the library? If you can reproduce it with the binary, try a debug build with full optimisations: http://users.telenet.be/darnley/x264/x264-debug+opts.7z
Another bug, which was just fixed, only appeared for one guy when the opts were on.

[gruntster]

I'm only getting the problem with the library. I can't get the CLI to fail. Another user is getting the same problem on Windows. Most users probably don't turn off the 8x8dct option (which Avidemux 2.4 turns on by default) so the reports will be low.

Is there an easy way to compile x264.exe to use libx264.dll rather than compiliing everything into the exe?

[LoRd_MuldeR]

Here are fresh builds, please test:
http://forum.doom9.org/showpost.php?p=1250507&postcount=1041

I included a debug build plus a debug build with optimizations ;)

[Dark Shikari]

Well, we already know where that crash is, because there's only one IDIV in macroblock_analyse.

if( i_intra_cost == COST_MAX )
i_intra_cost = i_cost * i_satd_intra / i_satd_inter + 1;

Er, is this supposed to be

if( i_intra_cost == COST_MAX )
i_intra_cost = i_cost * i_satd_intra / (i_satd_inter + 1);

Or do I have my C precedence rules wrong again?

[LoRd_MuldeR]

So the cause of the problem was found, right? But wouldn't that effect the CLI encoder too? :confused:

[Dark Shikari]

So the cause of the problem was found, right? But wouldn't that effect the CLI encoder too? :confused:Nor would revision 1077 have anything to do with it... :confused:

[LoRd_MuldeR]

Are you going to fix it anyway?

[gruntster]

I can reproduce a crash with basically every revision from r1077 to r1114. Debug builds or running the process through gdb stops it from crashing so perhaps it may not have been introduced in r1077 but it's the first revision I can reliably reproduce the fault.

[LoRd_MuldeR]

Debug builds or running the process through gdb stops it from crashing

Did you try a debug build with optimizations yet? Normally debug builds don't have optimizations enabled.

[gruntster]

Yes I have.

[gruntster]

Er, is this supposed to be

if( i_intra_cost == COST_MAX )
i_intra_cost = i_cost * i_satd_intra / (i_satd_inter + 1);


If it helps, this seems to make the problem disappear.

[Dark Shikari]

If that makes the problem disappear, there is something more subtle wrong; i_satd_inter should never, ever, ever be zero. A quick trace of the good shows that it being zero is in fact physically impossible if the code works as intended.

More precisely:

a->i_lambda * (log2f(i+1)*2 + 0.718f + !!i) + .5f;

i_satd_inter cannot equal zero unless this line, for i >= 0 and a->i_lambda >= 1, can equal zero.

[gruntster]

After adding logging, i_satd_inter equals 0 after about 35 frames. Any debug code you'd like me to add to pinpoint it further?

[Dark Shikari]

After adding logging, i_satd_inter equals 0 after about 35 frames. Any debug code you'd like me to add to pinpoint it further?See my edit in the above post...

[LoRd_MuldeR]

So is the fix (maybe better "workaround") proposed in post #5 still valid?

I mean, is it okay to apply that change until the actual cause of the problem is resolved ???

[gruntster]

a->i_lambda * (log2f(i+1)*2 + 0.718f + !!i) + .5f;

i_satd_inter cannot equal zero unless this line, for i >= 0 and a->i_lambda >= 1, can equal zero.

If I add logging around this line the problem goes away. Since this is one of the lines introduced in r1077 it tends to suggest something is going awry in this area.

[Dark Shikari]

So is the fix (maybe better "workaround") proposed in post #5 still valid?

I mean, is it okay to apply that change until the actual cause of the problem is resolved ???Theoretically, but it's clear that this bug can only be caused if something else is failing, so that change just covers it up.

[gruntster]

Perhaps MinGW's implementation of log2f is faulty?

[LoRd_MuldeR]

Theoretically, but it's clear that this bug can only be caused if something else is failing, so that change just covers it up.

Thanks. A temporary workaround is better than nothing. Better than a potential crash for sure :)

[Dark Shikari]

Perhaps MinGW's implementation of log2f is faulty?Surely couldn't be, as it works fine here.

Why not add an assert() and see if it crashes for you?

[gruntster]

assert(a->i_lambda * (log2f(i+1)*2 + 0.718f + !!i) + .5f);

doesn't fail before assert(i_satd_inter);

[Dark Shikari]

assert(a->i_lambda * (log2f(i+1)*2 + 0.718f + !!i) + .5f);

doesn't fail before assert(i_satd_inter);So this assert i_satd_inter only fails when using the library, but not the standalone executable?

[gruntster]

Assert fails for CLI too.

x264 --crf 26 test.yuv -o test.x264 640x352


x264 [info]: 640x352 @ 25.00 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 3.0
Assertion failed: i_satd_inter, file encoder/analyse.c, line 2410

[Dark Shikari]

Assert fails for CLI too.

x264 --crf 26 test.yuv -o test.x264 640x352


x264 [info]: 640x352 @ 25.00 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 3.0
Assertion failed: i_satd_inter, file encoder/analyse.c, line 2410
Can you upload the source file for me to test on?

[gruntster]

Here you go: http://www.razorbyte.com.au/dev_dump/x264test.zip

[gruntster]

Any luck? :)

[Dark Shikari]

Your executable fails, mine does not. This suggests miscompilation.

[gruntster]

I'm using MinGW's GCC 4.2.1 (http://sourceforge.net/project/showfiles.php?group_id=2435&package_id=241304&release_id=532062).

[Dark Shikari]

I'm using MinGW's GCC 4.2.1 (http://sourceforge.net/project/showfiles.php?group_id=2435&package_id=241304&release_id=532062).If you compile with something stable instead, does the problem go away?

[LoRd_MuldeR]

It seems my DLL's compiled with MinGW GCC 4.3.3 had the same problem. At least people reported the crash...

[gruntster]

If you compile with something stable instead, does the problem go away?

How is this not stable?

[LoRd_MuldeR]

How is this not stable?

The latest "stable" MinGW release still is 3.4.5.

However we need to use GCC 4.x.x to compile the DLL for Avidemux. Otherwise the memory alignment problem occurs...

[Dark Shikari]

However we need to use GCC 4.x.x to compile the DLL for Avidemux. Otherwise the memory alignment problem occurs...Despite the fact that I fixed that months ago and you confirmed that the fix worked?

[LoRd_MuldeR]

Despite the fact that I fixed that months ago and you confirmed that the fix worked?

I confirmed that it works without the "memory alignment" patch now.

But I did not know (until now) that the "fix" was also supposed to make it work with MinGW 3.4.5 again.

So I kept on using MinGW 4.3.3 ;)

[Dark Shikari]

I confirmed that it works without the "memory alignment" patch now.

But I did not know (until now) that the "fix" was also supposed to make it work with MinGW 3.4.5 again.It should work with all compilers, not just GCC.

[gruntster]

The latest "stable" MinGW release still is 3.4.5.


I don't think that answers my question. How is MinGW's GCC 4.2.1 unstable?

[Dark Shikari]

I don't think that answers my question. How is MinGW's GCC 4.2.1 unstable?Obviously if it miscompiles x264, it's unstable, right? ;)

[gruntster]

Ah, I see! :stupid:

[LoRd_MuldeR]

MinGW 3.4.5 compiles a working DLL indeed :)

Here is a fresh set of builds:
http://forum.doom9.org/showpost.php?p=1255177&postcount=1056