x264 DLL (r880)

[gruntster]

Producing workable x264 DLLs with GCC or MSVC no longer seems possible. Anybody else experiencing these issues?

GCC 4.2.1 (mingw32-2)
A runtime error was introduced in r742 and still exists in r880.

Backtrace of r880:
Program received signal SIGSEGV, Segmentation fault.
0x6f5acbe8 in predict_8x8_ddl_sse2 ()
from c:\dev\avidemux_2.4_build\libx264.dll
(gdb) bt
#0 0x6f5acbe8 in predict_8x8_ddl_sse2 ()
from c:\dev\avidemux_2.4_build\libx264.dll
#1 0x6f5b6dc8 in x264_slicetype_mb_cost (h=0x9ebf600, a=0x27bc58,
frames=0x27bc10, p0=0, p1=0, b=0, dist_scale_factor=128)
at encoder/slicetype.c:224
Backtrace stopped: frame did not save the PC

0x6f5acbe0 <predict_8x8_ddl_sse2+0>: mov 0x4(%esp),%eax
0x6f5acbe4 <predict_8x8_ddl_sse2+4>: mov 0x8(%esp),%ecx
0x6f5acbe8 <predict_8x8_ddl_sse2+8>: movaps 0x10(%ecx),%xmm3
0x6f5acbec <predict_8x8_ddl_sse2+12>: movups 0x11(%ecx),%xmm2
0x6f5acbf0 <predict_8x8_ddl_sse2+16>: movaps %xmm3,%xmm1
0x6f5acbf3 <predict_8x8_ddl_sse2+19>: pslldq $0x1,%xmm1
0x6f5acbf8 <predict_8x8_ddl_sse2+24>: movaps %xmm1,%xmm4
0x6f5acbfb <predict_8x8_ddl_sse2+27>: pavgb %xmm2,%xmm1
0x6f5acbff <predict_8x8_ddl_sse2+31>: pxor %xmm4,%xmm2
0x6f5acc03 <predict_8x8_ddl_sse2+35>: movaps %xmm3,%xmm0
0x6f5acc06 <predict_8x8_ddl_sse2+38>: pand 0x6f601210,%xmm2


MSVC6
I use MSVC6 to work around the GCC problem but the MSVC build is borked now too.

r878 introduced a compilation error:
C:\Dev\x264\encoder\me.c(190) : error C2143: syntax error : missing ';' before 'type'

Revisions before r878 produce a runtime error (introduced in r852). Backtrace of r877:
_x264_deblock_v_luma_sse2:
01CBFDD0 push ebx
01CBFDD1 push esi
01CBFDD2 mov eax,dword ptr [esp+0Ch]
01CBFDD6 mov ecx,dword ptr [esp+10h]
01CBFDDA mov edx,dword ptr [esp+14h]
01CBFDDE mov ebx,dword ptr [esp+18h]
01CBFDE2 mov esi,dword ptr [esp+1Ch]
01CBFDE6 lea esi,[ecx+ecx*2]
01CBFDE9 dec edx
01CBFDEA neg esi
01CBFDEC dec ebx
01CBFDED add esi,eax
01CBFDEF sub esp,24h
01CBFDF2 movaps xmm0,xmmword ptr [esi+ecx]
01CBFDF6 movaps xmm1,xmmword ptr [esi+ecx*2]
01CBFDFA movaps xmm2,xmmword ptr [eax]
01CBFDFD movaps xmm3,xmmword ptr [eax+ecx]


Commenting out the SSE2 deblocking functions as recommended here (http://forum.doom9.org/showthread.php?p=1145482#post1145482) gets a little further but crashes again here:
_x264_pixel_avg_weight_w8_sse2:
01D02600 push ebx
01D02601 push esi
01D02602 mov eax,dword ptr [esp+0Ch]
01D02606 mov ecx,dword ptr [esp+10h]
01D0260A mov edx,dword ptr [esp+14h]
01D0260E mov ebx,dword ptr [esp+18h]
01D02612 pshuflw xmm4,xmmword ptr [esp+1Ch],0
01D02619 movlhps xmm4,xmm4
01D0261C movaps xmm5,xmmword ptr [__except_list+0D0h (01db90d0)]
01D02623 psubw xmm5,xmm4

[LoRd_MuldeR]

Yes, ffdshow is experiencing this problem and apparently they solved it by completely disabling SSE2.
http://forum.doom9.org/showpost.php?p=1148058&postcount=3703

Sounds like a serious problem, because we either loose a lot of speed or we must keep on using the last version that still compiled properly :(
I really hope the devs will put some effort into this. Avidemux without x264 is kind of worthless for me...

[Dark Shikari]

It sounds like your compiler isn't obeying the alignment directives, in particular:

DECLARE_ALIGNED_16( uint8_t edge[33] );

in encoder/slicetype.c.

Going by your additional crash in pixel_avg, it seems that your compiler is just completely ignoring all stack alignment directives.

[clsid]

Unrelated to this problem, but updating GCC your will probably be useful. You can find a MinGW build of 4.2.4 here:
http://www.tdragon.net/recentgcc/

[gruntster]

It sounds like your compiler isn't obeying the alignment directives, in particular:

DECLARE_ALIGNED_16( uint8_t edge[33] );

in encoder/slicetype.c.

Going by your additional crash in pixel_avg, it seems that your compiler is just completely ignoring all stack alignment directives.

Not sure I'm understanding you correctly, are you saying that GCC and MSVC don't honour alignment directives since slicetype is thrown by GCC and pixel_avg is thrown by MSVC? If so, what compiler is recommended by the x264 team to produce a usable x264 DLL? I'd prefer to not use experimental compilers like TDM's.

[Dark Shikari]

Not sure I'm understanding you correctly, are you saying that GCC and MSVC don't honour alignment directives since slicetype is thrown by GCC and pixel_avg is thrown by MSVC? If so, what compiler is recommended by the x264 team to produce a usable x264 DLL? I'd prefer to not use experimental compilers like TDM's.GCC works absolutely fine for every single one of the following, for every version of GCC I have tested from 2.95 through 4.4:

1. Windows 32-bit executables
2. *nix 32-bit executables
3. *nix 64-bit executables
4. *nix 32-bit libraries, without PIC
5. *nix 32-bit libraries, with PIC
4. *nix 64-bit libraries, without PIC
5. *nix 64-bit libraries, with PIC

[LoRd_MuldeR]

So can x264 be compiled into the Avidemux executable, just like MEncoder does?

And why is there a difference for compiling the DLL at all ???
The resulting x264 machine code should be 100% identically for both, EXE and DLL, only linked differently. Right? :confused:

[gruntster]

Win32 DLLs worked up to r741 for GCC and r851 for MSVC6 so why not now? Are Win32 shared libs not supported since they're not on your list?

A x264 static library could be compiled into the Avidemux executable but it's undesirable especially with the plugin framework coming in 2.5.

[Dark Shikari]

Win32 DLLs worked up to r741 for GCC and r851 for MSVC6 so why not now? Are Win32 shared libs not supported since they're not on your list?No, its just that I've never tried it. I was implying since it worked everywhere else, it should work with Win32 DLLs too.

[MasterNobody]

gruntster
If you use GCC higher than 4.1.0 you may try to specify __attribute__((force_align_arg_pointer)) for x264_encoder_encode function (I think this must fix the problem with DLL-builds).

P.S. Most of SSE2 optimized functions are called inside of x264_slice_write function which is manually aligned with x264_stack_align function (they work good). But also SSE2 optimized functions called outside of x264_slice_write (for example, x264_encoder_encode->x264_slicetype_decide->x264_slicetype_analyse->x264_slicetype_frame_cost->x264_slicetype_mb_cost->predict_8x8c) and probably they cause crashes of DLLs (the problem is that calls outside of the DLL is not aligned to 16).

[gruntster]

If you use GCC higher than 4.1.0 you may try to specify __attribute__((force_align_arg_pointer)) for x264_encoder_encode function (I think this must fix the problem with DLL-builds).

(the problem is that calls outside of the DLL is not aligned to 16).

Thanks MasterNobody. This makes sense and fixes the problem for GCC. It would be nice to have a compiler-neutral fix so MSVC still works...:) I guess the stack would need to be manually aligned.

[akupenguin]

This makes sense and fixes the problem for GCC.
That's really strange, since we already have an asm implementation of force_align_arg_pointer that doesn't depend on specific gcc versions.

[gruntster]

That's really strange, since we already have an asm implementation of force_align_arg_pointer that doesn't depend on specific gcc versions.
It definitely works for GCC. Perhaps more functions than x264_slice_write need to have x264_stack_align?

[akupenguin]

But also SSE2 optimized functions called outside of x264_slice_write (for example, x264_encoder_encode->x264_slicetype_decide->x264_slicetype_analyse->x264_slicetype_frame_cost->x264_slicetype_mb_cost->predict_8x8c)
Ah, so it's not the asm functions that need their stack to be aligned, but rather asm functions operating on data that was allocated in a C function, while gcc depends on aligned stack too. Is x264_slicetype_decide sufficient, or are there others?
I guess I could put x264_stack_align around x264_encoder_encode, but that's kinda ugly: needs an extra wrapper function and extend x264_stack_align with varargs.

[gruntster]

Is x264_slicetype_decide sufficient, or are there others?I've already tried calling x264_slicetype_decide with x264_stack_align but a crash in predict_8x8_dll_sse2 still occurs.

Aligning x264_slicetype_decide with force_align_arg_pointer doesn't work either so I guess it must be something earlier in x264_encoder_encode.

[gruntster]

Ok, x264_ratecontrol_start and x264_slicetype_decide need to be aligned in x264_encoder_encode since they both call x264_slicetype_mb_cost.

[LoRd_MuldeR]

MasterNobody and gruntster, thanks for fixing the problem :)

BTW: Can you upload your working/fixed DLL to some place, so I can run a few tests?

[gruntster]

It only fixes the GCC build. The MSVC build is still borked (x264_deblock_v_luma_sse2).

I've uploaded a working GCC DLL here: http://www.razorbyte.com.au/dev_dump. It probably won't work with my last Avidemux build though because the core version of x264 has changed.

[LoRd_MuldeR]

I've uploaded a working GCC DLL here: http://www.razorbyte.com.au/dev_dump. It probably won't work with my last Avidemux build though because the core version of x264 has changed.

You are right. That DLL doesn't work with your latest Avidemux build: "Filter init failed"
Is there a specific reason why you prefer a MSVC DLL, as long as the MinGW/GCC DLL is working fine ???

[gruntster]

I guess I could put x264_stack_align around x264_encoder_encode, but that's kinda ugly: needs an extra wrapper function and extend x264_stack_align with varargs.
This is probably more efficient than aligning three functions calls (x264_ratecontrol_start, x264_slicetype_decide and x264_slice_write) on every call to x264_encoder_encode. I did a quick and dirty test using an extra wrapper (passing the args in an intptr_t array to save changing x264_stack_align) and it works well.

Is there a specific reason why you prefer a MSVC DLL, as long as the MinGW/GCC DLL is working fine ???I regard MSVC as a more solid compiler and with better debugger support (not to mention the added bonus of a decent IDE). GCC still has major deficiencies when it comes to Windows. Hopefully the Windows GCC Google SoC project (http://gcc.gnu.org/wiki/WindowsGCCImprovementsGSoC2008) will help with some of these. Basically MSVC makes things easier for me :).

[LoRd_MuldeR]

Thanks for the new build, seems to work fine :)

User data: x264
User data: core 60 r881M e098505
User data: H.264/MPEG-4 AVC codec
User data: Copyleft 2003-2008
User data: http://www.videolan.org/x264.html

x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64

:thanks:

[_xxl]

Can you post the used patch?

[LoRd_MuldeR]

And could you also make an "experimental" DLL with Psy RDO patch? :)
http://pastebin.com/m659606da

[gruntster]

Can you post the used patch?
This is the simplest:
diff --git a/encoder/encoder.c b/encoder/encoder.c
index cffaeeb..34cbe8c 100644
--- a/encoder/encoder.c
+++ b/encoder/encoder.c
@@ -1222,6 +1222,7 @@ static int x264_slices_write( x264_t *h )
* B 5 2*4
* B 6 2*5
****************************************************************************/
+__attribute__((force_align_arg_pointer))
int x264_encoder_encode( x264_t *h,
x264_nal_t **pp_nal, int *pi_nal,
x264_picture_t *pic_in,


And could you also make an "experimental" DLL with Psy RDO patch? :)

Posted here: http://avidemux.org/admForum/viewtopic.php?pid=27560#p27560

[LoRd_MuldeR]

Posted here: http://avidemux.org/admForum/viewtopic.php?pid=27560#p27560

Yes, thanks. Just finished a test encode and it does work :cool: