PDA

View Full Version : x264 r859 VS2008-build problem


Ajaja2005
29th May 2008, 22:12
I build x264 using Visual Studio 2008 (with nasm 2.02) and I get 100% crash with latest revision (x264-snapshot-20080528-2245.tar.bz2) at the start of encoding process. All previous revisions (my last build was x264-snapshot-20080519-2245.tar.bz2) were stable.

Unhandled exception at 0x0044c012 in x264.exe: 0xC0000005: Access violation reading location 0xffffffff.

Call stack:
x264.exe!_x264_deblock_v_luma_sse2() + 0x22 bytes
x264.exe!x264_frame_deblock_row(x264_t * h=0x00a15c90, int mb_y=0) Line 697 + 0x4d bytes C
x264.exe!x264_fdec_filter_row(x264_t * h=0x00a15c90, int mb_y=1) Line 902 + 0xd bytes C
x264.exe!x264_slice_write(x264_t * h=0x00a15c90) Line 1035 + 0xd bytes C
x264.exe!_x264_stack_align() + 0x14 bytes
...


_x264_deblock_v_luma_sse2:
0044BFF0 53 push ebx
0044BFF1 56 push esi
0044BFF2 8B 44 24 0C mov eax,dword ptr [esp+0Ch]
0044BFF6 8B 4C 24 10 mov ecx,dword ptr [esp+10h]
0044BFFA 8B 54 24 14 mov edx,dword ptr [esp+14h]
0044BFFE 8B 5C 24 18 mov ebx,dword ptr [esp+18h]
0044C002 8B 74 24 1C mov esi,dword ptr [esp+1Ch]
0044C006 8D 34 49 lea esi,[ecx+ecx*2]
0044C009 4A dec edx
0044C00A F7 DE neg esi
0044C00C 4B dec ebx
0044C00D 01 C6 add esi,eax
0044C00F 83 EC 24 sub esp,24h
>>>Crash here>>>>>>>
0044C012 0F 28 04 0E movaps xmm0,xmmword ptr [esi+ecx]
0044C016 0F 28 0C 4E movaps xmm1,xmmword ptr [esi+ecx*2]
0044C01A 0F 28 10 movaps xmm2,xmmword ptr [eax]
0044C01D 0F 28 1C 08 movaps xmm3,xmmword ptr [eax+ecx]
0044C021 66 0F 6E E2 movd xmm4,edx
0044C025 66 0F 6E EB movd xmm5,ebx
0044C029 F2 0F 70 E4 00 pshuflw xmm4,xmm4,0

Help, please!

Gabriel_Bouvigne
29th May 2008, 22:15
That is an alignment problem: [esi+ecx] is not mod16 (at least that is what is happening on my side)

Dark Shikari
29th May 2008, 22:19
I build x264 using Visual Studio 2008 (with nasm 2.02) and I get 100% crash with latest revision (x264-snapshot-20080528-2245.tar.bz2) at the start of encoding process. All previous revisions (my last build was x264-snapshot-20080519-2245.tar.bz2) were stable.Somehow I doubt r852 to r858 were stable ;)

Ajaja2005
3rd June 2008, 23:40
That is an alignment problem: [esi+ecx] is not mod16 (at least that is what is happening on my side)
Yes. I think, problem with stack aligment in
;-----------------------------------------------------------------------------
; void x264_deblock_h_luma_mmxext( uint8_t *pix, int stride, int alpha, int beta, int8_t *tc0 )
;-----------------------------------------------------------------------------
INIT_MMX
cglobal x264_deblock_h_luma_%1, 0,5
mov r0, r0m
mov r3, r1m
lea r4, [r3*3]
sub r0, 4
lea r1, [r0+r4]
%assign pad 0x78-(stack_offset&15)
SUB esp, pad
...
PUSH dword r4m
PUSH dword r3m
PUSH dword r2m
PUSH dword 16
PUSH dword r0
call x264_deblock_%2_luma_%1

If I change %assign pad 0x78-(stack_offset&15) to %assign pad 0x80-(stack_offset&15), for example, then program do not crash, but encoded video is broken :(

Dark Shikari
3rd June 2008, 23:48
This is because stack alignment is required for the SSE deblocking routines. A GCC-specific method is used for aligning the stack since no standard C method exists to do so.

Patches are welcome to add an MSVC-specific method for this.

LoRd_MuldeR
4th June 2008, 00:41
Too bad that MSVC is required to make a working DLL build of x264, e.g. for use with Avidemux :(

x264 is borked again, so no update.

Dark Shikari
4th June 2008, 00:46
Too bad that MSVC is required to make a working DLL build of x264, e.g. for use with Avidemux :(Or you can just comment out the SSE2 deblock function loading and stick to MMX, at the cost of a couple % speed loss.

LoRd_MuldeR
4th June 2008, 00:50
Or you can just comment out the SSE2 deblock function loading and stick to MMX, at the cost of a couple % speed loss.

My C coding skills are too bad to mess with the x264 code myself. Can a patch be provided? :thanks:

Dark Shikari
4th June 2008, 00:54
My C coding skills are too bad to mess with the x264 code myself. Can a patch be provided? :thanks:

common/frame.c

comment out the following:

if( cpu&X264_CPU_SSE2 )
{
pf->deblock_v_luma = x264_deblock_v_luma_sse2;
pf->deblock_h_luma = x264_deblock_h_luma_sse2;
pf->deblock_v_luma_intra = x264_deblock_v_luma_intra_sse2;
pf->deblock_h_luma_intra = x264_deblock_h_luma_intra_sse2;
}

LoRd_MuldeR
4th June 2008, 00:58
Thanks :)

akupenguin
4th June 2008, 01:20
Too bad that MSVC is required to make a working DLL build of x264, e.g. for use with Avidemux :(
x264 on mingw should support building a dll. If that doesn't work, how long has it been broken, and why hasn't anyone complained? (Not that I would have fixed it, I just like to know these things.)

LoRd_MuldeR
4th June 2008, 01:25
x264 on mingw should support building a dll. If that doesn't work, how long has it been broken, and why hasn't anyone complained? (Not that I would have fixed it, I just like to know these things.)

I think Gruntster, one of the Avidemux developer and the one providing us with Win32 builds, has complained a while back.
But there was not much response, as most people seem to use the EXE file with a suitable front-end. Avidemux seems to be one of the few applications actually using the DLL...

It's good to hear that you take care of the DLL, akupenguin :)

gruntster
10th June 2008, 19:20
I'm having no luck with r877 (DLL) using MSVC6 or GCC 4.2.1.

MSVC6: crashes in x264_deblock_v_luma_sse2. Commenting out the deblocking functions as suggested above doesn't fully resolve the problem, a crash still occurs in x264_pixel_avg_weight_w8_sse2.
GCC 4.2.1 (mingw32-2): still exhibits the predict_8x8_ddl_sse2 problem as reported here (http://forum.doom9.org/showthread.php?p=1111948#post1111948) and here (http://forum.doom9.org/showthread.php?p=1114346#post1114346).