x264 Windows x64 support

[MasterNobody]

Can somebody with Core2 (I am interested in SSSE3 and SSE4) to test this package of x264 for Windows x64: http://stashbox.org/328100/x264win64_mingw.rar.
Needed to check:
1) Does it starts at all (It is first time I compile in MinGW for x64)
2) Doesn't checkasm crash and goes through all tests
3) Does x264 correctly work

[LoRd_MuldeR]

1) Does it starts at all (It is first time I compile in MinGW for x64)

Yes.

2) Doesn't checkasm crash and goes through all tests

Yup. All tests are "[OK]" :)

3) Does x264 correctly work

Encodes. First test looks good :cool:

But no Avisynth input due to lack of Avisynth x64, so I tested a raw YUV sample.

C:\Downloads\x264win64_mingw>x264.exe --crf 22 --output c:\wurst.mkv C:\Downloads\foreman_qcif.176x144.yuv
x264 [info]: file name gives 176x144
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 1.1
x264 [info]: slice I:2 Avg QP:22.99 size: 5381 PSNR Mean Y:39.55 U:43.83 V:45.32 Avg:40.66 Global:40.35
x264 [info]: slice P:298 Avg QP:26.81 size: 647 PSNR Mean Y:36.15 U:41.04 V:41.95 Avg:37.30 Global:37.21
x264 [info]: mb I I16..4: 0.5% 0.0% 99.5%
x264 [info]: mb P I16..4: 0.1% 0.0% 0.9% P16..4: 39.8% 25.0% 16.8% 0.0% 0.0% skip:17.4%
x264 [info]: SSIM Mean Y:0.9636046
x264 [info]: PSNR Mean Y:36.170 U:41.057 V:41.972 Avg:37.326 Global:37.228 kb/s: 135.74

encoded 300 frames, 270.51 fps, 136.11 kb/s

Good work :thanks:

[Dark Shikari]

Can somebody with Core2 (I am interested in SSSE3 and SSE4) to test this package of x264 for Windows x64: http://stashbox.org/328100/x264win64_mingw.rar.
Needed to check:
1) Does it starts at all (It is first time I compile in MinGW for x64)
2) Doesn't checkasm crash and goes through all tests
3) Does x264 correctly work:thanks:

[Snowknight26]

checkasm crashes after outputting this:

x264: using random seed 46208040
x264: MMX
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- pixel var : [OK]
- pixel hadamard_ac : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- sub_dct4 : [OK]
- add_idct4 : [OK]
- dct4x4dc : [OK]
- idct4x4dc : [OK]
- zigzag_frame : [OK]
- zigzag_field : [OK]
- mc luma : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
- deblock : [OK]
- quant : [OK]
- coeff_last : [OK]
- cabac : [OK]
x264: MMX Cache64
- pixel sad : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
x264: SSE2Slow
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]
- pixel sa8d : [OK]
- pixel var : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- ssim : [OK]
- sub_dct4 : [OK]
- sub_dct8 : [OK]
- add_idct4 : [OK]
- add_idct8 : [OK]
- hpel filter : [OK]
- intra pred : [OK]
- deblock : [OK]
- quant : [OK]
- dequant : [OK]
- denoise dct : [OK]
- decimate_score : [OK]
- coeff_last : [OK]
x264: SSE2Fast
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel satd : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- pixel var : [OK]
- pixel hadamard_ac : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- zigzag_frame : [OK]
- mc luma : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
x264: SSE2Fast Cache64
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- mc luma : [OK]
x264: SSE3
- pixel sad : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
x264: SSSE3
- pixel satd : [OK]
- pixel sa8d : [OK]
- pixel hadamard_ac : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- zigzag_frame : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
- quant : [OK]
- denoise dct : [OK]
- decimate_score : [OK]
- coeff_last : [OK]
x264: SSSE3 Cache64
- pixel sad : [OK]
- pixel sad_aligned : [OK]

x264 crashes after it outputting this:
x264.exe --crf 22 --output test.mkv test.yuv 1916x818
x264 [warning]: width or height not divisible by 16 (1916x818), compression will
suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 4.0

[LoRd_MuldeR]

checkasm crashes after outputting this:

What is your CPU? :confused:

[Snowknight26]

Q6600.

[LoRd_MuldeR]

Q6600.

It doesn't crash on my Q6600. I ran it several times. Mine is a "Kentsfield", your's too?

[Snowknight26]

Yes, it is. (http://valid.canardpc.com/show_oc.php?id=463136)

[LoRd_MuldeR]

Yes, it is. (http://valid.canardpc.com/show_oc.php?id=463136)

But it's a G0 stepping, while mine is a B3 stepping. Also are you on Vista x64 or Windows XP x64?

[Snowknight26]

Vista x64.

[LoRd_MuldeR]

Vista x64.

So it's either a G0-specific problem or a Vista-specific problem ;)

[ghostonline]

B3 is more buggy than G0. it is either due to the OS or the overclock...

[Audionut]

Checkasm runs fine here on a G0 stepping Q6600 on Vista 64.

Will try x264 in an hour or 2.

[squid_80]

Link seems to be down?

[Audionut]

Mirror (http://rapidshare.com/files/174112242/x264win64_mingw.rar)

[squid_80]

Ok, checkasm runs fine but the CLI won't encode from avs or uncompressed YV12 avi input, it just exits at frame 1.

Edit: It's actually crashing and being silently terminated because there's no function unwind info. Here's where it crashes:
image00000000_00400000+0xbeb54:
00000000`004beb54 660f7f7c2418 movdqa oword ptr [rsp+0x18],xmm7 ss:00000000`0022bb18=000000000230867100000000004c5038
If you kept the map file you should be able to find which function is responsible.

[Audionut]

Checkasm runs fine.

E:\x264>checkasm
x264: using random seed 3034587448
x264: MMX
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- pixel var : [OK]
- pixel hadamard_ac : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- sub_dct4 : [OK]
- add_idct4 : [OK]
- dct4x4dc : [OK]
- idct4x4dc : [OK]
- zigzag_frame : [OK]
- zigzag_field : [OK]
- mc luma : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
- deblock : [OK]
- quant : [OK]
- coeff_last : [OK]
- cabac : [OK]
x264: MMX Cache64
- pixel sad : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
x264: SSE2Slow
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]
- pixel sa8d : [OK]
- pixel var : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- ssim : [OK]
- sub_dct4 : [OK]
- sub_dct8 : [OK]
- add_idct4 : [OK]
- add_idct8 : [OK]
- hpel filter : [OK]
- intra pred : [OK]
- deblock : [OK]
- quant : [OK]
- dequant : [OK]
- denoise dct : [OK]
- decimate_score : [OK]
- coeff_last : [OK]
x264: SSE2Fast
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel satd : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- pixel var : [OK]
- pixel hadamard_ac : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- zigzag_frame : [OK]
- mc luma : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
x264: SSE2Fast Cache64
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- mc luma : [OK]
x264: SSE3
- pixel sad : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
x264: SSSE3
- pixel satd : [OK]
- pixel sa8d : [OK]
- pixel hadamard_ac : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- zigzag_frame : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
- quant : [OK]
- denoise dct : [OK]
- decimate_score : [OK]
- coeff_last : [OK]
x264: SSSE3 Cache64
- pixel sad : [OK]
- pixel sad_aligned : [OK]


X264 does not.


E:\x264>x264 --crf 22 -o e:\test.264 e:\test.avs
avis [info]: 1280x528 @ 23.98 fps (3429 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 3.1

E:\x264>

[Snowknight26]

Audionut, I don't think checkasm completed successfully because thats exactly where mine stops. It outputs an ASCII smile when it finishes, so obviously ours didn't.

[Audionut]

Oh. Indeed. Thanks.

[squid_80]

Audionut, I don't think checkasm completed successfully because thats exactly where mine stops. It outputs an ASCII smile when it finishes, so obviously ours didn't.
Oh, now I see checkasm is also crashing on a similar instruction:
image00000000_00400000+0x9dba4:
00000000`0049dba4 660f7f7c2418 movdqa oword ptr [rsp+0x18],xmm7 ss:00000000`0022ecf8=00000000003d81fd00000000004a4088How come checkasm doesn't print the function being tested before actually running the test, and print the [OK] afterwards?

[Snowknight26]

Yea, thats the same instruction that I was getting when running it through WinDBG.

[akupenguin]

How come checkasm doesn't print the function being tested before actually running the test, and print the [OK] afterwards?
Because it's simpler this way, especially if I want to print extra diagnostics without screwing up the formatting, or I don't want to print anything if there turns out not to be an asm version of the current cputype. And I've never had an opportunity to wonder what was crashing, since I normally use checkasm to verify changes to one function at a time. And the next step after a crash is to get a backtrace, which would tell me the function name if I didn't know it already.

[MasterNobody]

Here is test2 version (where I hopefully fixed previous crash): x264win64_mingw_test2.rar (http://www.mediafire.com/?4otiwbjmrex)
Also compiled x264_avs.exe (x264.exe with AviSynth support) but don't know would it work (because there is no AviSynth for x64 as I know), it depends from AVIFIL32.dll (does it exists on all Windows x64 versions? Vista?)

[squid_80]

Avisynth x64 (http://forum.doom9.org/showthread.php?t=87623). It works with x264_avs.exe (crash is fixed too).
Any chance of adding pthread support? Compared to 32-bit with --threads 1 it looks 10% faster.

[burfadel]

Whats the performance of the 64 bit version like? since most of the optimisations are 32 bit(?) only?

[Dark Shikari]

since most of the optimisations are 32 bit(?) only?Since when? This would be news to me.

[squid_80]

Whats the performance of the 64 bit version like? since most of the optimisations are 32 bit(?) only?
If you're talking about MasterNobody's 64-bit x264, I just said it's 10% faster. If you're talking about avisynth x64, it varies greatly depending on the script; in a lot of cases there isn't fallback C code for avisynth's asm so I just left the functions out. This means a script may not open but if it does it should always (in theory) be faster. AFAIK available plugins are limited to the ones listed on my homepage.

[Audionut]

Just to confirm that it's working here too. And thread support would be great.

[kemuri-_9]

the mingw64 team already has a working copy of win x64 pthreads, but first let's get the asm situation settled first ok?
pthreads is external to x264; the asm is internal to it, so asm needs to get fixed.

[squid_80]

Well for starters it appears the asm is already working.
The last time I built pthreads for win64 (which admittedly was over 2 years ago) there was a small handle leak issue that Cef found and fixed. Apart from that all I had to do was change the target architecture, so IMO it's not like I was asking for a huge time-consuming task to be performed.

[kemuri-_9]

well, mingw 64 is currently using gcc 4.4.0 which is still in development (aka broken?),
so it would likely be easier to get a fully capable x64 version if the msvc project was reinstated and brought up to date

[RunningSkittle]

c:\x264_64>x264_avs --crf 22 -o D:\dvdrips\Testing\test.264 D:\dvdrips\pulp.avs
x264 [error]: could not open input file 'D:\dvdrips\Testing\pulp.avs'

Q6600 G0 (333*9)

c:\x264_64>checkasm
x264: using random seed 213923
x264: MMX
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- pixel var : [OK]
- pixel hadamard_ac : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- sub_dct4 : [OK]
- add_idct4 : [OK]
- dct4x4dc : [OK]
- idct4x4dc : [OK]
- zigzag_frame : [OK]
- zigzag_field : [OK]
- mc luma : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
- deblock : [OK]
- quant : [OK]
- coeff_last : [OK]
- cabac : [OK]
x264: MMX Cache64
- pixel sad : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
x264: SSE2Slow
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]
- pixel sa8d : [OK]
- pixel var : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- ssim : [OK]
- sub_dct4 : [OK]
- sub_dct8 : [OK]
- add_idct4 : [OK]
- add_idct8 : [OK]
- hpel filter : [OK]
- intra pred : [OK]
- deblock : [OK]
- quant : [OK]
- dequant : [OK]
- denoise dct : [OK]
- decimate_score : [OK]
- coeff_last : [OK]
x264: SSE2Fast
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel satd : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- pixel var : [OK]
- pixel hadamard_ac : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- zigzag_frame : [OK]
- mc luma : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
x264: SSE2Fast Cache64
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- mc luma : [OK]
x264: SSE3
- pixel sad : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
x264: SSSE3
- pixel satd : [OK]
- pixel sa8d : [OK]
- pixel hadamard_ac : [OK]
- intra satd_x3 : [OK]
- intra sad_x3 : [OK]
- esa ads: [OK]
- zigzag_frame : [OK]
- mc chroma : [OK]
- mc wpredb : [OK]
- hpel filter : [OK]
- lowres init : [OK]
- intra pred : [OK]
- quant : [OK]
- denoise dct : [OK]
- decimate_score : [OK]
- coeff_last : [OK]
x264: SSSE3 Cache64
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- mc luma : [OK]
x264: PHADD
- pixel satd : [OK]
- zigzag_frame : [OK]
x264: All tests passed Yeah :)

:)

[LoRd_MuldeR]

well, mingw 64 is currently using gcc 4.4.0 which is still in development (aka broken?),
so it would likely be easier to get a fully capable x64 version if the msvc project was reinstated and brought up to date

Well, MatserNobody used "gcc: 4.4.0 20080921 (experimental)" and apparently his build is now fully working (it only lacks pthreads).

[squid_80]

well, mingw 64 is currently using gcc 4.4.0 which is still in development (aka broken?),
so it would likely be easier to get a fully capable x64 version if the msvc project was reinstated and brought up to date
At least MSVC x64 builds shouldn't be crippled as much as x32, since the stack alignment problems shouldn't happen.

[LoRd_MuldeR]

At least MSVC x64 builds shouldn't be crippled as much as x32, since the stack alignment problems shouldn't happen.

According to checkasm.exe all ASM functions (inclduing SSE) do work in MasterNobody's build, created with MinGW x64.

If so, no need for MSVC x64. Or did he disable some ASM functions in that build? :confused:


[EDIT]

Think I read your post wrong :o

You said that MSVC x64 is less crippled than MSVC x86, not that MSVC x64 is less crippled than MinGW x64.

[squid_80]

Yeah x64 has a 16-byte aligned stack so MSVC shouldn't be able to screw up (in theory). But on the downside you lose inline asm.

[kemuri-_9]

well, the problem i've been having is actually getting
gcc 4.4.0 to accept the external pthreads library that the mingw64 team created,
and since it doesn't work tried compiling from the source, but then it doesn't contain all the function definitions that are necessary to actually work.

and yes, the stack alignment problem supposedly does not exist in the x64 msvc version,
so the only thing it would be missing is some inline asm compared to a speculated fully working x64 mingw version...

Edit:
mingw64 will have a better grounding once gcc 4.4.0 (which it's basing upon) has stabilized... (stabilized at least enough for an official release).

[roozhou]

well, the problem i've been having is actually getting
gcc 4.4.0 to accept the external pthreads library that the mingw64 team created,
and since it doesn't work tried compiling from the source, but then it doesn't contain all the function definitions that are necessary to actually work.

and yes, the stack alignment problem supposedly does not exist in the x64 msvc version,
so the only thing it would be missing is some inline asm compared to a speculated fully working x64 mingw version...

Edit:
mingw64 will have a better grounding once gcc 4.4.0 (which it's basing upon) has stabilized... (stabilized at least enough for an official release).

I wonder whether there is a way to use Windows API instead of pthread, which finally calls Windows API, to implement multithreading on win64.

[LoRd_MuldeR]

I wonder whether there is a way to use Windows API instead of pthread, which finally calls Windows API, to implement multithreading on win64.

The is a way to use the native Windows API, of course.

But the big point for pthreads is that it's portable between POSIX (Linux, Unix, etc) and Windows.
If you re-write x264 to use the Windows API instead of pthreads, then it wouldn't work for Linux anymore!
And you certainly don't wont to maintain two completely different code paths for Linux and Windows...

[roozhou]

The is a way to use the native Windows API, of course.

But the big point for pthreads is that it's portable between POSIX (Linux, Unix, etc) and Windows.
If you re-write x264 to use the Windows API instead of pthreads, then it wouldn't work for Linux anymore!
And you certainly don't wont to maintain two completely different code paths for Linux and Windows...

Code from xvidcore/motion_smp.h

#ifdef WIN32

# include <windows.h>
# define pthread_t HANDLE
# define pthread_create(t,u,f,d) *(t)=CreateThread(NULL,0,f,d,0,NULL)
# define pthread_join(t,s) { WaitForSingleObject(t,INFINITE); \
CloseHandle(t); }
# define sched_yield() Sleep(0);
static __inline int pthread_num_processors_np()
{
DWORD p_aff, s_aff, r = 0;
GetProcessAffinityMask(GetCurrentProcess(), (PDWORD_PTR) &p_aff, (PDWORD_PTR) &s_aff);
for(; p_aff != 0; p_aff>>=1) r += p_aff&1;
return r;
}

[LoRd_MuldeR]

So you want to implement your own wrapper for the Windows API to get a pthread-style interface.

What is the benefit over using the pthreads library, which basically does the same, is already working stable, saves you a lot of work and keeps your code clean & simple?

[squid_80]

Code from xvidcore/motion_smp.h

#ifdef WIN32

# include <windows.h>
# define pthread_t HANDLE
# define pthread_create(t,u,f,d) *(t)=CreateThread(NULL,0,f,d,0,NULL)
# define pthread_join(t,s) { WaitForSingleObject(t,INFINITE); \
CloseHandle(t); }
# define sched_yield() Sleep(0);

Xvid doesn't use any* synchronization methods though, I think x264 does.

(*Sleep() is not a synchronization primitive (http://www.virtualdub.org/blog/pivot/entry.php?id=62).)

[roozhou]

So you want to implement your own wrapper for the Windows API to get a pthread-style interface.

What is the benefit over using the pthreads library, which basically does the same, is already working stable, saves you a lot of work and keeps your code clean & simple?

That's not true. To build under MSVC one needs to download and compile pthread (it's neither in x264's git repository nor shipped with Visual Studio). This does cost me more time than writing a simple wrapper for Windows API as well as producing larger and slower binaries.

[Dark Shikari]

That's not true. To build under MSVC one needs to download and compile pthread (it's neither in x264's git repository nor shipped with Visual Studio). This does cost me more time than writing a simple wrapper for Windows API as well as producing larger and slower binaries.You do know the reason that pthreads were switched to in the first place, right?

They were very significantly faster than native threads. I doubt that Windows threads sped up that much in the 2 intervening years...

[roozhou]

You do know the reason that pthreads were switched to in the first place, right?

They were very significantly faster than native threads. I doubt that Windows threads sped up that much in the 2 intervening years...

This is very interesting because pthread uses native threads at last. If native thread runs slower than pthread that means you are not using threads in the right way.

[LoRd_MuldeR]

This is very interesting because pthread uses native threads at last. If native thread runs slower than pthread that means you are not using threads in the right way.

So just because the pthreads library makes use of the Windows API, doesn't mean that every single operation in the pthreads API will be mapped 1:1 to a Windows API function!

Whenever an operation can be handled in a library, which is running in user mode, an expensive syscall to the os kernel (switch context from user mode to kernel mode and back to user mode) can be avoided.

For the same reason pthreads on Linux now uses Futexes in order to avoid expensive syscalls whenever possible....

(BTW: In fact the Win32 API is a "high level" API (implemented by kernel32.dll and friends). There also is the "native" API (implemented in ntdll.dll), which usually is never called directly by an application. Only indirectly through Win32 API calls. The calls to the "native" API finally trap into kernel mode and execute the corresponding kernel function. So much for Win32 API is fast ^^)

[MasterNobody]

MSVC 2008 build with pthreads: x264win64_r1057_msvc2008.rar (http://stashbox.org/328761/x264win64_r1057_msvc2008.rar)
I have problems to compile with pthreads in MinGW. Would try it little more and then post patch in mailing list.

[MasterNobody]

Here is MinGW build with pthreads: x264win64_r1057_mingw.rar (http://stashbox.org/328884/x264win64_r1057_mingw.rar)
Now you can test speed between MSVC / MinGW and between 32bit / 64bit (32bit build: http://mirror01.x264.nl/x264/revision1057/x264.exe)

P.S. 64bit is not profiled (because I use cross-compiler on 32bit system)

[kemuri-_9]

MasterNobody, can you toss me the patches you have?
i can try and see if i can replicate everything in my MinGW64 environment.

[MasterNobody]

MasterNobody, can you toss me the patches you have?
i can try and see if i can replicate everything in my MinGW64 environment.

http://mailman.videolan.org/pipermail/x264-devel/2008-December/005300.html

[kemuri-_9]

confirmed as working (and thanks for fixing pthreads64 as well)

here's a build to match the current similar megui build

Platform: X86_64
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

fprofiled: yes

mods:
x264_win64_support.01.r1057.diff
x264_mingw_has_strtok_r.diff
x264_mingw_aligned_04.diff
x264_hrd_pulldown.09_interlace.diff


x264_x64.exe (http://kemuri9.net/dev/x264/x264_x64.exe)

getting a 5-10% speedup on my own quick tests compared to x86

Edit:
now with mp4 output support

[Audionut]

Nice work guys. I wonder if I can get a compiler working.

[kemuri-_9]

got gpac to compile in mingw64...
whoever decided that char *'s should be compared by using != in zlib should be shot.

Edit:
it's broken for being used to test string non-equivalency...

[squid_80]

got gpac to compile in mingw64...
whoever decided that char *'s should be compared by using != in zlib should be shot.

What's wrong with checking non-equivalency for a pointer?

[asarian]

x264_x64.exe (http://kemuri9.net/dev/x264/x264_x64.exe)

getting a 5-10% speedup on my own quick tests compared to x86

Pardon my ignorance in the matter, but I've been having bad experiences with running the x86 version of x264 in a 64-bit OS (Windows 2008 Server). Keep getting the dreaded "x264 stopped responding" error ever so often. So, does this one run stable inside Windows 2008 (Datacenter)?

And is the 'Ugly fades' issue, mentioned here:

http://forum.doom9.org/showthread.php?t=143444

also resolved in this version? Otherwise there's no point in having the extra threads for my multi-core environment.

Thanks

[Ranguvar]

got gpac to compile in mingw64...
whoever decided that char *'s should be compared by using != in zlib should be shot.

fix nfo pls? ;)

[squid_80]

got gpac to compile in mingw64...
whoever decided that char *'s should be compared by using != in zlib should be shot.

Edit:
it's broken for being used to test string non-equivalency...
Oh you mean char arrays not pointers :eek:

Yeah that's pretty dumb, almost as much as using strcmp for checking security signatures (thanks nintendo!).

[burfadel]

Pardon my ignorance in the matter, but I've been having bad experiences with running the x86 version of x264 in a 64-bit OS (Windows 2008 Server). Keep getting the dreaded "x264 stopped responding" error ever so often. So, does this one run stable inside Windows 2008 (Datacenter)?

Thanks

Next time that happens, check at what point it crashes. I have seen that happen a couple of times (out of hundreds of encodes), and for me it happens after encoding. The encoded file is intact and you can manually mux it with YAMB, well at least in my case anyway!

I thought it was an issue regarding one of the filters I was using, avisynth, or staxrip :)

[asarian]

Next time that happens, check at what point it crashes. I have seen that happen a couple of times (out of hundreds of encodes), and for me it happens after encoding. The encoded file is intact and you can manually mux it with YAMB, well at least in my case anyway!

I thought it was an issue regarding one of the filters I was using, avisynth, or staxrip :)

it has happened to me once after encoding, but often in-between somewhere. This "Program X stopped responding" feature in Vista and Windows 2008 is real annoying. Especially since I suspect x264 doesn't really hang, but is just being very busy (as it should be).

[burfadel]

Ah ok! For the couple of times it has happened to me, and its not just with the latest versions, I'm not sure how far back it would go (hasn't always done it), its always at the end of the encode! Its not a major issue since the encoded file is still fine, but it would be good if it didn't happen :) - I'm guessing that it would be a very difficult problem to find, and it may not even be x264 which is causing it. It could be avisynth... Are you using Avisynth v2.5.8 RC4?

[asarian]

Ah ok! For the couple of times it has happened to me, and its not just with the latest versions, I'm not sure how far back it would go (hasn't always done it), its always at the end of the encode! Its not a major issue since the encoded file is still fine, but it would be good if it didn't happen :) - I'm guessing that it would be a very difficult problem to find, and it may not even be x264 which is causing it. It could be avisynth... Are you using Avisynth v2.5.8 RC4?

I still use AviSynth 2.5.7, as I heard 2.5.8 is so radically different, it hardly even compares to 2.5.7 (okay, I exaggerate a bit, but there's some truth to it).

How could AviSynth be causing it, though, I wonder? I mean, even if it stopped serving frames, why would that cause x264 to hang? (as in: become unpollable to Windows). Still, using v2.5.8 RC4 might be worth the experiment.

[burfadel]

I haven't had any troubles with v2.5.8... except for that one possible problem which doesn't seem to do with avisynth after all! I hadn't mentioned about the 'crash' (not responding dialogue, nothing actually crashes!) because its not even repeatable, if you encode the same thing again the problem isn't there! I wonder if anyone else has found this?

[Audionut]

Guys, please take these OT posts to a different thread.

[asarian]

Guys, please take these OT posts to a different thread.

Trying to determine what causes x264 to become unresponsive in a Windows 64-bit environment is not off-topic for this thread.

[squid_80]

Yes it is. This thread is about x264 compiled specifically for windows x64. Your problem concerns running x86 x264 under a certain 64-bit version of windows. Since it's not a problem that is special to the builds in this thread it's OT.

[burfadel]

It is a little off topic, it was more a side mention as this problem could still apply to the x264 x64 version. As for me it only occurs at the very end of encoding (quite literally the file has been completely written), it could be related to the closing of the threads or something? - I'm not sure exactly what I'm talking about here but those that know what they're doing should know what I mean! If there is an issue with the closing of the threads/programme, it would also apply to the x64 version? It could well be a bug in Vista as well?...

I should also point out that calling it a Windows 64-bit environment is incorrect, as there are two versions of 64 bit Windows (including Vista). There's the x64 version of Vista and the not well known IA64 (Intel Itanium) version of Vista which are incompatible with each other :)

[techouse]

But x64 is the more used one; haven't seen IA64 in years.

P.S.: Nice to see that there's some *nix competition under construction now :)

[burfadel]

I didn't even know there was an IA64 version of Vista until I saw separate Vista updates for it on the Microsoft download site :). There must be a large enough number of IA64 systems out there for Microsoft to justify releasing an IA64 version of Vista.

Its a shame that a lot of programmes and filters used by people are no longer being developed, so there's almost no hope for x64 versions, such as most avisynth plugins, besweet (and the inability currently to use such things are Nero AAC encoder as part of the x64 chain), Virtualdubmod... and thats assuming Megui/Staxrip and others in x64.

I have heard there are different ways to get around this, one simple method being to stream frames to a memory buffer. This is done by using 32 bit x264 like normal, but with using a possible 'pass frames' method, stream the frames to a memory buffer (as long as its set up correctly) that can be separately accessed using a 64 bit version of x264. The x64 x264 can be accessed in theory by using the command prompt's 'call' command from x264 x86, such as 'call x264x64.cmd', where the .cmd file instructs to load the x64 version of x264. I'm not sure whether that would work or not, but it would be a good medium term workaround if it did and you'd still have a noticeable performance benefit.

[techouse]

I think that using a 64-bit *nix system with mplayer as the decoder is simpler than your method... Even for a *nix newbie.

[LoRd_MuldeR]

Can't we pipe from 32-Bit ffmpeg or 32-Bit MEncoder to a 64-Bit x264 encoder ???

[burfadel]

There is a way to do it. Technically there's no reason a 32 bit process can't call a 64 bit process and vice-versa, its just Microsoft disallows it citing 'security' reasons. There are workarounds for it, at least from what I've heard :)

[LoRd_MuldeR]

There is a way to do it. Technically there's no reason a 32 bit process can't call a 64 bit process and vice-versa, its just Microsoft disallows it citing 'security' reasons. There are workarounds for it, at least from what I've heard :)

I know that a 32-Bit Process can call a 64-Bit one for sure.
For example ProcessExplorer (http://technet.microsoft.com/de-de/sysinternals/bb896653.aspx) itself is a 32-Bit process, but it creates a 64-Bit child process when running on 64-Bit Windows.
The question is: Can STDOUT be redirect from a 32-Bit process to a 64-Bit one? And if so, how to do it?

[squid_80]

There is a way to do it. Technically there's no reason a 32 bit process can't call a 64 bit process and vice-versa, its just Microsoft disallows it citing 'security' reasons. There are workarounds for it, at least from what I've heard :)

I've never heard of this restriction. I know for a fact it's possible to pipe from 32-bit avs2yuv to 64-bit x264, similar to what Lord_MuldeR describes.

[LoRd_MuldeR]

What did I miss? :confused:

http://img101.imageshack.us/img101/28/avs2yuvdi4.png

[burfadel]

Thats pretty much what the restriction is, you can't call a 64 bit process from 32 bit and vice versa (hence the claim thats its not a valid Win32 application) :(

[clsid]

You are confusing DLLs with processes I think.

[LoRd_MuldeR]

Thats pretty much what the restriction is, you can't call a 64 bit process from 32 bit and vice versa (hence the claim thats its not a valid Win32 application) :(

You are confusing DLLs with processes I think.

Yes, passing data via STDOUT from a 32-Bit process to a 64-Bit process (or vice versa) should be possible.
That's because in that case we have to two completely separate processes. No 32-Bit and 64-Bit code is mixed in the same process.
I suspect I did not call it properly, because I didn't work with x264 and input from STDOUT so far...

[deets]

great stuff, is a nice 10% faster here as well :) ill keep an eye on this one

[LoRd_MuldeR]

LOL. Stupid my used wrong syntax to create the pipe and overwrote x264_x64.exe with random data :D

It does work now, like this:
http://img376.imageshack.us/img376/305/avs2yuv2pr8.png

However I get some continuous horizontal shift in the output file. I suspect the color format is wrong. Any ideas?


[EDIT]

Here is a sample of what I get:
http://www.mediafire.com/file/kcy4n2yi2mm/x264_x64-avs2yuv.7z

[asarian]

It is a little off topic, it was more a side mention as this problem could still apply to the x264 x64 version.

Precisely. I only referenced my bad experience with 32-bit x264 under Windows 2008 64-bit, with the question added whether this 64-bit version would not have similar problems.

Judging from LoRd_MuldeR's examples above, it seems you cannot just so easily create a 'drop-in' replacement environment for x264-64bit, so I may just stick bit x264 in a 32-bit OS for now.

[Blue_MiSfit]

I smell the need for some MeGUI updates soon to automate using 64 bit x264 :)

~MiSfit

[skystrife]

LOL. Stupid my used wrong syntax to create the pipe and overwrote x264_x64.exe with random data :D

It does work now, like this:
http://img376.imageshack.us/img376/305/avs2yuv2pr8.png

However I get some continuous horizontal shift in the output file. I suspect the color format is wrong. Any ideas?


[EDIT]

Here is a sample of what I get:
http://www.mediafire.com/file/kcy4n2yi2mm/x264_x64-avs2yuv.7z

When piping using stdout x264 is expecting raw yuv input, and avs2yuv is outputting y4m. The detection for this is based on file extension, and stdout has none, so x264 wrongly assumes yuv input.

Try using avs2yuv -raw file.avs - | x264 --stuff.

/me curses the lack of named pipes in windows.

[burfadel]

Yes, passing data via STDOUT from a 32-Bit process to a 64-Bit process (or vice versa) should be possible.
That's because in that case we have to two completely separate processes. No 32-Bit and 64-Bit code is mixed in the same process.
I suspect I did not call it properly, because I didn't work with x264 and input from STDOUT so far...

I guess passing data via stdout is kind of like sending the frames to a buffer? :)

I wasn't sure of the exact issues, but it does appear then that it is possible to use x64 x264 using 32 bit avisynth & Gui frontends???

[LoRd_MuldeR]

I guess passing data via stdout is kind of like sending the frames to a buffer?

I pipe is nothing but a FIFO buffer. Similar to a message queue.

I wasn't sure of the exact issues, but it does appear then that it is possible to use x64 x264 using 32 bit avisynth & Gui frontends???

Should be possible indeed! The front-end needs to call like this:

cmd.exe /c ""C:\Path to Tools\avs2yuv.exe" "C:\Path to Source\input.avs" -raw - | "C:\Path to Tools\x264_x64.exe" --crf 22 --progress --output "C:\Path to Output\output.mkv" -"

[roozhou]

I pipe is nothing but a FIFO buffer. Similar to a message queue.



Should be possible indeed! The front-end needs to call like this:

cmd.exe /c ""C:\Path to Tools\avs2yuv.exe" "C:\Path to Source\input.avs" -raw - | "C:\Path to Tools\x264_x64.exe" --crf 22 --progress --output "C:\Path to Output\output.mkv" -"

Are there mplayer/mencoder builds for win64? It should make things easier.

[LoRd_MuldeR]

When piping using stdout x264 is expecting raw yuv input, and avs2yuv is outputting y4m. The detection for this is based on file extension, and stdout has none, so x264 wrongly assumes yuv input.

Try using avs2yuv -raw file.avs - | x264 --stuff.

That works flawlessly! Thank you!

/me curses the lack of named pipes in windows.

Windows does have named pipes. No idea how to create them from in the shell, but:
http://msdn.microsoft.com/en-us/library/aa365150(VS.85).aspx

[LoRd_MuldeR]

Are there mplayer/mencoder builds for win64? It should make things easier.

Not really. Even if you had a working MEncoder x64 build with x264 x64 enabled, you'd still need Avisynth x64 :rolleyes:
Unless you don't work with Avisynth and rely on MEncoder's decoders/filters instead, of course...

The good thing with the "pipe" method is that we can use good old 32-Bit Avisynth including all the famous filters ;)

[Snowknight26]

My preliminary results show that the x64 version is 15% faster.

1080p video, ~15000kbps, essentially max settings.
x86: 1000 frames @ 2.11fps
x64: 1000 frames @ 2.44fps


C:\temp\Encoding\verticallimit>cd ..

C:\temp\Encoding>avs2yuv.exe verticallimit\verticallimit.avs -raw - | x264.x64
--pass 2 --bitrate 15093 --stats "verticallimit/verticallimit.stats" --bframes
3 --level 4.1 --ref 4 --mixed-refs --no-fast-pskip --b-pyramid --weightb --direc
t auto --deblock -3:-3 --subme 9 --trellis 2 --psy-rd 1.0:0 --analyse all --8x8d
ct --no-dct-decimate --me umh --threads auto --thread-input --progress --no-psnr
--output "verticallimit/verticallimit.mkv" - 1912x1040
x264 [warning]: width or height not divisible by 16 (1912x1040), compression wil
l suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
verticallimit\verticallimit.avs: 1912x1040, 24000/1001 fps, 1000 frames
x264 [info]: profile High, level 4.1
x264 [info]: slice I:14 Avg QP:15.25 size:128229
x264 [info]: slice P:505 Avg QP:19.12 size: 89484
x264 [info]: slice B:481 Avg QP:21.82 size: 53901
x264 [info]: consecutive B-frames: 26.7% 20.1% 14.3% 38.9%
x264 [info]: mb I I16..4: 29.0% 60.4% 10.5%
x264 [info]: mb P I16..4: 1.8% 7.0% 0.5% P16..4: 47.1% 24.7% 8.7% 0.1% 0
.1% skip:10.0%
x264 [info]: mb B I16..4: 0.0% 0.8% 0.0% B16..8: 57.6% 1.1% 2.2% direct:
11.2% skip:27.1% L0:44.2% L1:50.7% BI: 5.1%
x264 [info]: 8x8 transform intra:73.0% inter:45.9%
x264 [info]: direct mvs spatial:87.3% temporal:12.7%
x264 [info]: ref P L0 75.6% 13.4% 6.7% 4.4%
x264 [info]: ref B L0 68.2% 22.5% 9.3%
x264 [info]: ref B L1 92.2% 7.8%
x264 [info]: SSIM Mean Y:0.9486720
x264 [info]: kb/s:14582.2

encoded 1000 frames, 2.44 fps, 14582.61 kb/s

C:\temp\Encoding>ping 127.0.0.1

Pinging 127.0.0.1 with 32 bytes of data:
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128
Reply from 127.0.0.1: bytes=32 time<1ms TTL=128

Ping statistics for 127.0.0.1:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 0ms, Average = 0ms

C:\temp\Encoding>x264 --pass 2 --bitrate 15093 --stats "verticallimit/verticalli
mit.stats" --bframes 3 --level 4.1 --ref 4 --mixed-refs --no-fast-pskip --b-pyra
mid --weightb --direct auto --deblock -3:-3 --subme 9 --trellis 2 --psy-rd 1.0:0
--analyse all --8x8dct --no-dct-decimate --me umh --threads auto --thread-input
--progress --no-psnr --output "verticallimit/verticallimit.mkv" "verticallimit/
verticallimit.avs"
avis [info]: 1912x1040 @ 23.98 fps (1000 frames)
x264 [warning]: width or height not divisible by 16 (1912x1040), compression wil
l suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [warning]: 2nd pass has fewer frames than 1st pass (1000 vs 179031)
x264 [info]: profile High, level 4.1
x264 [info]: slice I:14 Avg QP:15.11 size:131874
x264 [info]: slice P:505 Avg QP:18.97 size: 93347
x264 [info]: slice B:481 Avg QP:21.68 size: 56405
x264 [info]: consecutive B-frames: 26.7% 20.1% 14.3% 38.9%
x264 [info]: mb I I16..4: 28.6% 60.5% 10.9%
x264 [info]: mb P I16..4: 1.9% 7.7% 0.6% P16..4: 46.0% 24.9% 8.8% 0.1% 0
.1% skip: 9.9%
x264 [info]: mb B I16..4: 0.0% 0.9% 0.0% B16..8: 57.5% 1.1% 2.3% direct:
11.5% skip:26.6% L0:44.0% L1:50.4% BI: 5.6%
x264 [info]: 8x8 transform intra:74.3% inter:45.0%
x264 [info]: direct mvs spatial:87.3% temporal:12.7%
x264 [info]: ref P L0 75.6% 13.3% 6.7% 4.4%
x264 [info]: ref B L0 68.5% 22.3% 9.3%
x264 [info]: ref B L1 92.2% 7.8%
x264 [info]: SSIM Mean Y:0.9500119
x264 [info]: kb/s:14599.9

encoded 1000 frames, 2.11 fps, 14600.31 kb/s

C:\temp\Encoding>pause
Press any key to continue . . .

[qyqgpower]

the output of mingw x64 version seems to be different from msvc and normal 32bit version. (of course, without --threads option)

mingw x64
x264 [info]: SSIM Mean Y:0.9804991
x264 [info]: PSNR Mean Y:44.020 U:47.821 V:48.225 Avg:44.906 Global:41.938 kb/s:1574.50
encoded 2184 frames, 19.59 fps, 1574.64 kb/s

msvc2008 x64
x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1574.53
encoded 2184 frames, 19.65 fps, 1574.68 kb/s

mingw x86
x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1574.53
encoded 2184 frames, 18.56 fps, 1574.68 kb/s

[roozhou]

Windows does have named pipes. No idea how to create them from in the shell, but:
http://msdn.microsoft.com/en-us/library/aa365150(VS.85).aspx

Rename attached file to Namedpipe.exe

usage:

Namedpipe pipename exename args

1) pipename must start with \\.\pipe\
2) exename is the program that you want to redirect its stdin
3) args is extra command-line arguments for exename, remember to put at least 2 spaces between exename and args

e.g. pipe mplayer's to neroAacEnc

1) in 1st cmd window, type "Namedpipe \\.\pipe\1.wav neroAacEnc.exe -q 0.3 -ignorelength -if - -of xxx.mp4"
2) in 2nd cmd window, type "mplayer xxx.xxx -vc NULL -vo NULL -ao fast:file=\\.\pipe\1.wav"

Source code(compiled in VC2005)

#pragma comment(linker,"/INCREMENTAL:NO /SECTION:.text,EWR /ALIGN:0x1000")
#include <Windows.h>

#define BUFSIZE 0x400000
char processname[256];
char pipename[64];
char *cmd,*tmp;
HANDLE hPipe/*,hCP,hChildIn*/;
STARTUPINFOA stinfo={0};
PROCESS_INFORMATION pinfo={0};
SECURITY_ATTRIBUTES sa={sizeof(SECURITY_ATTRIBUTES),NULL,TRUE};
void WINAPI Main()
{
DWORD quoteon,state,cmdend;
cmd = GetCommandLineA();
quoteon = 0;
state = 0;
cmdend=0;
tmp = pipename;
while (!cmdend)
{
switch(*cmd)
{
case 0:
cmdend=1;
*tmp=0;
goto END;
case '\"':
quoteon = !quoteon;
break;
case ' ':
case '\t':
if (quoteon)
{
switch(state)
{
case 2:
case 4:
*tmp++ = *cmd;
}
}
else
{
switch(state)
{
case 0:
state++;
break;
case 4:
cmdend = 1;
case 2:
*tmp = 0;
state++;
}
}
break;
default:
switch(state)
{
case 3:
tmp = processname;
case 1:
state++;
case 2:
case 4:
*tmp++ = *cmd;
}
}
cmd ++;
}
END:
if (state<2 || quoteon)
return;
if (INVALID_HANDLE_VALUE == (hPipe = CreateNamedPipeA(pipename,PIPE_ACCESS_DUPLEX,PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT,PIPE_UNLIMITED_INSTANCES,BUFSIZE,BUFSIZE,0,&sa)))
return;
ConnectNamedPipe(hPipe,NULL);
stinfo.cb = sizeof(STARTUPINFO);
stinfo.dwFlags = STARTF_USESTDHANDLES;
stinfo.hStdInput = hPipe;//hChildIn;
stinfo.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
stinfo.hStdError = GetStdHandle(STD_ERROR_HANDLE);
if (CreateProcessA(processname,cmd,0,0,TRUE,0,NULL,NULL,&stinfo,&pinfo))
{

WaitForSingleObject(pinfo.hProcess,INFINITE);
}
}

[kemuri-_9]

the output of mingw x64 version seems to be different from msvc and normal 32bit version. (of course, without --threads option)

which mingw64 build did you use?
...asking as there's quite a few roaming around now...

[qyqgpower]

which mingw64 build did you use?
...asking as there's quite a few roaming around now...
x64 builds are from #47 and #48
x86 build is from MeGUI's auto update (Skystrife's patched build)

commandline:
avs2yuv -raw "NCOP.avs" - | "x264.exe" --crf 22 --progress --sar 32:27 --output "F:\temp.mkv" - 720x480

I did another test without piping with x86 build:

x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1510.04
encoded 2184 frames, 19.81 fps, 1510.18 kb/s
output is exactly at same size (17,226,071 bytes) so the bitrate calculation here is more precision, but this is not very important.
The key is that the speed penalty of piping is larger than the speed advantage of x264 win64 build.

[roozhou]

x64 builds are from #47 and #48
x86 build is from MeGUI's auto update (Skystrife's patched build)

commandline:
avs2yuv -raw "NCOP.avs" - | "x264.exe" --crf 22 --progress --sar 32:27 --output "F:\temp.mkv" - 720x480

I did another test without piping with x86 build:

x264 [info]: SSIM Mean Y:0.9804984
x264 [info]: PSNR Mean Y:44.019 U:47.820 V:48.224 Avg:44.905 Global:41.938 kb/s:1510.04
encoded 2184 frames, 19.81 fps, 1510.18 kb/s
output is exactly at same size (17,226,071 bytes) so the bitrate calculation here is more precision, but this is not very important.
The key is that the speed penalty of piping is larger than the speed advantage of x264 win64 build.

Piping of Windows cmd.exe sucks.

[Dark Shikari]

Is the piping penalty mitigated by multithreading?

[kemuri-_9]

piping is also not required if one has avisynth x64.

but now one may say 'but none of my filters work in x64!'

you can either
1. create a lossless of your filtering from the x86 version to plainly open in avisynth x64 with AVISource()
(as there are x64 versions of lags and ffdshow)

2. pipe from avisynth x86 to x64 using the TCPDeliver plugin
(compatible x86 and x64 versions are on squid_80's avs x64 filter collection page - url is floating around somewhere on doom9...)
(also not sure if this is going to be more efficient than stdout piping or not... - my guess is it'll be worse to some degree)

3. limit what you're doing to what there is x64 filters for
(i lol inside as i write this last choice)

the options are there... though with differing degrees of feasibility...

[burfadel]

Nag Microsoft to allow high level mixed mode x86/64 instead of disallowing it altogether. Its purely a security reason why its disallowed, not a technical one :) By high level I mean non system level operations, such as antivirus & firewall (which require a 'driver' to allow access to the system, in which case the driver can be 64 bit and the programme 32 bit), device drivers etc that allow 'low level' access to the system. It is quite pointless disallowing mixed mode for instances of passing processed data in my opinion.

[LoRd_MuldeR]

Nag Microsoft to allow high level mixed mode x86/64 instead of disallowing it altogether. Its purely a security reason why its disallowed, not a technical one :) By high level I mean non system level operations, such as antivirus & firewall (which require a 'driver' to allow access to the system, in which case the driver can be 64 bit and the programme 32 bit), device drivers etc that allow 'low level' access to the system. It is quite pointless disallowing mixed mode for instances of passing processed data in my opinion.

Well, I'm not aware of any OS that allows 32-Bit and 64-Bit code to run in the context of the same process.
And it is possible to pass data between 32-Bit and 64-Bit processes, either trough pipes or through network sockets.

[squid_80]

Its purely a security reason why its disallowed, not a technical one :)
It's got nothing to do with security. How do you mix code that has different registers and pointer sizes?
If you want to do IPC between 32/64-bit programs the quickest method is to map the same chunk of memory into both processes and restrict access with an event/semaphore etc.

[burfadel]

Thats why I mentioned 'high level' access. It doesn't matter if there are different registers and pointer sizes for high level access, which this is. If it were a programme loading a DLL, then it would matter :) Even encoding using x264 and Avisynth is high level, as Avisynth merely serves the frames (that is, they're separate)!

[LoRd_MuldeR]

If an application (e.g. x264) uses Avisynth input, then it unavoidably needs to load the Avisynth DLL.
And mapping a 32-Bit DLL into the context of a 64-Bit process (or vice versa) isn't possible for the reasons mentioned by squid_80.

So you got exactly two options to use 64-Bit x264 with Avisynth input:
* Use 64-Bit Avisynth and accept the limitations it currently has.
* Call 32-Bit Avisynth from a separate 32-Bit process and transfer the "raw" data to the 64-Bit x264 process (may it be via pipe, via network socket or maybe via shared memory).

I don't know if it is possible to share memory between 32-Bit and 64-Bit processes on Windows, but pipes do work for sure.

[burfadel]

Ah ok! I was forgetting about the Avisynth DLL, in the context for which its loaded!

[squid_80]

3. limit what you're doing to what there is x64 filters for
(i lol inside as i write this last choice)Hey it works for me. I stopped doing x64 ports of avisynth plugins because I had everything I commonly use and everytime I ported a plugin at someone else's request they wouldn't bother giving me feedback on whether it even worked or not.

There was a brief attempt at creating a fourth option: Use avisynth x64 and use a filter32() plugin to use 32-bit filters. This worked by spinning off a 32-bit child avisynth process and using shared memory for IPC. It didn't get far off the ground though because I'm primarily a codemonkey and kinda suck at analysis and design.
Even encoding using x264 and Avisynth is high level, as Avisynth merely serves the frames (that is, they're separate)! AVIStreamRead is definitely not what I'd call a high-level call. For a start the function call uses four pointers. To go high level you'd have to use COM objects with custom marshalling code. Down that route lies madness.
I don't know if it is possible to share memory between 32-Bit and 64-Bit processes on Windows, but pipes do work for sure.Definitely possible. The only tricky part is you either use a static name for the shared memory object (bad because the name is visible across the whole OS) or use a temporary name and end up with a chicken and egg scenario; the child process needs the object name to do IPC, but the host process needs IPC to tell the child process what the object name is.

[imk]

If you want to compile with ICC instead of MSVC.

I put the files together here:
http://imk.cx/pc/x264/x264_icc_windows.tar.lzma

It's pretty straight forward.

Here's three different benchmark results:

http://pastie.org/346781
http://pastie.org/346664
http://pastie.org/346782

I ran the commands as many times as it took to get consistent results and then I took the highest score. Overall there was a standard deviation of +/- 0.2 FPS.

[mariush]

A few days ago I thought it would be fun to test if x264 would run in an emulated OS so I installed the latest version of Virtualbox on my Windows 2003 R2 32bit, and installed Windows 2003 R2 64 bit on the emulator.
This x264 64 bit version passed all asm tests just fine but I didn't bother copying video in the OS image and trying to compress stuff... My guess it would work about as fast as in 32 bit, as Virtualbox supports virtualization and my Q6600 also has is.

[LoRd_MuldeR]

A few days ago I thought it would be fun to test if x264 would run in an emulated OS so I installed the latest version of Virtualbox on my Windows 2003 R2 32bit, and installed Windows 2003 R2 64 bit on the emulator.
This x264 64 bit version passed all asm tests just fine but I didn't bother copying video in the OS image and trying to compress stuff... My guess it would work about as fast as in 32 bit, as Virtualbox supports virtualization and my Q6600 also has is.

When you run a 64-Bit guest system on a 32-Bit host system, which is possible with the latest version of VirtualBox, how does it work? Since the host OS is 32-Bit, the CPU is running in "Legacy Mode". That doesn't allow VirtualBox to switch to "64-Bit Mode", even if it uses its system driver to execute in "Ring 0" of the (32-Bit) host OS. Hence VirtualBox can't allow the guest OS to execute any instructions directly on the hardware. Consequently I have to assume that VirtualBox will emulate the CPU for the 64-Bit guest system entirely in software! If that is the case, it would totally destroy your encoding speed...

[mariush]

It runs just as fast as the normal Windows 2003 32 bit OS. I couldn't spot any difference and the processor usage was basically about 1%. Quite different comparing to running Windows 2003 32 bit inside Microsoft's VirtualPC.

If you have some specific tests/benchmarks you'd like me to run, feel free to suggest and if I have time I'll run them.

ps. Here's how it's reported by CPU-Z:

http://savedonthe.net/image/165/Clipboard02.png http://savedonthe.net/image/166/Clipboard03.png

So apparently the processor is not detected exactly as the processor is configured on my system (all stock settings), but otherwise seems to work just fine.

[LoRd_MuldeR]

According to the VBox manual:

If you want to use 64-bit guest support on a 32-bit host operating system, you must also select a 64-bit operating system for the particular VM. Since supporting 64 bits on 32-bit hosts incurs additional overhead, VirtualBox only enables this support upon explicit request.

Well, "additional overhead" is not very precise. You may run an encode in your virtual 64-Bit system and encode the same file with identical settings in your native 32-Bit system. Then compare the FPS values.

Note: You don't need to copy any files over to your guest OS, because VBox supports shared folders. That is: You can access the host filesystem from within the guest OS :)


[EDIT]

VBox still requires a 64-Bit CPU with VT-x or AMD-V support to run a 64-Bit guest on a 32-Bit host OS. So it seems VT-x/AMD-V allows the VM to run 64-Bit code natively, even if the host OS is 32-Bit.

But running a 64-Bit guest OS without 64-Bit hardware is not possible. Except for pure software emulation, of course. But that would be sloooooow...

[mariush]

I'm not installing any "64-bit guest support" so I don't think that paragraph applies. Maybe if I were to use shared folders or something like that. Usually I just set a FTP server and transfer the files I need inside the emulated os-es.
I'll try to do a test encoding soon, not right now as it's 6.14 AM here and I haven't slept at all.

[LoRd_MuldeR]

I'm not installing any "64-bit guest support" so I don't think that paragraph applies.

Your host OS is Windows 2003 R2 32-Bit and you installed Windows 2003 R2 64-Bit in the VirtualBox. Hence you are definitely using the "64-bit guest support" of VBox ;)

Maybe you mixed that up with "Guest Additions". These are required for Shared Folder support and stuff, but the cited paragraph isn't talking about them...

(BTW: The guest system isn't "emulated" but "virtualized". Otherwise you'd never get an acceptable performance. VBox only emulates in software when it can't be avoided)

I'll try to do a test encoding soon, not right now as it's 6.14 AM here and I haven't slept at all.

5:19 AM here :D

[mariush]

I've managed to do a test encoding on the computer and inside the Virtualbox emulated Windows 2003 64bit..

The sample was a bit over a minute, from Heroes, 1280x720.
The parameters were:

C:\temp\x264win64_mingw>x264.exe --bitrate 4096 --level 4.1 --ref 4 --mixed-refs
--bframes 3 --b-adapt 2 --weightb --direct auto --deblock -1:-1 --subme 7 --tre
llis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 50000 --vbv-maxra
te 50000 --me umh --threads auto --thread-input --progress --no-psnr --no-ssim -
-progress --output C:\encode.h264 c:\sample.yuv 1280x720


Here's how it works in 64bit:


x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x264 [info]: profile High, level 4.1
x264 [info]: slice I:12 Avg QP:13.24 size: 58848
x264 [info]: slice P:585 Avg QP:14.97 size: 31822
x264 [info]: slice B:929 Avg QP:16.93 size: 13128
x264 [info]: consecutive B-frames: 6.3% 20.3% 46.2% 27.2%
x264 [info]: mb I I16..4: 10.8% 75.0% 14.2%
x264 [info]: mb P I16..4: 7.8% 28.7% 3.0% P16..4: 39.3% 13.5% 5.6% 0.0% 0.0% skip: 2.1%
x264 [info]: mb B I16..4: 1.4% 1.1% 0.1% B16..8: 38.7% 0.7% 1.1% direct:25.4% skip:31.5% L0:44.5% L1:51.0% BI: 4.5%
x264 [info]: final ratefactor: 21.50
x264 [info]: 8x8 transform intra:70.1% inter:66.4%
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: ref P L0 76.0% 13.2% 7.0% 3.8%
x264 [info]: ref B L0 88.3% 7.9% 3.7%
x264 [info]: kb/s:4130.8

encoded 1526 frames, 1.66 fps, 4130.96 kb/s


What I found interesting was that Virtualbox used only 2 cores of the Q6600, and these two cores were at a pretty steady 25% usage. The other two cores were unused, running at about 1-4%.
If it were to use the full four cores and at a higher load, it probably would get closer to the results below.

Here's the normal run on the Windows 2003 R2 32bit:


x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.1
x264 [info]: slice I:11 Avg QP:13.33 size: 59475
x264 [info]: slice P:561 Avg QP:14.91 size: 32500
x264 [info]: slice B:954 Avg QP:16.88 size: 13317
x264 [info]: consecutive B-frames: 4.4% 18.9% 47.9% 28.8%
x264 [info]: mb I I16..4: 9.6% 76.9% 13.5%
x264 [info]: mb P I16..4: 8.0% 29.9% 3.3% P16..4: 38.3% 13.2% 5.4% 0.0% 0.0% skip: 2.0%
x264 [info]: mb B I16..4: 1.4% 1.2% 0.1% B16..8: 38.7% 0.7% 1.1% direct:25.5% skip:31.3% L0:44.4% L1:51.0% BI: 4.6%
x264 [info]: final ratefactor: 21.45
x264 [info]: 8x8 transform intra:70.1% inter:66.3%
x264 [info]: direct mvs spatial:99.6% temporal:0.4%
x264 [info]: ref P L0 76.0% 13.2% 7.0% 3.8%
x264 [info]: ref B L0 88.3% 8.1% 3.6%
x264 [info]: kb/s:4140.4

encoded 1526 frames, 7.52 fps, 4140.54 kb/s


I've installed guest additions in Virtualbox and I'll restart the image and run another test later with the additions active. Will probably edit this post and paste that result here.

ps, later edit:

rebooted virtual os, now with guest additions enabled, increased memory size to 1536MB just for kicks, tried changing the ide controller to sata in case virtual drive speed was limited by the ide controller but windows blue screened at boot.

I aborted the test at about 48%, because I didn't see much improvement:


x264 [info]: using cpu capabilities: MMX2 SSE2 SSE3 Cache64
x264 [info]: profile High, level 4.1
x264 [info]: slice I:4 Avg QP:13.08 size: 59964
x264 [info]: slice P:266 Avg QP:15.02 size: 31061
x264 [info]: slice B:372 Avg QP:16.92 size: 12741
x264 [info]: consecutive B-frames: 8.9% 27.3% 38.1% 25.7%
x264 [info]: mb I I16..4: 10.2% 73.7% 16.2%
x264 [info]: mb P I16..4: 7.1% 26.3% 2.9% P16..4: 41.2% 14.2% 5.8% 0.0% 0
.0% skip: 2.5%
x264 [info]: mb B I16..4: 1.2% 1.0% 0.1% B16..8: 39.6% 0.7% 1.0% direct:
24.0% skip:32.4% L0:45.3% L1:50.4% BI: 4.3%
x264 [info]: final ratefactor: 21.50
x264 [info]: 8x8 transform intra:70.3% inter:66.1%
x264 [info]: direct mvs spatial:99.7% temporal:0.3%
x264 [info]: ref P L0 75.6% 13.4% 7.1% 4.0%
x264 [info]: ref B L0 88.6% 7.8% 3.6%
x264 [info]: kb/s:4125.1

aborted at input frame 642
encoded 642 frames, 1.63 fps, 4125.36 kb/s



Again, the processor is not used as it should be, but it definitely uses more than one physical core (the CPU inside the image is detected as one core, one thread Core 2 Duo by CPU-Z) :

http://savedonthe.net/image/167/Clipboard03.png

[LoRd_MuldeR]

What I found interesting was that Virtualbox used only 2 cores of the Q6600, and these two cores were at a pretty steady 25% usage. The other two cores were unused, running at about 1-4%.

I think the virtual machine only exposes one single CPU to the guest OS...

[imk]

Yes. VirtualBox only exposes one CPU to the OS. However, VirtualBox itself is multi-threaded, so there will be speedups versus just a single CPU.

[MasterNobody]

Can somebody run this checkasm (http://stashbox.org/359637/checkasm.exe) and say me at which stage it crash (or may be it run correctly but this is highly doubtful). I currently can't test it myself because I am on 32-bit PC.

[Snowknight26]

Microsoft Windows [Version 6.0.6001]
Copyright (c) 2006 Microsoft Corporation. All rights reserved.

C:\temp>checkasm.exe
x264: using random seed 1123929904
x264: MMX
- pixel sad : [OK]
- pixel sad_aligned : [OK]
- pixel ssd : [OK]
- pixel satd : [OK]

C:\temp>

[dj_tjerk]

It just prints the first two lines here, no results whatsoever :| (but that might be my system... Win2003 x64 on an ol' X2 4400)

[LoRd_MuldeR]

It just prints the first two lines here, no results whatsoever :| (but that might be my system... Win2003 x64 on an ol' X2 4400)

Same here:

C:\Downloads>checkasm
x264: using random seed 1962273904
x264: MMX

C:\Downloads>

[MasterNobody]

Thanks. Then I will debug it tomorrow on 64-bit PC and upload new version with new patch.

[MasterNobody]

OK. Here is new checkasm: http://stashbox.org/360692/checkasm.zip. It contains 3 versions: 1 - check only results (and support large addresses), 2 - check result and nonvolatile registers (don't support large addresses), 3 - check result and nonvolatile registers (and support large addresses but will crash if they really used). So 1 and 2 must run fully correct and 3 can crash on the start if addresses large than 32 bit used. I am particularly interested in Core2, Core i7 and Phenom results (of all checkasm versions) so specify yours CPU type in the posts.

[vucloutr]

Q6600: "x264: All tests passed Yeah :)" for all three executables.

funny thing is it does even say [OK] for PHADD which is not supported by the Q6600 (right?)

[Dark Shikari]

Q6600: "x264: All tests passed Yeah :)" for all three executables.

funny thing is it does even say [OK] for PHADD which is not supported by the Q6600 (right?)It's supported, just not used, because it's slower on pre-Penryn CPUs.

[kemuri-_9]

Phenom here
CPUFlags = MMX2 SSE2Fast SSEMisalign LZCNT

checkasm1:
no errors

checkasm2 & 3:

x264: SSE_Misalign
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- mc luma : [OK]
failed to preserve register
- hpel filter : [FAILED]

[RickA]

Greets,

Q6600 here and all tests passed.

Glad there is activity on this thread again. I'm really excited to see some work done towards a 64bit x264 for Windows. So much so that I am trying to learn to compile Handrake and x264 via the instructions here http://trac.handbrake.fr/wiki/CompileOnWindows . Have the suggested software installed. Just need to figure out where to go from there.

Unfortunetly, my searches for guides have turned up more info on Linux compiling and less info on 32bit to 64bit while staying within the Windows realm. If anyone has some good suggestive reading for someone new to compiling code I'd welcome it. Willing to help the cause in whatever capacity I can.

Cheers,
Rick

[MasterNobody]

Phenom here
CPUFlags = MMX2 SSE2Fast SSEMisalign LZCNT

checkasm1:
no errors

checkasm2 & 3:

x264: SSE_Misalign
- pixel sad_x3 : [OK]
- pixel sad_x4 : [OK]
- mc luma : [OK]
failed to preserve register
- hpel filter : [FAILED]


Try with updated version: http://stashbox.org/360826/checkasm.zip

Also it is interesting to test with this registry hack (need reboot):
VirtualAlloc usually returns low addresses before high addresses. Therefore, your process may not use very high addresses unless it allocates a lot of memory or has a fragmented virtual address space. To force allocations to allocate from higher addresses before lower addresses for testing purposes, specify MEM_TOP_DOWN when calling VirtualAlloc or set the following registry value to 0x100000:

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Memory Management\AllocationPreference
checkasm1&2 must work correctly and checkasm3 must crash with this hack.

[kemuri-_9]

Try with updated version: http://stashbox.org/360826/checkasm.zip

that set has no errors for me.

[MasterNobody]

If there are no other error reports than here is new patch: x264_win64_support.02.r1077.diff (http://stashbox.org/361189/x264_win64_support.02.r1077.diff)

kemuri-_9
Did you try to test with registry hack?

[kemuri-_9]

Did you try to test with registry hack?

no... i hate rebooting the computer
and you shouldn't have people hack the registry for it to work anyway.

[MasterNobody]

and you shouldn't have people hack the registry for it to work anyway.
This is not for its working this is only for testing of large address support (addresses which can't fit in 32bit).

[kemuri-_9]

a possibility for testing is by using large mallocs and increment into it for what you actually needed for work/manipulation.

#define ADDR_32_MAX 2 * 1024 * 1024 * 1024
char *baseptr, *ptr;
baseptr = malloc( ADDR_32_MAX + i_size );
if( !baseptr )
{
printf( "large malloc failed (no resources)\n" );
exit(1);
}
ptr = baseptr + ADDR_32_MAX;

....

free( baseptr );

something like that.

[squid_80]

You'd rather make all mallocs attempt to grab >2GB (which wouldn't really be sufficient, it would need to be >4GB) rather than just flipping a registry flag?

[imk]

If there are no other error reports than here is new patch: x264_win64_support.02.r1077.diff (http://stashbox.org/361189/x264_win64_support.02.r1077.diff)

First, the old patch still works for me. However, since this is newer, I'm trying to make a build using this one instead.

How should this be compiled now? I noticed you added definition checks for WIN64 rather than ARCH_X86_64.

Previously, when compiling the objects, I would specify /DARCH_X86_64. When compiling x264 itself, I would specify /DWIN32 /DARCH_X86_64. The flags I used with yasm were -DARCH_X86_64 -D_WIN64.

Using these flags now will result in yasm spitting out errors.

If I remove -DARCH_X86_64 and change -D_WIN64 to -DWIN64, then yasm will error out with "invalid size for operand #." However, if I keep -DARCH_X86_64 and change -D_WIN64 to -DWIN64 (so the line is -DARCH_X86_64 -DWIN64) then it will work fine. I'm not sure if this is the right way to do it since you do specific checks for WIN64, else ARCH_X86_64.

[kemuri-_9]

it should be -DWIN64 -DARCH_X86_64

technically it shouldn't need -DWIN64 since

%ifdef ARCH_X86_64
%ifidn __OUTPUT_FORMAT__,win64
%define WIN64
%endif
%endif


is there to automatically add the effect of -DWIN64 when -f win64 -DARCH_X86_64 is met

the checks he has on several occasions are

ifdef WIN64
....
elifdef ARCH_X86_64
...


which actually works, since it hits the WIN64 block only when both are defined, but it kills readability imo, so i can understand where you're getting confused.

they should be switched to

ifdef ARCH_X86_64
ifdef WIN64
...
else
...
endif
...


for easier clarity imo

[sshoecraft]

Excellent job, MasterNobody!

Now can we get ya to look at the assembler code in mplayer/mencoder? :D

I was able to compile a win64 version of x264 with GPAC (mp4) & pthreads support.

I then compiled a win64 version of mencoder/mplayer - with x264/faac/pthreads - but without the filter/swscale/liba52 asm code...

EDIT: looks like mencoder is crashing in x264_encoder_close - still debugging :/

[imk]

If you want to compile with ICC instead of MSVC.

I put the files together here:
http://imk.cx/pc/x264/x264_icc_windows.tar.lzma

It's pretty straight forward.

I updated this with the latest Win64 patch. I also got rid of all the .cmd files and created a single Makefile for compilation.

I've been maintaining more benchmarks here:
http://spreadsheets.google.com/pub?key=pbffjdC6iUPWs2HtYHwZ2VQ&hl=sv

[MasterNobody]

I need somebody with Core2 / Core i7 / Phenom CPU to run this version of checkasm (both 32-bit and 64-bit):
checkasm.zip (http://stashbox.org/378305/checkasm.zip)
Used patch:
x264_win64_support.05.r1090.diff (http://stashbox.org/378306/x264_win64_support.05.r1090.diff)

[Snowknight26]

All pass on both on my Q6600.

[Audionut]

Q6600

Windows 7 64 all ok
Windows XP 32 in a VM all ok.

[kemuri-_9]

all ok on my Phenom

[darkip]

All ok on my Core2Duo Extreme x7900

[tph]

All ok on C2D E6400 @ Vista64.

[LoRd_MuldeR]

All okay. 32-Bit and 64-Bit binary. Tested on Q6600, WindowsXP x64-Edition.

[avivahl]

I need somebody with Core2 / Core i7 / Phenom CPU to run this version of checkasm (both 32-bit and 64-bit):
checkasm.zip (http://stashbox.org/378305/checkasm.zip)
Used patch:
x264_win64_support.05.r1090.diff (http://stashbox.org/378306/x264_win64_support.05.r1090.diff)Both pass w/ ALL OK.
Vista x64+SP1. Core 2 Duo E6750.

[kemuri-_9]

x264_win64_support.06.r1093.diff

this one's checkasm works fine on my phenom in both win x64 and ubuntu x64 (vmware) (checked x86 and x64 builds in each)

[DarkZell666]

All ok (checkasm32 & 64) on Q9300 inside Vista x64+SP1 inside VMWare running on Windows XP SP3 32-bit host ;)
(Edit: using "Intel-VTx or AMD-V" execution mode for the guest of course).

[prettyboy85712]

Both pass w/ ALL OK.
Vista x64+SP1 Core i7 920

[imk]

Intel Core 2 Duo E6600 / Vista x64
Both tests passed.


However, using both x264_win64_support.05.r1090.diff or x264_win64_support.06.r1093.diff will cause x264 crash when compiling with ICC. x264_win64_support.02.r1077.diff still works fine, however.

[darkip]

However, using both x264_win64_support.05.r1090.diff or x264_win64_support.06.r1093.diff will cause x264 crash when compiling with ICC. x264_win64_support.02.r1077.diff still works fine, however.
I also found the same result when compiling using ICC

[MasterNobody]

imk, darkip
May be you would provide more information than simple "x264 crash when compiling with ICC"? I don't have ICC so I can't test it myself. Provide at least exception code, disassembled code near crashed instruction (with mark on crashed instruction), registers state, CPU type, OS type.
Also how do you compile it? YASM now needs "-f win32 -m amd64 -DARCH_X86_64" and may be "-DPREFIX" params.

[imk]

I was not aware of the need for -f win32 now. I've been using -f x64 (same as -f win64). I will try with that first and post here again.

[kemuri-_9]

it changed from -f win64 to -f win32 at akupenguin's request...

Yes, `yasm -f win32` is the same as -f win64, and this way is more convenient for ffmpeg's configure, I think.

[imk]

Yeah, I just compiled a build using -f win32 and it works fine now. Thanks. :)

[skystrife]

I'm getting something odd occurring with my builds that I'm having some trouble pinpointing. The build compiles just fine and runs without crashing, but if I use my three sample videos that I use for fprofiling generally, the encoded videos of the compiled build end up being corrupt (I can upload a sample if needed). However, if I fprofile with just one video, the resulting encode from that binary is fine.

My three clips are a 200-frame sample of Elephants Dream, a 200-frame sample of Dark Shikari's LosslessTouhou.mkv, and a 100-frame sample of crowdrun.

tl;dr:


Zero Video Test = fine


One Video Test

elephants_dream = fine
LosslessTouhou = fine
crowd = fine


Two Video Test

elephants_dream & touhou = fine
touhou & crowd = fine
elephants_dream & crowd = fine

Testing elephants_dream + crowd atm, but I find it bizarre that fprofiling with the three videos would cause such an issue to occur, while it doesn't happen with just one video. Does anyone have an idea of what may be happening? (sources are are in y4m format)

(I am using the latest patch from the mailing list, but this problem is replicated using the r1077 patch as well.)

EDIT: the problem only occurs with all three videos. o_O

[Dark Shikari]

I could imagine that a slightly different fprofile would result in a slightly different compilation, which could in theory be the difference between a correct compilation and a miscompilation.

[kemuri-_9]

which mingw 64 binary build are you using?

periodically about once a week i try the latest builds and they've continually been broken (miscompile x264 at the least)

not to mention that mingw64 does use gcc 4.4.0 which has yet to release a stable version.

[skystrife]

gcc version 4.4.0 20081123 (experimental) (GCC)

I tried upgrading to the latest snapshot after the first few tries with all three vids failed, but the latest one fails on ./configure. -_-;

[komisar]

skystrife, try using a different compiler eg "gcc version 4.3.4 (prerelease)". I have no problem with him.

[skystrife]

Do you by chance have a link to a 64-bit compilation compatible gcc 4.3.4 for Windows? I can only seem to find the 4.4.0 series that the mingw-w64 project provides.

[komisar]

skystrife, i have build all mingw-w64 by hand from svn.
svn link for gcc: svn://gcc.gnu.org/svn/gcc/branches/gcc-4_3-branch

P.S. If you interest, I archive my cross 32 and 64 directory of mingw64 with gcc 4.3.4 for you...

[darkip]

It seems that latest recent revision of x264 doesn't work with x264_win64_support.06.r1093.diff, I think they've played with some of the assembly

[kemuri-_9]

It seems that latest recent revision of x264 doesn't work with x264_win64_support.06.r1093.diff, I think they've played with some of the assembly

Dark_Shikari added in some new assembly:
x264_add16x16_idct_dc_[mmx,sse2,ssse3]

and altered the quant asm functions to have a return type of int rather than void while adding in versions that see benefit on sse4 processors only (so thus called sse4 versions).

[Dark Shikari]

I have reports that MasterNobody's x64 patch is causing serious video corruption issues with some settings, so I would advise against using it until they are fixed.

[MasterNobody]

I have reports that MasterNobody's x64 patch is causing serious video corruption issues with some settings, so I would advise against using it until they are fixed.
May be you will report this to me, so I can reproduce it (also which version of patch were used? And which build was it?).

[Dark Shikari]

May be you will report this to me, so I can reproduce it (also which version of patch were used? And which build was it?).

http://www.stfcc.org/misc/x264/bug

[MasterNobody]

http://www.stfcc.org/misc/x264/bug
Version of patch? 32-bit or 64-bit build of x264? fprofiled build (if yes than what about without profiling because gcc 4.4.0 is very unstable at profiling)?

As I guess it was buggy (due fprofiling) skystrife's build: x264.1093M.x64.exe (http://skystrife.com/x264/x264.1093M.x64.exe)

[MasterNobody]

By the way:
New version of patch compatible with revision 1096:
x264_win64_support.07.r1096.diff (http://stashbox.org/383993/x264_win64_support.07.r1096.diff)
and binaries of checkasm for testing:
checkasm_1096.zip (http://stashbox.org/384000/checkasm_1096.zip)

[komisar]

IMHO this is the gcc-bug. I test Dark Shikari sequence and don't see this bug.
x264.1093kMod.k8.x64.exe (http://komisar.gin.by/x264.1093kMod.k8.x64.exe) fprofiled with x264_win64_support.06.r1093.diff
x264.1093kMod.k8.x64.exe --version
x264 0.66.1093kMod 1df50b9
built on Jan 29 2009, gcc: 4.3.4 20090128 (prerelease) (by_Komisar)

x264.1096M.x64.kom.exe (http://komisar.gin.by/test/x264.1096M.x64.kom.exe) with x264_win64_support.07.r1096.diff
x264.1096M.x64.kom.exe --version
x264 0.66.1096M 4c171c3
built on Feb 1 2009, gcc: 4.3.4 20090128 (prerelease) (by_Komisar)

x264.1096M.x64.kom.profiled.exe (http://komisar.gin.by/test/x264.1096M.x64.kom.profiled.exe) fprofiled with x264_win64_support.07.r1096.diff

[Dark Shikari]

If there's a bug in GCC, you should try to find exactly what code is being miscompiled. (I've found in the past it was possible to massage the code into making GCC compile it correctly.)

[komisar]

As you may remember, I tried to use gcc-4.4 earlier and pointed out issues with fprofile. But while gcc-4.4 very unstable I refused to use it.

[kemuri-_9]

komisar, could you make a x86_64 capable binary set of gcc 4.3.3 since it's officially out now?

cross-compiler from i686 or native x86_64 bins, whatever works for you/whatever you've been doing.

or even list the steps you've been taking to compile the x86_64 versions so we can make our own.

Edit:
skystrife, in the meantime have you checked to see if the x86_64 bin set 20090110 in mingw64's build list has the same problems?

[komisar]

kemuri-_9, ok. This my toolchain for crosscompile x86_64-pc-mingw32:
http://komisar.gin.by/test/mingw.komisar.7z
Step for make you own toolchain described in "mingw-w64/mingw-w64-doc/howto-build/mingw-w64-howto-build.txt" from
svn https://mingw-w64.svn.sourceforge.net/svnroot/mingw-w64/trunk

P.S. for correct compiling x264 with my mingw need patch of Makefile:libx264.a: .depend $(OBJS) $(OBJASM)
$(AR) rc libx264.a $(OBJS) $(OBJASM)
$(RANLIB) libx264.a
and%.o: %.asm
$(AS) $(ASFLAGS) -o $@ $<
# delete local/anonymous symbols, so they don't show up in oprofile
-@ $(STRIP) -x $@
and after configure change config.makARCH=X86_64
SYS=MINGW
AR=x86_64-pc-mingw32-ar
RANLIB=x86_64-pc-mingw32-ranlib
STRIP=x86_64-pc-mingw32-strip
CC=x86_64-pc-mingw32-gcc
P.P.S. Dark Shikari, gcc version 4.4 is unusable with fprofile. His make wrong .gcda-files for second pass of make with fprofiled.

[Snowknight26]

As I guess it was buggy (due fprofiling) skystrife's build: x264.1093M.x64.exe (http://skystrife.com/x264/x264.1093M.x64.exe)

Good guess. That was the build being used.

Just tested both of komisar's r1096 builds and it does not seem to happen.

[MasterNobody]

By the way I am waiting for test results of checkasm on Core2 / Core i7 / Phenom CPUs: post1244319 (http://forum.doom9.org/showthread.php?p=1244319#post1244319)

[darkip]

By the way I am waiting for test results of checkasm on Core2 / Core i7 / Phenom CPUs: post1244319 (http://forum.doom9.org/showthread.php?p=1244319#post1244319)

All tests pass on my Core2Duo Exteme x7900

[RickA]

All tests passed on my Q6600 for checkasm_1096 both 32 and 64. Really appreciate the work you all are doing on this. :-)

Cheers,
Rick

[DarkZell666]

Checkasm_1096 all ok (32 and 64) on Q9300 (Vista x64).

[paulvdb]

Checkasm_1096 all ok on i7 (XP x64).

[Esurnir]

Checkasm_1096_32/64 all ok on Memrom Core 2 Duo T5800 Vista 64 bit

[LoRd_MuldeR]

Checkasm_1096 all OK on Q6600, 32-Bit and 64-Bit, WindowsXP x64-Edition

[kemuri-_9]

oh right... the checkasms are fine on my phenom.

[TEB]

checkasm is fine here vista64 sp1 + Q9550

[Buggle]

By the way:
New version of patch compatible with revision 1096:
x264_win64_support.07.r1096.diff (http://stashbox.org/383993/x264_win64_support.07.r1096.diff)
and binaries of checkasm for testing:
checkasm_1096.zip (http://stashbox.org/384000/checkasm_1096.zip)
Both checks are fully OK
System: Merom T5500 on Windows 7 x64

PS. I run the checks in a normal cmd instance, or is administrative access necessary for this check?

[Buggle]

As I do not understand 90% of what is going around here, I was hoping if it were possible for someone to create some sort of a x264 x64 site somewhere, maybe even on the doom9 servers or on x264.nl. Such a site would track the developments that surround the x64 edition of x264, give some heads up of where to start (including the possibilities of 64bit encoding and what is needed for it) and give users like me, who have no clue about programming, compiling or patches or anything, a heads-up of where they can help to speed up development. Or just make this one a sticky and update the first post to reflect 'things that are done and/or things to be done'. I would personally prefer the latter option.

The reason for this is because a speed up of around 10% is quite a large amount in the land of encoding to my eyes, and x64 is becoming more and more 'regular' nowadays with computers having 4+GB, especially with Vistas high memory requirements and memory prices this low (I bought Kingston Hyper-X 2GB DDR2 CAS5 for 25 euros the other day). I find it surprising that so little is done (at least, that is my perception) to fully support x64 from the ground up. In a year or five it should be the main focus at the expense of x32, in my eyes.

Please don't take this the wrong way, of course I am glad that at least are people putting effort in developing this in the first place and I really appreciate that. And if I could, I would help. I just think that this topic requires and deserves more attention and coordination. Which way you turn it, it is still the future.

[kemuri-_9]

x64 has been supported for a long time, just not on windows since it has its (guess what) own standard for it.

if you were on a linux system, you'd have no problems with x64 support at all.
it's windows that's throwing the kink into the compatability and with the main devs not using windows,
it takes a windows user that knows what's going on (like MasterNobody) to work on the support for it.

[Buggle]

x64 has been supported for a long time, just not on windows since it has its (guess what) own standard for it.

if you were on a linux system, you'd have no problems with x64 support at all.
it's windows that's throwing the kink into the compatability and with the main devs not using windows,
it takes a windows user that knows what's going on (like MasterNobody) to work on the support for it.

Hmm, I didn't know that. As I said, I know nothing about programming and its kinks (of course I do know MS creates its own standards for just about everything) so I thought having platform independent code means it can automatically run on windows. Guess it doesn't.

But with the (I presume) majority part of the user base running windows my previous post hasn't lost it's value, I guess. It would still be nice to make this a sticky and adjust the opening post a little bit to reflect what is going on in this thread.

[laserfan]

...if you were on a linux system, you'd have no problems with x64 support at all.Is there a thread or site somewhere that 64-bit encoding using x264 is discussed?

Might be fun to dual-boot to a Linux platform if it's not TOO difficult, or require tons of "tribal knowledge".

[kemuri-_9]

Is there a thread or site somewhere that 64-bit encoding using x264 is discussed?

Might be fun to dual-boot to a Linux platform if it's not TOO difficult, or require tons of "tribal knowledge".

can always join #x264 on irc.freenode.net for discussion about x264.

as for the dual-booting part, most windows users i know usually find ubuntu flavors the easiest to deal with.

anyways, i've gotten x264 to build with komisar's gcc bin set (solving its quirks to suit my needs).
ended up using ./configure --host=x86_64-pc-mingw32 --extra-ldflags=-lavifil32 (outside of my pthread and gpac dependencies) to get it to compile properly.

skystrife:
have you tried komisar's gcc 4.3.4 (pre) and see if the problems still persist?

[skystrife]

Compiled my own gcc 4.3.4 (as well as the rest of the mingw-w64 package) and all is well now. Stupid broken 4.4.0... why would they (the mingw-w64 team) even be distributing it if it's so experimental?

[LoRd_MuldeR]

Stupid broken 4.4.0... why would they (the mingw-w64 team) even be distributing it if it's so experimental?

Well, experimental means that bugs are to be expected. If nobody tries to use it on a "real" project, nobody will encounter and report the remaining bugs...

[Ranguvar]

I thought having platform independent code means it can automatically run on windows. Guess it doesn't.

The C code in x264 is indeed platform independent - mostly, anyways. But the reason x264 is so ridiculously fast is because a lot of the code is handwritten assembly - the one-step-away-from-the-machine language which C is compiled (converted) into. Since the compiler is not magic, handwritten assembly is a lot of times faster. The downside is that assembly is different for every CPU architecture. Today, that's not a huge deal, since x86 and x86-64 pretty much dominate (if you don't have one of those, either you're using an old Mac or you don't need to read this, chances are). But, you run into good old MS making up their own 64-bit standard, and there's problems. You _can_ disable the handwritten assembly and use compiled C for everything - that will work 64-bit on 64-bit Windows. But the performance loss for losing the assembly will far excede the 64-bit boost.

[squid_80]

I find it surprising that so little is done (at least, that is my perception) to fully support x64 from the ground up. In a year or five it should be the main focus at the expense of x32, in my eyes.
I did a lot of working porting avisynth, avisynth plugins and various codecs to x64: http://members.optusnet.com.au/squid_80/

Judging from the feedback I got, no-one really cares. Certainly no-one ever offered to help port avisynth plugins, which is the major thing holding back avisynth64 (Also belligerent plugin authors who refuse to release their sources even when confidentiality is promised).

[Buggle]

The C code in x264 is indeed platform independent - mostly, anyways. But the reason x264 is so ridiculously fast is because a lot of the code is handwritten assembly - the one-step-away-from-the-machine language which C is compiled (converted) into. Since the compiler is not magic, handwritten assembly is a lot of times faster. The downside is that assembly is different for every CPU architecture. Today, that's not a huge deal, since x86 and x86-64 pretty much dominate (if you don't have one of those, either you're using an old Mac or you don't need to read this, chances are). But, you run into good old MS making up their own 64-bit standard, and there's problems. You _can_ disable the handwritten assembly and use compiled C for everything - that will work 64-bit on 64-bit Windows. But the performance loss for losing the assembly will far excede the 64-bit boost.
Yeah, I know about the assembly codes existence, but since that is direct 'processor language' it should work independent from the OS, right? Or is on Windows the assembly code somehow not directly run on the proc but 'translated' by the windows application interface and only then passed on? I always understood assembly to be the mother of all platform independent (meaning OS) code since it directly instructs the processor. And since x64 is a hardware standard...
But obviously I'm misunderstanding something here :(

[Buggle]

I did a lot of working porting avisynth, avisynth plugins and various codecs to x64: http://members.optusnet.com.au/squid_80/

Judging from the feedback I got, no-one really cares. Certainly no-one ever offered to help port avisynth plugins, which is the major thing holding back avisynth64 (Also belligerent plugin authors who refuse to release their sources even when confidentiality is promised).
Then here I am thanking you for the work you did. I just didn't know about it, and will go about using it once I reinstalled my desktop to Vista x64, or start using it on my 7 x64 laptop :). I normally use MeGUI, but at Xvid times I used to do things a little more manual with GordianKnot (good old days...) so there shouldn't be a reason for me to not use this.

Could I have both the normal Avisynth and the 64 bit installed, or do I need to choose between them? I only use a few filters (SplineResize, Yadif, FieldDeinterlace, DeGrainMedian, TemporalSoften, Convolution3D, VagueDenoiser and some other softener standard in megui which I cannot find the name of) so it shouldn't be much of a problem to get that to work with some basic batch scripts.
What I'm just worried about is the resizing calculation, which I have never learned to do manual (used to use GK and now MeGUI). I would want to use MeGUI to set up all the stuff like the script, settings of x264, zones and calculate bitrates and then encode in x64 mode. There now is a 64 bit x264 and obviously a x64 avisynth, so that's all one would need. DGDecode can still be 32 bit, since it doesn't 'serve' any frames or anything, right?

Maybe it would help to poke Doom9 somewhat about this subject? Frontpage or even a new guide would increase profile a little, I guess. Would give your work the appreciation it deserves.

Edit: looked at the files on your site, but I wouldn't know where to begin: many files I do not know or do not recognize, since I (being the lazy windows user that I am) always just install programs :P. But I guess I have to register the avisynth dll by using the *.reg file, but does it then have all the usual 'included' avisynth filters? Maybe you can give some heads up about what all the files are.

[nm]

Yeah, I know about the assembly codes existence, but since that is direct 'processor language' it should work independent from the OS, right? Or is on Windows the assembly code somehow not directly run on the proc but 'translated' by the windows application interface and only then passed on? I always understood assembly to be the mother of all platform independent (meaning OS) code since it directly instructs the processor. And since x64 is a hardware standard...
Instructions are the same, but the function calling convention is different in Windows: http://en.wikipedia.org/wiki/X86_calling_conventions#Microsoft_x64_calling_convention

[Buggle]

Found an interesting link that illustrates you are ahead of your time, squid ;) http://www.dailytech.com/article.aspx?newsid=13983

[paulvdb]

I did a lot of working porting avisynth, avisynth plugins and various codecs to x64: http://members.optusnet.com.au/squid_80/

Judging from the feedback I got, no-one really cares. Certainly no-one ever offered to help port avisynth plugins, which is the major thing holding back avisynth64 (Also belligerent plugin authors who refuse to release their sources even when confidentiality is promised).
I do care and would like to thank you. The biggest problem I've had so far was that there were no (recent) 64 bits x264 builds. Now that there are 64 bits x264 builds available I have started using it.

[STaRGaZeR]

I did a lot of working porting avisynth, avisynth plugins and various codecs to x64: http://members.optusnet.com.au/squid_80/

Judging from the feedback I got, no-one really cares. Certainly no-one ever offered to help port avisynth plugins, which is the major thing holding back avisynth64 (Also belligerent plugin authors who refuse to release their sources even when confidentiality is promised).

At that time I consider normal that feedback, after all 64-bit OSes were rare. But now they're not. I've been using your Decomb64 for real time IVTC in ffdshow for quite some time now, thanks :)

On a side note TIVTC would be a really nice addition. Or do you recommend another filter for hybrid material as Decomb is not very good at it?

[kemuri-_9]

I did a lot of working porting avisynth, avisynth plugins and various codecs to x64: http://members.optusnet.com.au/squid_80/

Judging from the feedback I got, no-one really cares. Certainly no-one ever offered to help port avisynth plugins, which is the major thing holding back avisynth64 (Also belligerent plugin authors who refuse to release their sources even when confidentiality is promised).

i have to admit, i only use avisynth x64 when opening lossless avis and tossing them to x264,
maybe with some minor filtering like a downsize and or vsfilter.

there's just not enough filters available to realistically be able to filter within avisynth x64,
most of the filters would require major asm overhauls which is just too time consuming if done by a single person.

and realistically, i also don't see a major effort to port them as the work required is too astronomically high.
people can live with x86 filtering until the point in time when 3.0 becomes stable.

[squid_80]

3.0 has been in development longer than the avisynth64 branch, and there hasn't been a single preview build. Make your own conclusions about that.

Updating the asm isn't hard, especially when using intel's compiler since it allows inline asm. But the issue I had is that people would ask for plugins to be ported and I'd do them, then not get any feedback. So rather than waste my time I did the handful of plugins I personally use on a regular basis for standard conversions (NTSC->PAL) and that's where I stopped. Now I'm way too busy working (anyone remember my user title used to be unemployed?) to do it any more.

There doesn't seem to be many new plugins coming out these days, maybe with a couple of people helping out it would be possible to do a big push and get all the commonly used (open-source) plugins ported.

[Buggle]

What would 3.0 bring that it would be that special and solve all our problems?

I would be glad to help where I can, but as I said I do not know anything (well, almost) about programming. Plus I'm currently quite occupied with both my masters degree and work. Still, I'd be glad to squeeze in some time to help the community, since I've been a thankful visitor of Doom9 for quite some time already (actually, from the moment DivX 5.0 came out, which is probably somewhere around 2002 or so) and would like to donate some effort in return.

I think it would be best to start checking the needs of the user base: what is x264 used most often for and which needs does that address, or in other words, which filters are most beneficial for that? As almost anyone who is currently encoding in x264 will have x64 compatible hardware, it really is a chicken and egg story. But maybe we shouldn't get our hopes up too high: windows 7 isn't out yet, and that will probably be 'our big break', since it will probably boost the amount of x64 installations.

[Ranguvar]

What would 3.0 bring that it would be that special and solve all our problems?

Most of the people you see eagerly anticipating 3.0 (like me) are wanting AviSynth untied from the Winblows platform - meaning, it will use GStreamer or something else instead of DirectShow/VfW. Cross-platform is good (GStreamer supports Linux, Windows, Mac, BSD...), and no matter what anyone says, AviSynth over avs2yuv or avsproxy is but a clumsy hack (not the quality of the tools, it's inherent) that isn't really acceptable as a long-term solution.

There's a lot of other stuff that was supposed to come in 3.0, too... http://en.wikipedia.org/wiki/AviSynth#AviSynth_3.0_code_rewrite

[kemuri-_9]

BugMaster:
akupenguin was asking about the status of this in the dev channel,
to be informed that the current patch is broken with the latest asm additions
think you could get a fix so it could make GIT sometime in the near future?

[MasterNobody]

Sorry for 2 days delay. I have a lot of work on my real job.
Here is update version of patch compatible with revision 1101:
x264_win64_support.08.r1101.diff (http://stashbox.org/392450/x264_win64_support.08.r1101.diff)
and binaries of checkasm for testing:
checkasm_1101.zip (http://stashbox.org/392451/checkasm_1101.zip)
also this time I compile binaries of x264 as reference variant for testing (they are not profiled so NOT the quickest one):
x264_1101.zip (http://stashbox.org/392453/x264_1101.zip)

P.S. As always I am waiting for testing on Core2 / Core i7 / Phenom CPUs.

[LoRd_MuldeR]

P.S. As always I am waiting for testing on Core2 / Core i7 / Phenom CPUs.

Thanks. Everything is fine (32-Bit and 64-Bit) on my Core 2 :D

[vucloutr]

checkasm all test ok for 32bit and 64bit with Q6600 & Server2008x64

[RickA]

All 32 & 64bit tests passed here as well. Q6600 w/ Server 2008 x64. :-)

[DarkZell666]

Checkasm 1101 ok on Q9300 + Vista x64.

[kemuri-_9]

All ok on my phenom.

[paulvdb]

Checkasm 1101 all ok on i7 + XP x64.

[lexor]

Checkasm is all good on i7+win7... yes I'm late for the party, but I just got the new toy and are currently in the process of testing it :)

[Dark Shikari]

My latest massive addition of assembly functions (http://git.videolan.org/?p=x264.git;a=commit;h=32615747d3ab5648b666b4f55531f47b3c075521) has broken the patch; MasterNobody says a fix will be up soon.

[darkip]

x264_win64_support.09.r1106.diff seems to be up, has anyone got a checkasm built yet?

[komisar]

x264_win64_support.09.r1106.diff seems to be up, has anyone got a checkasm built yet?
http://stashbox.org/396151/checkasm64.zip

[LoRd_MuldeR]

x264: All tests passed Yeah :)

[darkip]

All ok on my X7900 Core2Duo Extreme

[paulvdb]

checkasm ok on i7 / XP x64.

[MasterNobody]

And here is my official announce of new patch (too late ;) ).
patch: x264_win64_support.09.r1106.diff (http://stashbox.org/396156/x264_win64_support.09.r1106.diff)
checkasm: checkasm_r1107.zip (http://stashbox.org/396406/checkasm_r1107.zip)
x264: x264_r1107.zip (http://stashbox.org/396412/x264_r1107.zip)
As always need testing on Core 2 / Core i7 / Phenom CPUs both 32-bit and 64-bit checkasm

[tph]

All passed on C2D E6400, Vista64.

[paulvdb]

32 bit and 64 bit checkasm both ok i7 / XP x64

[kemuri-_9]

All ok on the phenom here....

[LoRd_MuldeR]

As always need testing on Core 2 / Core i7 / Phenom CPUs both 32-bit and 64-bit checkasm

Both are good http://forum.gleitz.info/images/smilies/thumb.gif

[lexor]

win7+i7, all clear for both checks.

[Esurnir]

all clear on 32 + 64 bit on vista 64 bit memrom core 2 duo.

[bob0r]

Simple question: Can i compile x264-64bit on a Windows XP 32bit, 32bit CPU system?

[MasterNobody]

Simple question: Can i compile x264-64bit on a Windows XP 32bit, 32bit CPU system?
Yes, you can (this is how I compile my builds). You can use cross-compiler from MinGW-W64 project (http://sourceforge.net/projects/mingw-w64/) but it is better to get komisar's build (http://komisar.gin.by/tools/mingw.generic.gcc433.7z) which use GCC 4.3.3 (not GCC 4.4.0 experimental) or compile it yourself. The only thing you can't do is to profile 64-bit builds on 32-bit system.

P.S. Command line for cross-compilation will look like this:
sh ./configure --host=x86_64-pc-mingw32 --cross-prefix=x86_64-pc-mingw32-
make

[bob0r]

Grab mingw-w64-bin_i686-mingw_20090210.zip ?
Stuff like yasm is included? Assuming you need 64bit of that too.

[kemuri-_9]

Grab mingw-w64-bin_i686-mingw_20090210.zip ?
Stuff like yasm is included? Assuming you need 64bit of that too.

you can use the x86 yasm you already have, since yasm natively handles all the formats itself independent of the compiler used for it.

[bob0r]

Ok, that means i can also use the 32bit stuff like ncftpput.exe in the 64bit environment?

[MasterNobody]

Grab mingw-w64-bin_i686-mingw_20090210.zip ?
It is better to get some of the alternative builds. Because GCC 4.4.0 used there is unstable now and miscompiles x264 code.

[bob0r]

It is better to get some of the alternative builds. Because GCC 4.4.0 used there is unstable now and miscompiles x264 code.

Which do you recommend for now?
Direct url surfnet.nl would be nice :)

[MasterNobody]

Which do you recommend for now?
Direct url surfnet.nl would be nice :)
I use already posted above: http://komisar.gin.by/tools/mingw.generic.gcc433.7z

[bob0r]

Ah thanks, i thought x264 build :)

[techouse]

I get this error when compiling it: http://pastebin.ca/1335317

[kemuri-_9]

I get this error when compiling it: http://pastebin.ca/1335317


x86_64-pc-mingw32-gcc -o x264.exe x264.o matroska.o muxers.o libx264.a -lvfw32 -s -fprofile-generate


it's using libvfw32, it needs to be using libavifil32.
so add in --extra-ldflags=-lavifil32 on the ./configure
as a simple fix.

the ./configure needs to be altered slightly to not check for libvfw32 if the arch is x86_64, since it doesn't hold all the necessary functionality on x86_64

[MasterNobody]

I get this error when compiling it: http://pastebin.ca/1335317
x86_64-pc-mingw32-gcc -o x264.exe x264.o matroska.o muxers.o libx264.a -lvfw32 -s -fprofile-generateThere is no 64-bit version libvfw.a of this library instead must be used libavifil32.a. For some reason configure choose wronge library (probably because directory with 32-bit libs is in search library path).
If you use komisar's mingw build then you probably need to make this changes:
1) move "<mingw_root>\include"=>"<mingw_root>\i686-pc-mingw32\include";
2) move all from "<mingw_root>\lib"=>"<mingw_root>\i686-pc-mingw32\lib" except "<mingw_root>\lib\gcc" directory;
3) reconfigure junction of "<mingw_root>\mingw\include" and "<mingw_root>\mingw\lib" to new locations.
Then both i686 and x86_64 would work correctly.

[MasterNobody]

the ./configure needs to be altered slightly to not check for libvfw32 if the arch is x86_64, since it doesn't hold all the necessary functionality on x86_64
There was ideas to change the order of library checks in configure:
-lvfw32 <=> -lavifil32
-lwsock32 <=> -lws2_32
This probably would work because libavifil32.a and libws2_32.a exists in both 32-bit and 64-bit. But I think the most correct way is fixing the reason of problem (removing 32-bit libs from 64-bit gcc lib search) and not making workarounds.

[techouse]

There is no 64-bit version libvfw.a of this library instead must be used libavifil32.a. For some reason configure choose wronge library (probably because directory with 32-bit libs is in search library path).
If you use komisar's mingw build then you probably need to make this changes:
1) move "<mingw_root>\include"=>"<mingw_root>\i686-pc-mingw32\include";
2) move all from "<mingw_root>\lib"=>"<mingw_root>\i686-pc-mingw32\lib" except "<mingw_root>\lib\gcc" directory;
3) reconfigure junction of "<mingw_root>\mingw\include" and "<mingw_root>\mingw\lib" to new locations.
Then both i686 and x86_64 would work correctly.

Even moar errors?!!?!?! http://pastebin.com/m44be6e92

[MasterNobody]

Even moar errors?!!?!?! http://pastebin.com/m44be6e92
You forgot the "--cross-prefix=x86_64-pc-mingw32-" option.

[techouse]

I get the same result with and without it. Just tell me how I can properly reconfigure junction, cause I think I f****d it up and that that's causing all the problems.... :/

P.S.: I need the cross option even if I'm on Windoze64?!

[komisar]

techouse
#!/bin/sh

PREF=/mingw

junction -d $PREF/mingw/lib
junction $PREF/mingw/lib $PREF/i686-pc-mingw32/lib

junction -d $PREF/mingw/include
junction $PREF/mingw/include $PREF/i686-pc-mingw32/include

junction -d $PREF/mingw/lib64
junction $PREF/mingw/lib64 $PREF/x86_64-pc-mingw32/lib

junction -d $PREF/mingw/include64
junction $PREF/mingw/include64 $PREF/x86_64-pc-mingw32/include


If you compile to i686-pc-mingw32 -- no use prefix
For compile to x86_64-pc-mingw32 -- use --cross-prefix=x86_64-pc-mingw32-

P.S. Soon I post new version of my mingw without this tricks...

[komisar]

Please, test updated mingw toolchain:Update mingw-w64 to rev.609;
Fix bug in gcc with crtfastmath;
No need "junction" for cross-compile;
All x32 compile with CFLAGS="-O2 -pipe -fomit-frame-pointer -march=pentium2 -mtune=generic -mno-cygwin";
All x64 compile with CFLAGS="-O2 -pipe -fomit-frame-pointer -mtune=generic -mno-cygwin"
Correct detect library e.g. libvfw32, libavifil32


http://komisar.gin.by/tools/20090214_gcc433.mingw.generic.7z

P.S. Who build x264 for x86_64-pc-mingw32 check processing for files > 2 GB. (may incorrect detect usage fseeko64/ftello64)

[bob0r]

Hmm it worked.
With skystrife's help u managed to compile x264bit.
(skystrife also tested the build, thanks!)

However, i still can't fprofile, so i'll just be hosting skystrife's generic builds instead!

@Komisar:
In your package you may also want to include yasm.exe and a 64bit compiled gpac library + include files. Also pthreads if needed.
This way your package only requires a fstab addition and it workks.


f:/MinGW /mingw
f:/mingw.generic /mingw


That seems to work here + i could still use all old gcc and other stuff. (Only yasm.exe had to be copied and possible some more files for other projects, but for x264 it worked nice.)

[Buggle]

Recently I see there are x64 builds appearing on x264.nl (plus the addition of win64 support in the main branch or svn or what you call it) and here on Doom9. Is this because of the great effort by MasterNobody (meaning all stuff written will have to be adjusted by him everytime), or has something fundamentally changed due to those changes in the source code?
It would appear that 64bit is finally really getting off the ground, hopefully MeGUI will follow soon...

[LoRd_MuldeR]

Recently I see there are x64 builds appearing on x264.nl (plus the addition of win64 support in the main branch or svn or what you call it) and here on Doom9. Is this because of the great effort by MasterNobody (meaning all stuff written will have to be adjusted by him everytime), or has something fundamentally changed due to those changes in the source code?
It would appear that 64bit is finally really getting off the ground, hopefully MeGUI will follow soon...

See here:
http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=f43e22a7873ea3811bbc15e30d67681f23249087

It seems the Win64 patch, that had been maintained separately before, was merged into the x264 Git repository...

[komisar]

bob0r, x64 build from x264.nl dont process files > 2G... Need update cross-compile toolchain.

[LoRd_MuldeR]

bob0r, x64 build from x264.nl dont process files > 2G... Need update cross-compile toolchain.

Bobor just mirrors Skystrife's x64 builds, so Skystrife needs to update then...

[kemuri-_9]

bob0r, x64 build from x264.nl dont process files > 2G... Need update cross-compile toolchain.

skystrife uses his own cross-compiled build iirc.

which is it: you can't read >2 GB files or write >2 GB files or both?

[MasterNobody]

skystrife uses his own cross-compiled build iirc.

which is it: you can't read >2 GB files or write >2 GB files or both?
Mostly read (except AVI/AVS input) but may be also write (if it use fseek/ftell). This may happen if 'configure' doesn't define
#define fseek fseeko64
#define ftell ftello64
which may happen if something wrong in compilation chain (for example, if there is no crtfastmath.o in GCC as was in first cross-compilation GCC build by komisar)

[kemuri-_9]

Mostly read (except AVI/AVS input) but may be also write (if it use fseek/ftell). This may happen if 'configure' doesn't define
#define fseek fseeko64
#define ftell ftello64
which may happen if something wrong in compilation chain (for example, if there is no crtfastmath.o in GCC as was in first cross-compilation GCC build by komisar)

if -ffast-math fails from lack of crtfastmath.o,
./configure fails pretty much all the custom configures, so it doesn't configure the ftello64/seeko64 properly.

Edit:
hmm... seemingly the cross-compile is missing crtfastmath.o by default...
oh, i see, found the file, but there's compilation issues:
crtfastmath.c:

#define FE_NONIEEE_ENV 0x0009a04d0270037f
...
__asm__ __volatile__ ("mov.m ar.fpsr=%0" : : "r"(FE_NONIEEE_ENV));



$ gcc crtfastmath.c
Assembler messages:
Error: no such instruction: `mov.m ar.fpsr=%rax'


so how did komisar go about fixing this issue?

[akupenguin]

#define FE_NONIEEE_ENV 0x0009a04d0270037f
...
__asm__ __volatile__ ("mov.m ar.fpsr=%0" : : "r"(FE_NONIEEE_ENV));

You're looking at __ia64_set_fast_math(). That's Itanium, not x86_64.

[kemuri-_9]

ah it seems i was...


$ gcc -D__x86_64__ gcc-snapshot/gcc/config/i386/crtfastmath.c -c -o crtfastmath.o
$ mv crtfastmath.o /prefix/x86_64-pc-mingw32/lib

works fine...
strange at how it doesn't compile and install it automatically with gcc's makefile though...

but that fixes the issue nevertheless...

[komisar]

issue to fix configuration of gcc when his build:
in gcc/config.gcc add in section (for 4.3.3 version)i[34567]86-*-mingw32* | x86_64-*-mingw32*)
tm_file="${tm_file} i386/unix.h i386/bsd.h i386/gas.h dbxcoff.h i386/cygming.h i386/mingw32.h"
xm_file=i386/xm-mingw32.h
tmake_file="${tmake_file} i386/t-cygming i386/t-mingw32"
target_gtfiles="\$(srcdir)/config/i386/winnt.c"
extra_options="${extra_options} i386/cygming.opt"
extra_objs="winnt.o winnt-stubs.o"
cxx_target_objs=winnt-cxx.o
default_use_cxa_atexit=yes
case ${enable_threads} in
"" | yes | win32)
thread_file='win32'
tmake_file="${tmake_file} i386/t-gthr-win32"
;;
esac
case ${target} in
x86_64-*-mingw*)
tmake_file="${tmake_file} i386/t-crtfm"
;;
*)
;;
esac
case ${target} in
*mingw32crt*)
tm_file="${tm_file} i386/crtdll.h"
;;
*mingw32msv* | *mingw32*)
;;
esac
;;
in gcc/config.gcc add in section (for 4.3.4 version)i[34567]86-*-mingw32* | x86_64-*-mingw32*)
tm_file="${tm_file} i386/unix.h i386/bsd.h i386/gas.h dbxcoff.h i386/cygming.h i386/mingw32.h"
xm_file=i386/xm-mingw32.h
tmake_file="i386/t-cygming i386/t-mingw32"
target_gtfiles="\$(srcdir)/config/i386/winnt.c"
extra_options="${extra_options} i386/cygming.opt"
extra_objs="winnt.o winnt-stubs.o"
cxx_target_objs=winnt-cxx.o
default_use_cxa_atexit=yes
case ${enable_threads} in
"" | yes | win32)
thread_file='win32'
tmake_file="${tmake_file} i386/t-gthr-win32"
;;
esac
case ${target} in
x86_64-*-mingw*)
tmake_file="${tmake_file} i386/t-crtfm"
;;
*)
;;
esac
case ${target} in
*mingw32crt*)
tm_file="${tm_file} i386/crtdll.h"
;;
*mingw32msv* | *mingw32*)
;;
esac
;;

P.S. Change full section of configuration.

[Egh]

Mostly read (except AVI/AVS input)

As I recently found out by myself x264 fails to read any AVI over 2GB regardless of the build, it is per design and that fact is conveniently omitted from the documentation :))

[bob0r]

@komisar mind compiling a proper fprofiled x264bit build for us? (gpac/pthreads) :)

[komisar]

Information: for correct building libgcov.a (need for profiling) in gcc configure must use "--prefix=/XXX --with-sysroot=/XXX" for x86_64-pc-mingw32.

P.S. If anybody interest, I publish my last mingw cross-compile toolchain...

[JediNight]

I recently moved to a 64bit OS -- is it worthwhile to use the 64bit build of x264 yet, or should I stick to the 32bit for now? And can it be used with megui at all, or is it CLI only for now?

[deets]

I recently moved to a 64bit OS -- is it worthwhile to use the 64bit build of x264 yet, or should I stick to the 32bit for now? And can it be used with megui at all, or is it CLI only for now?

ripbot264 will use the 64 bit in the recent build.

cant use megui as yet, well not easily anyway.

[kemuri-_9]

x264_x64_1114M.exe (http://kemuri9.net/dev/x264/x264_x64_1114M.exe)

patches:
x264_check_mingw_has_strtok_r.diff (http://kemuri9.net/dev/x264/patches/x264_check_mingw_has_strtok_r.diff)
x264_hrd_pulldown.09_interlace.diff

gcc 4.3.3
+pthreads, gpac, avs

[bob0r]

Information: for correct building libgcov.a (need for profiling) in gcc configure must use "--prefix=/XXX --with-sysroot=/XXX" for x86_64-pc-mingw32.

P.S. If anybody interest, I publish my last mingw cross-compile toolchain...

Always do!

[komisar]

I hope that the more critical bugs fixed...

MinGW cross-compile toolchain:
komisar.gin.by/mingw/ (http://komisar.gin.by/mingw/)

Generic configuration compiled with
"--disable-shared --enable-static" options for configure script

***************************************
:32-bit i686-pc-mingw32
CFLAGS="-march=pentium2 -mtune=generic"
mingwrt-3.15.2-mingw32
w32api-3.13-mingw32
zlib-1.2.3
gmp-4.2.3
mpfr-2.3.2
binutils-cvs (GNU Binutils) 2.19.51.20090220
gcc version 4.3.4 20090220 (prerelease) (x32.generic.Komisar) (GCC)
pthread version svn-2.9.0.0
GPAC version 0.4.6-DEV (build 1) cvs 20090219

:64-bit x86_64-pc-mingw32
CFLAGS="-mtune=generic"
mingw-w64 from svn rev.635
zlib-1.2.3
binutils-cvs (GNU Binutils) 2.19.51.20090220
gcc version 4.3.4 20090220 (prerelease) (x64.generic.Komisar) (GCC)
pthread version svn-2.9.0.0
GPAC version 0.4.6-DEV (build 1) cvs 20090219

[komisar]

For simple self-building cross-compile toolchain I make small scripts.
Grab it from http://komisar.gin.by/mingw/

[frencher]

Hi roozhou,

Have you update of "Namedpipe.exe" ? :thanks:

I have Windows 10 x64, my problem is with large cmd.
My problem is with a large number of file in a home made GUI, "Namedpipe.exe" remains on stand-by from time to time.
".\NamedPipe.exe" "\\.\pipe\pipe_88741.y4m" ".\x264_x64.exe" --crf 28 --preset ultrafast --force-cfr --stdin y4m --output ".\TMP_Video.24" - | ".\mplayer.exe" ".\MOV_0715.mov" -lavdopts threads=11 -vf scale=160:90 -correct-pts -fps 30000/1001 -noframedrop -nofontconfig -vo yuv4mpeg:file="\\.\pipe\pipe_88741.y4m" -ao pcm:fast:file=".\TMP_Audio.wav"

http://i67.tinypic.com/awqlw9.png

Rename attached file to Namedpipe.exe

usage:

Namedpipe pipename exename args

1) pipename must start with \\.\pipe\
2) exename is the program that you want to redirect its stdin
3) args is extra command-line arguments for exename, remember to put at least 2 spaces between exename and args

e.g. pipe mplayer's to neroAacEnc

1) in 1st cmd window, type "Namedpipe \\.\pipe\1.wav neroAacEnc.exe -q 0.3 -ignorelength -if - -of xxx.mp4"
2) in 2nd cmd window, type "mplayer xxx.xxx -vc NULL -vo NULL -ao fast:file=\\.\pipe\1.wav"

Source code(compiled in VC2005)

#pragma comment(linker,"/INCREMENTAL:NO /SECTION:.text,EWR /ALIGN:0x1000")
#include <Windows.h>

#define BUFSIZE 0x400000
char processname[256];
char pipename[64];
char *cmd,*tmp;
HANDLE hPipe/*,hCP,hChildIn*/;
STARTUPINFOA stinfo={0};
PROCESS_INFORMATION pinfo={0};
SECURITY_ATTRIBUTES sa={sizeof(SECURITY_ATTRIBUTES),NULL,TRUE};
void WINAPI Main()
{
DWORD quoteon,state,cmdend;
cmd = GetCommandLineA();
quoteon = 0;
state = 0;
cmdend=0;
tmp = pipename;
while (!cmdend)
{
switch(*cmd)
{
case 0:
cmdend=1;
*tmp=0;
goto END;
case '\"':
quoteon = !quoteon;
break;
case ' ':
case '\t':
if (quoteon)
{
switch(state)
{
case 2:
case 4:
*tmp++ = *cmd;
}
}
else
{
switch(state)
{
case 0:
state++;
break;
case 4:
cmdend = 1;
case 2:
*tmp = 0;
state++;
}
}
break;
default:
switch(state)
{
case 3:
tmp = processname;
case 1:
state++;
case 2:
case 4:
*tmp++ = *cmd;
}
}
cmd ++;
}
END:
if (state<2 || quoteon)
return;
if (INVALID_HANDLE_VALUE == (hPipe = CreateNamedPipeA(pipename,PIPE_ACCESS_DUPLEX,PIPE_TYPE_BYTE | PIPE_READMODE_BYTE | PIPE_WAIT,PIPE_UNLIMITED_INSTANCES,BUFSIZE,BUFSIZE,0,&sa)))
return;
ConnectNamedPipe(hPipe,NULL);
stinfo.cb = sizeof(STARTUPINFO);
stinfo.dwFlags = STARTF_USESTDHANDLES;
stinfo.hStdInput = hPipe;//hChildIn;
stinfo.hStdOutput = GetStdHandle(STD_OUTPUT_HANDLE);
stinfo.hStdError = GetStdHandle(STD_ERROR_HANDLE);
if (CreateProcessA(processname,cmd,0,0,TRUE,0,NULL,NULL,&stinfo,&pinfo))
{

WaitForSingleObject(pinfo.hProcess,INFINITE);
}
}