Compile for MinGW32 with Cygwin

[zerowalker]

I want to build x264 without having to rely on that damn cygwin_1.dll , and from what i understand i need to change how it compiles, so it doesn't compile the "cygwin way".

I couldn't get MinGW32 Working at all, so i am stuck using Cygwin.

So, can i do this, from what have read it seems like changing
--host solves this, but for me the "make" will fail at the end if i use x86_64-w64-mingw32.

So pretty much stuck, have searched all over the place, but it seems to be a complete jungle.


./configure --host=x86_64-w64-mingw32

collect2: error: ld returned 1 exit status
Makefile:170: recipe for target 'x264.exe' failed
make: *** [x264.exe] Error 1

[LoRd_MuldeR]

Not sure whether Cygwin can create binaries that do not depend on "cygwin_1.dll", but why not simply use MinGW to make a "native" Windows binary, like everybody else does?

Setting up MinGW is not rocket science. And comping x264 on Win32 (with MinGW) has been discussed here so many times before...

:search:

[zerowalker]

I have tried to get MinGW to work.
I think i got the 32bit version to work. The 64bit version is a jungle.

I have spent hours, and got nowhere, even the "all in one" installation things isn't useful.
May be doing it totally wrong, i would hope so at least.

[LoRd_MuldeR]

Well, have you tried following this guide, for example?
http://doom10.org/index.php?topic=26.0

And if you have a specific problem with MinGW, what exactly is it? :confused:

Simply unzipping Komisar's GCC 4.8.2 (http://komisar.gin.by/mingw/index.html) (multilib) package, mounting it in the MSYS shell and then running this:

git clone git://git.videolan.org/x264.git
cd x264
./configure --host=x86_64-pc-mingw32
make

...creates a working 64-Bit build for me ;)

(BTW: I am using this (http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/msys%2B7za%2Bwget%2Bsvn%2Bgit%2Bmercurial%2Bcvs-rev13.7z/download) MSYS package, provided by the MinGW-Builds project)

[zerowalker]

Getting the MinGW-w64 to work.

But according to that guide, which i have actually followed before (though didn't get which msys, i mean the one stated is very old, and the other's are just sources).
I should be able to compile to 64bit with the 32bit version?

Then what is MinGW-w64?

[LoRd_MuldeR]

MinGW-w64 is a fork of the original MinGW.org project.

Initially, MinGW-w64 was forked with the goal of adding 64-Bit support (which MinGW didn't have at that time), but that doesn't mean that 64-Bit support is its only purpose. Neither do you need a 64-Bit compiler to create 64-Bit binaries.

Actually, nowadays most people seem to prefer MinGW-w64 over MinGW.org. For example, official Qt binaries for Windows (32-Bit and 64-Bit) are made with MinGW-w64...

See also:
http://qt-project.org/wiki/MinGW

[zerowalker]

Ah i see.

Well i couldn't get MinGW-w64 to work.
It had this nice installer.

It installed all fine, but i couldn't use it, i didn't know if i was supposed to manually add Msys or something later.
And running the Bash just return an Echo.

[LoRd_MuldeR]

Again: All you need is the MSYS shell (e.g. from here (http://sourceforge.net/projects/mingwbuilds/files/external-binary-packages/msys%2B7za%2Bwget%2Bsvn%2Bgit%2Bmercurial%2Bcvs-rev13.7z/download)) and a suitable build of MinGW-w64, e.g. the Komisar's GCC 4.8.2 package from his site (http://komisar.gin.by/mingw/index.html). You don't need any installer.

Just unzip the MSYS package to, for example, "C:\Sandbox\MSYS", and the MinGW package to, for example, "C:\Sandbox\MinGW". Then open the MSYS shell, mount your MinGW path to /mingw and run the commands that I posted above (https://forum.doom9.org/showpost.php?p=1687794&postcount=4).

umount /mingw <- just to be sure
mount c:/sandbox/mingw /mingw
gcc -v <- should print the GCC version now

[zerowalker]

Okay it seems to work now, just trying to figure out how to link "libwinpthread-1.dll".

[LoRd_MuldeR]

Okay it seems to work now, just trying to figure out how to link "libwinpthread-1.dll".

Why do you want to link "libwinpthread" when it is known to have problems with x264, while "pthread-w32" has no such issue? And why link it as a DLL?

Also, if you are using Komisar's GCC package, then it already includes the "pthread-w32" library - and x264 will pick it up automatically!

Simply run "./configure" from the x264 source directory and you should see "thread: posix", which means that x264 has found the PThreads library and is going to link against it.

(And since the included pthread-w32 library was compiled as a static library, no separate DLL file will be needed at runtime)

[zerowalker]

It asks for it, else i can't start x264.

And yes it does say Posix.

Not that good at this stuff, so pretty confused.

Mind i am doing an 64bit build, if that has anything to do with it (host target configure as the guide you linked stated).

[LoRd_MuldeR]

If your x264 binary asks for a "libwinpthread-1.dll" at runtime, then your obviously have built x264 with libwinpthreads - the shared (DLL) variant.

You probably don't want libwinpthreads, because it has problems with x264. If you want POSIX threading, use the pthread-w32 library instead.

As said before, if you compile with Komisar's GCC 4.8.2 package, it already includes pthread-w32. And it includes a static variant, so no DLL needed.

[zerowalker]

I am compiling with Komisar, and it says "POSIX" on the Threading.

$ ./configure --cross-prefix=x86_64-w64-mingw32- --host=x86_64-pc-mingw32
platform: X86_64
system: WINDOWS
cli: yes
libx264: internal
shared: no
static: no
asm: yes
interlaced: yes
avs: avisynth
lavf: no
ffms: no
mp4: no
gpl: yes
thread: posix
opencl: yes
filters: crop select_every
debug: no
gprof: no
strip: no
PIC: no
bit depth: 8
chroma format: all

You can run 'make' or 'make fprofiled' now.

[Groucho2004]

Add these:
--enable-static
--enable-strip

Also, I personally use "--enable-win32thread" because it is slightly faster than pthreads (on my Intel i5).

[zerowalker]

Nice works!

How important is the fprofiled thing?
Couldn't really find out much about it.

[Groucho2004]

How important is the fprofiled thing?
I tried it with several different clips but never noticed any speed improvement worth mentioning. Other's mileage may vary.

I did however notice a slight improvement with this switch (probably not recommended for a AMD CPU):
--extra-cflags="-march=corei7"

Most of the critical code in libx264 is highly optimized ASM, not much one can improve with profiling and/or compiler switches.

[zerowalker]

I see, i find it weird to begin with that you can actually profile a code like that, for me it doesn't make sense.
But then again i don't understand how it even works.

Interesting about the march settings, actually searched on thet before.
I got an i5, so i guess i should do tat, but with corei5? (i5 760)?

EDIT:

Oh wait, you had i5 as well, and noticed "corei5" doesn't exist.

Won't there be compatibility issues?

[Groucho2004]

Won't there be compatibility issues?It either works or not.

[zerowalker]

Ah okay.

Well will do that, and an fprofile on a yuv444p video (not sure if that's supported though).

Then i should be good to go:)

[LoRd_MuldeR]

I see, i find it weird to begin with that you can actually profile a code like that, for me it doesn't make sense.
But then again i don't understand how it even works.

Without profiling, the compiler can only do static analysis. But with profiling it can actually analyse the code while running. Basically, the profiler determines how much CPU time is spent in each function or code block, based on a real-world input. This way we can find the "hot" functions (code blocks) where most of the CPU time is spent and the "cold" functions (code blocks) that are rarely ever called. Of course it then makes sense to turn on more aggressive optimizations in the "hot" code, while optimization of the "cold" code can be reduced to a minimum. Anyway, IMO, with x264, fprofile isn't worth the extra time it takes...

[zerowalker]

Ah, that makes sense. No idea how it works but i grasp it at least.

Kinda failed though with my fprofiling as i have been using my Disk which has probably made the encoding slower for that time, shouldn't matter that much if anything though as it's not even finished,
probably shouldn't have given it a 1:35min 1080p file.


But then well, even if it's not worth the time, no idea in stopping now, and well, i can at least take pride in that i have done it later;P

[LoRd_MuldeR]

Oh wait, you had i5 as well, and noticed "corei5" doesn't exist.

Won't there be compatibility issues?

What the "-march" option controls is:

1. It chooses the database of how many CPU cycles a specific instructions takes on the specific CPU model - which the compiler will use to make its decisions. Choosing a different CPU model than what you will actually be running on is sub-optimal, but the code will still run perfectly fine.

2. It chooses which instructions (MMX, SSE, AVX, etc) are allowed to be used. This can obviously have a big impact on which CPU's the resulting binary will run. If, for example, you compile for some CPU that has AVX but then try to run on a CPU without AVX support, you are going to get crashes.


Having said that, "-march=corei7" is simply an alias for the first Core-i generation, so whether you have a Core-i7 or Core-i5 shouldn't matter. Actually, in newer GCC, this has been replaced by "-march=sandybridge", "-march=ivybrdige", "-march=haswell" and so on ;)


No idea how it works but i grasp it at least.

Profiling works by inserting "instrumentation" code.

Put simply, it adds this to every function:
void someFunction(void)
{
_counter_someFunction++;
[...]
}

Of course "instrumentation" code is only added to the build used for profiling, not the final build.

[zerowalker]

Ah, then i was correct in one way.

When you say newer GCC, is that the one i am using (komisar), or is it even newer (guess it is, as they should have removed corei7).

But well, good that it's a generic Core-I then, as i don't have AVX for example.


EDIT:

Also been wondering, what is the "Avisynth Support" in x264 i saw during Configure?
Does x264 support Avisynth natively now?

[LoRd_MuldeR]

When you say newer GCC, is that the one i am using (komisar), or is it even newer (guess it is, as they should have removed corei7).

GCC 4.8.2:
https://gcc.gnu.org/onlinedocs/gcc-4.8.2/gcc/i386-and-x86-64-Options.html#i386-and-x86-64-Options

GCC 4.9.0:
https://gcc.gnu.org/onlinedocs/gcc-4.9.0/gcc/i386-and-x86-64-Options.html#i386-and-x86-64-Options

[Groucho2004]

Actually, in newer GCC, this has been replaced by "-march=sandybridge", "-march=ivybrdige", "-march=haswell" and so on ;)
Ha, haven't looked at the documentation in ages. I'll give "-march=sandybridge" a go. :)

[LoRd_MuldeR]

Also been wondering, what is the "Avisynth Support" in x264 i saw during Configure?
Does x264 support Avisynth natively now?

Of course it does (on Windows). It has been like that since somewhere in 2005 :p

(But of course a 64-Bit x264 can only work with 64-Bit Avisynth, and a 32-Bit x264 only with 32-Bit Avisynth)

[zerowalker]

But wait, what happened to avs4x264pipe?

[LoRd_MuldeR]

But wait, what happened to avs4x264pipe?

What should have happened to it?

Piping Avisynth input into x264 from a separate program (e.g. avs4x264pipe) still can make sense. For example, if you wish to use 32-Bit Avisynth with 64-Bit x264 or the other way around. Or if you simply wish to run x264 and Avisynth in separate address spaces. Or if you are on Linux and want to use a native Linux x264 with Avisynth running as a (separate) Wine process. Or...

[zerowalker]

But if i feed .avs, do i need to tell it the size and everything?
As with avs4x264pipe that's automated.

Didn't know it was that useful though, quite some handshaking abilities.

[LoRd_MuldeR]

But if i feed .avs, do i need to tell it the size and everything?

No, you don't. x264 has proper Avisynth input support. You can simply open .avs file, like in any other Avisynth-enabled application.

It has been like that since I came in touch with x264 for the first time ;)

[zerowalker]

Doesn't work for me.

Seems to be Avisource not working though.

avs [info]: trying FFmpegSource2... not found
avs [info]: trying DSS2... not found
avs [info]: trying DirectShowSource... not found
avs [error]: unable to find source filter to open `ss.mkv'
raw [error]: raw input requires a resolution.
x264 [error]: could not open input file `ss.mkv' via any method!

EDIT 2:

Okay ignore that -_-


Had messed up, first i outputed to the .avs so it became 0 byte file, then i changed position (output -> input), and then back again,
and didn't even read the last output until i posted it here.

I got it working now;P

[zerowalker]

How can i enable libswscale?
I downloaded the libpack, and should be pointing to it, but it doesn't seem to work.

$ ./configure --cross-prefix=x86_64-w64-mingw32- --host=x86_64-pc-mingw32 --enable-static --enable-strip --enable-win32thread --extra-cflags="-march=corei7"
--extra-cflags="-I/libpack/i686-w64-mingw32/include" --extra-ldflags="-L/libpack/i686-w64-mingw32/lib"

"C:\msys\1.0" seems to be the Home directory so put them there.

I get a lot of "ms-dos path" errors for Cygwin though (i am using Msys).


I also put the libraries/includes in the Mingw32 paths for both 32/64bit as i thought they would be on automatically then.

[nhakobian]

x264 has proper Avisynth input support. You can simply open .avs file, like in any other Avisynth-enabled application.

It has been like that since I came in touch with x264 for the first time ;)

(adding an exception to LoRd_MuldeR's comment above)

...unless you want to pipe from 32-bit Avisynth to 64-bit x264 (since not all Avisynth plugins have 64-bit support).

[LoRd_MuldeR]

(adding an exception to LoRd_MuldeR's comment above)

...unless you want to pipe from 32-bit Avisynth to 64-bit x264 (since not all Avisynth plugins have 64-bit support).

Yeah, as mentioned here:
https://forum.doom9.org/showpost.php?p=1687871&postcount=28



How can i enable libswscale?
I downloaded the libpack, and should be pointing to it, but it doesn't seem to work.

Provided you have checked out x264 source to "C:\Sandbox\x264" and that the libpack was extracted to "C:\Sandbox\libpack", simply open MSYS, change path to x264 source directory and enter:

./configure --host=x86_64-pc-mingw32 --enable-static --enable-strip --extra-cflags="-I../libpack/x86_64-w64-mingw32/include" --extra-ldflags="-L../libpack/x86_64-w64-mingw32/lib"

And if anything is not recognized as it is supposed to be, look at your "config.log" to see what exactly went wrong! Since making wild guesses of what might have went wrong can be quite unrewarding ;)


I get a lot of "ms-dos path" errors for Cygwin though (i am using Msys).

How can you get "errors for Cygwin" when you are using MSYS/MinGW now? :confused:

Sounds like your build environment is severely messed up! If so, I would highly suggest to make a clean re-install of everything. Again, only MSYS + MinGW is needed to build x264. Well, those plus YASM.

(Especially make sure you don't have any Cygwin stuff in your PATH before launching the MSYS shell!)

[foxyshadis]

If you have Win7 Pro, you have XP Mode available. Otherwise, I'm sure you have an old OEM copy of XP lying around somewhere from an old PC. VPC or Virtualbox + a VM = development goodness.

Mulder has years of experience keeping that dev system up to date, but setting it up and reconfiguring it always makes me hate life, so I keep both a Linux playground and cross-compiling box and an XP games and MSYS box. Keeps all of the tweaking separate from my main system while still being easy to run, and I can revert or start from scratch if I totally screw things up -- I'm quite good at that, you see. It's easier to test lots of things cleanly, or even have lots of snapshots in various states until you get one working fully.

[nhakobian]

...so I keep both a Linux playground and cross-compiling box...

I've used Linux machines to cross-compile for Windows and in most cases it works very well. The only times I've had issues is where the build system uses custom programs for code generation/parsing.

In these cases the cross-compiling tool tries to compile the custom code generator as a windows program, but then needs to execute it in a Linux box to generate the source for further compilation. I remember that compiling Python in particular had this issue (but they were working on a fix to their build scripts; not sure if it ever happened).

[zerowalker]

How can you get "errors for Cygwin" when you are using MSYS/MinGW now?

Sounds like your build environment is severely messed up! If so, I would highly suggest to make a clean re-install of everything. Again, only MSYS + MinGW is needed to build x264. Well, those plus YASM.

Will do that, removing Cygwin seems aa pain though, not sure if i did it corretly, but hopefully.

So redoing everything.

Noticed that komisar has made a new build, should i use that?
I am guessing that's it's incompatible with the libpack though?

EDIT:

Okay seems to be going good with the normal GCC (the newer was problematic, couldn't even compile with it for some reason, probably did something wrong).

Libpack is working as well (lsmash is too old though?).

I can also use Posix instead of win32-thread, though not actually sure which i should use. I am guessing win32 as it seems to have better performance (at least on i5).