Current Patches, Where to get them, How they affect speed/output

[JEEB]

x264 r1318 64bit unpatched:
download (http://jeeb.fiveforty.jp/x264/revision1318/x264.exe) ; hash (http://jeeb.fiveforty.jp/x264/revision1318/x264.md5)

built on Oct 30 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, otherwise defaults

________________________________________________________________________________

x264 r1318 32bit
download (http://jeeb.fiveforty.jp/x264/1318/x264.exe) ; release notes (http://jeeb.fiveforty.jp/x264/1318/relnotes.txt)

built on Oct 30 2009, gcc: 4.3.4 20090220 (prerelease) (x32.generic.Komisar)
fprofiled, -march=i686


x264 r1318 64bit
download (http://jeeb.fiveforty.jp/x264/1318_x64/x264.exe) ; release notes (http://jeeb.fiveforty.jp/x264/1318_x64/relnotes.txt)

built on Oct 30 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, -march=core2


patched with:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://www.mediafire.com/download.php?iyhjod2e2dn)

[Fr4nz]

Where are techouse and imk? I want them... :\

[juGGaKNot]

Where are techouse and imk? I want them... :\

Patch not enough ? :)

or do you want 4.4.2 ?

[Fr4nz]

Patch not enough ? :)

or do you want 4.4.2 ?

Well, I was interested in ICC x264 compiled versions...

PS: Imk has been missing for 3 weeks in this forum...I'm starting to worry... :\
Anyone has infos about him?

[aegisofrime]

imk's builds have never worked for me :( Does it have anything to do with the fact that I'm using the Phenom II?

[XhmikosR]

I don't know why they don't work for you, but they always worked for me (Intel Q6600). And speed improvement wasn't bad. :)

[LoRd_MuldeR]

imk's builds have never worked for me :( Does it have anything to do with the fact that I'm using the Phenom II?

By default ICL 11 uses "/arch:SSE2", so you need a SSE2-capable CPU at least. That should be okay for the Phenom II though.

But if the "/Qx" option is used (maybe imk used that switch), ICL generates a binary that will run on Intel CPU's only. It will simply abort on Non-Intel CPU's :rolleyes:

However with "/arch:IA32 /QaxSSE2" it should generate a binary that will run on any x86 CPU, but still use a SSE2 optimized path on Intel CPU's.

[roozhou]

So far I have not seen any ICL x264 build running faster than GCC 4 or MSVC build. I tested on Core2, P4E and K8, all slower than GCC and MSVC build.

IMO from /arch:SSE2 programs only benefits on running SSE2 float point instructions instead of FPU instructions.

[LoRd_MuldeR]

Well, you could also use ""/arch:IA32 /QaxSSE2,SSE4.1" to generate three code paths (generic x86, SSE2-optimized and SSE4.1-optimized).

Not sure if x264 does benefit form that though, because I did not test it yet ;)

[aegisofrime]

Thanks for the reply Lord Mulder. I had to search the thread to find Imk's last post.

The relevant portion of his build info is here:

Patches used:
x264_icc_09_win.diff
x264_win_zone_parse_fix_06.diff

Common build arguments for icc:
-O3 -Qprec-div- -Qstd=c99 -I. -I.. -I./resources -I../common
-I../encoder -I../extras -I./pth -DNDEBUG -DHAVE_MMX
-DAVIS_INPUT -DHAVE_PTHREAD -DPTW32_STATIC_LIB -DUSE_REAL_PTHREAD
-D_MBCS -EHsc -MT -W0 -nologo -c

Specific build arguments for icc:
-QxSSE2
-Qprof_gen -Qprof_dir. (for generating profiling information)
-Qipo -Qprof_use -Qprof_dir. (for using the profiling information)
-DARCH_X86 (for 32-bit build)
-DARCH_X86_64 (for 64-bit build)

Additional flags used for icc:
-D_LIB (when building libx264.lib)
-D_CONSOLE -DWIN32 (when building x264.exe)
-DMP4_OUTPUT
-DIMF (if built with IMF)

Looks like /qx is used?

@roozhou: I have seen before a build comparison page, the location of which escapes me now, that shows Imk's builds consistently being a few FPS faster than gcc builds. Which in the course of a 2 hr video can be quite substantial I guess.

[Fr4nz]

@roozhou: I have seen before a build comparison page, the location of which escapes me now, that shows Imk's builds consistently being a few FPS faster than gcc builds. Which in the course of a 2 hr video can be quite substantial I guess.

Well, IIRC the speedup reported by Imk was in order of 2-3% ...minimal, but better than nothing :D

[XhmikosR]

I had done a little benchmark (http://forum.doom9.org/showthread.php?p=1246872#post1246872).

EDIT: imk has also posted a few benchmarks (http://spreadsheets.google.com/pub?key=pbffjdC6iUPWs2HtYHwZ2VQ&hl=sv).

[LoRd_MuldeR]

Thanks for the reply Lord Mulder. I had to search the thread to find Imk's last post.

The relevant portion of his build info is here:

Specific build arguments for icc:
-QxSSE2
-Qprof_gen -Qprof_dir. (for generating profiling information)
-Qipo -Qprof_use -Qprof_dir. (for using the profiling information)
-DARCH_X86 (for 32-bit build)
-DARCH_X86_64 (for 64-bit build)

Looks like /qx is used?

Yes. That build will run on SSE2-capable Intel CPU's only!

I would recommend imk to use "/arch:IA32 /QaxSSE2" or at least "/arch:SSE2 /QaxSSE2" to make builds that are optimized specifically for Intel CPU's and still run on Non-Intel's.

More info: http://cache-www.intel.com/cd/00/00/22/23/222300_222300.pdf

[Fr4nz]

Yes. That build will run on SSE2-capable Intel CPU's only!

I would recommend imk to use "/arch:IA32 /QaxSSE2" or at least "/arch:SSE2 /QaxSSE2" to make builds that are optimized for Intel CPU's and still run on Non-Intel's.

Unfortunately Imk seems to be missing...is here anyone able to make ICC builds, like the imk ones?

[JEEB]

The last I saw the Intel C/C++ compilers for Windows

A) weren't free even for personal use
B) had some bugs around here and there in binaries (if that was just due to some compiler options, I do not know)

[LoRd_MuldeR]

Unfortunately Imk seems to be missing...is here anyone able to make ICC builds, like the imk ones?

The problem is that the MSVC project files for x264 have been removed. And don't know to compile x264 with ICC without a MSVC project file ;)

Nor do I currently have the time to find it out. Compiling with GCC is easy, because you can use the original configure/make files.

The last I saw the Intel C/C++ compilers for Windows

* A) weren't free even for personal use

You can get a 30 days evaluation copy for free. That's not the problem at all.

[JEEB]

You can get a 30 days evaluation copy for free. That's not the problem at all.

Oh yes, you can get an evaluation copy indeed. But that would really restrict the time area I could build stuff in ;).

And yeah, the MSVC project files got removed :3

[LoRd_MuldeR]

Oh yes, you can get an evaluation copy indeed. But that would really restrict the time area I could build stuff in ;).

You can request a new evaluation copy after 30 days. Not really nice, but at least a way to use ICC. If it really produces faster code, it's worth the effort, I think ;)

And yeah, the MSVC project files got removed :3

So how do people build x264 with ICC? From the commandline or from MSVC? With a fully custom build script / project file? Or even with a modified x264 make file?

If somebody can put me to the right direction, I probably could manage to make a build...

[XhmikosR]

@LM: Download the latest (http://imk.cx/pc/x264/x264-r1271M-imk-win.7z) available ICL build from imk and see the readme. The process is well described although I did tried in the past to build an ICL build of x264 but didn't succeed.

EDIT: BTW if you figure out how to compile an ICL build, share your knowledge. :)

[LoRd_MuldeR]

@LM: Download the latest (http://imk.cx/pc/x264/x264-r1271M-imk-win.7z) available ICL build from imk and see the readme. The process is well described although I did tried in the past to build an ICL build of x264 but didn't succeed.

EDIT: BTW if you figure out how to compile an ICL build, share your knowledge. :)

It seems x264 sources need to be patched for ICL. But the provided patch fails with current x264.

Until somebody updates the ICL patch for up-to-date x264, I give up ;)

[XhmikosR]

I know, I was hoping that you were gonna do it. :p

[roozhou]

@roozhou: I have seen before a build comparison page, the location of which escapes me now, that shows Imk's builds consistently being a few FPS faster than gcc builds. Which in the course of a 2 hr video can be quite substantial I guess.

A few FPS faster, are you joking? Even doing real-time encoding, a few FPS faster means more than 10% speed boost!!

If both built with profiling, the performance difference should always be less than 1%.

[brunogm]

Just a question, if it is possible to use the LLVM 2.6/ (http://lwn.net/Articles/358646/) and if someone already tested?

[boxXx]

One tiny little question, i updated my x264 to the newest build but now when i want to run my Process it says :

[Error] An error occurred: x264 [error]: could not open input file 'J:\rescue\1.avs'

Any idea ? i tried some older versions but it wont work at all...

[bob0r]

Open 'J:\rescue\1.avs' with VirtualDub. And see what error it gives.

[boxXx]

does it just fine, its when i start the process, when i open it in Vdub there's no problem.

[J_Darnley]

Do you have 64-bit x264? Do you have 64-bit avisynth?
If you said yes to the first, you need to say second. Or pipe it with something like avs2yuv

[Audionut]

Weightp just got commited. :)

[juGGaKNot]

Indeed, am 2 bugs fixed on it.

but

All weightp modes may cause minor breakage in non-compliant decoders that take shortcuts in deblocking reference frame checks.
"Smart" may cause serious breakage in non-compliant decoders that take shortcuts in handling of duplicate reference frames.

What decoders ?

[Astrophizz]

CoreAVC for one, as has already been stated. Fix should be in 2.0.

[juGGaKNot]

About that i know but divx one ?

[jpsdr]

It seems x264 sources need to be patched for ICL. But the provided patch fails with current x264.

Until somebody updates the ICL patch for up-to-date x264, I give up ;)

Does compile with Intel compiler realy worth it ?
I mean, is all the code of x264 written in C (or C++),
or are the critical parts written in assembler ?
In the last case, compile with Intel compiler
is not realy worthwile.

[jethro]

Does compile with Intel compiler realy worth it ?
I mean, is all the code of x264 written in C (or C++),
or are the critical parts written in assembler ?
In the last case, compile with Intel compiler
is not realy worthwile.

Maybe this is true, but it is faster in real encodings.

I had done a little benchmark (http://forum.doom9.org/showthread.php?p=1246872#post1246872).

EDIT: imk has also posted a few benchmarks (http://spreadsheets.google.com/pub?key=pbffjdC6iUPWs2HtYHwZ2VQ&hl=sv).

[DarkZell666]

Does compile with Intel compiler realy worth it ?
I mean, is all the code of x264 written in C (or C++),
or are the critical parts written in assembler ?
In the last case, compile with Intel compiler
is not realy worthwile.

The critical parts are written in assembly language indeed, so the benefit from using compiler X vs compiler Y (be it icc, msvc, gcc or anything else) won't be more than 1%, give or take.
However, 1% is still better than nothing ;)

[roozhou]

The critical parts are written in assembly language indeed, so the benefit from using compiler X vs compiler Y (be it icc, msvc, gcc or anything else) won't be more than 1%, give or take.
However, 1% is still better than nothing ;)

The fact is benefit of compiling with profiling is ~3% using gcc and ~5% using msvc. Since the asm part benefit little from profiling, apparently C language part of x264 does take a significant part in performance.

[Dark Shikari]

The fact is benefit of compiling with profiling is ~3% using gcc and ~5% using msvc. Since the asm part benefit little from profiling, apparently C language part of x264 does take a significant part in performance.On a Core i7, 40% of time is spent in C functions, roughly.

[rack04]

x264_x86_r1330M (http://www.mediafire.com/?znexi3tmrzm)

Built by rack04 on November 9, 2009, 9:17:38 AM CST
gcc.exe (GCC) 4.3.4 (x86.core2.Komisar)
--extra-cflags="-march=core2"
make fprofiled

Patched with:

x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1336048#post1336048)
x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)

[aegisofrime]

Did anyone else find R1330 slower? Usually on my TGMC encodes I get from 10 to 11 FPS, but now I'm running at 7.8 FPS. Is it because of Weighted-P?

[JEEB]

x264 r1332 64bit unpatched:
download (http://jeeb.fiveforty.jp/x264/revision1332/x264.exe) ; hash (http://jeeb.fiveforty.jp/x264/revision1332/x264.md5)

built on Nov 10 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, otherwise defaults

________________________________________________________________________________

x264 r1332 32bit
download (http://jeeb.fiveforty.jp/x264/1332/x264.exe) ; release notes (http://jeeb.fiveforty.jp/x264/1332/relnotes.txt)

built on Nov 10 2009, gcc: 4.3.4 20090220 (prerelease) (x32.generic.Komisar)
fprofiled, -march=i686


x264 r1332 64bit
download (http://jeeb.fiveforty.jp/x264/1332_x64/x264.exe) ; release notes (http://jeeb.fiveforty.jp/x264/1332_x64/relnotes.txt)

built on Nov 10 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, -march=core2


patched with:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://www.mediafire.com/download.php?iyhjod2e2dn)


Okies, doesn't seem to be miscompiled (tested with the patched 64bit build). Although if you do find something, please do mention.

[wyti]

Yes weightedp slow things down, you can desactivate it and see if it help

[aegisofrime]

Yes weightedp slow things down, you can desactivate it and see if it help

Is the quality worth the performance tradeoff? Hope this question is okay, since the thread is about patches and how they affect output afterall. :thanks:

[microchip8]

Is the quality worth the performance tradeoff? Hope this question is okay, since the thread is about patches and how they affect output afterall. :thanks:

It helps a lot in fades, esp with mbtree on

[juGGaKNot]

If you have a lot of fades, for my source it is.

[wyti]

Weighted P is only used on fade detection for now, may weighted P be useful for something else (and on the future have a weightedp 3 who would increase quality in a lot of case ) or is weightedp only useful on fades ?

[LoRd_MuldeR]

I have noticed it is used also in sequences that contain no fades. So it seems to do 'something' even on non-fade sequences.

Well, maybe those sequences are "fades" in Weight-P's definition, but I wouldn't have classified those as fades.

Anyway, I think we can assume that Weight-P may even help in places where we (as humans) wouldn't have expected fades...

[microchip8]

there was a weightp 3 option, which was k-means, but it was dropped for some reasons and you'll have to ask the devs why. Note that dropped in this case does not mean that it'll never hit x264 at some point in the future. There were some issues with it and it was decided (IIRC) to work on it after weightp was committed so in the future the devs may very well add it at some point

[rack04]

x264_x86_r1332M (http://www.mediafire.com/?1dkzyxz0jyo)

Built by rack04 on November 10, 2009, TIME 9:04:01 CST
GCC 4.3.4 (x86.core2.Komisar)
--extra-cflags="-march=core2"
make fprofiled

Patched with:

x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1336048#post1336048)
x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)

[techouse]

x264_x64_r1332_unpatched (http://techouse.project357.com/builds/revision1332/x264.exe) | MD5 (http://techouse.project357.com/builds/revision1332/x264.md5)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), unpatched, generic, fprofiled

________________________________________________________________________________

x264_x86_r1332_techouse (http://techouse.project357.com/builds/x264_x86_r1332_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x86_r1332_techouse.txt)
GCC 4.4.2 20091019 (x86.core2.Komisar), fprofiled, -march=core2

x264_x64_r1332_techouse (http://techouse.project357.com/builds/x264_x64_r1332_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x64_r1332_techouse.txt)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), fprofiled, -march=core2

Patches used:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1336048#post1336048)


P.S.: The reason why i haven't built any new revisions in the past 2 weeks is that my PSU died and I had to wait 2 damn weeks for a new one. Yea...shit happens...

[techouse]

x264_x64_r1336_unpatched (http://techouse.project357.com/builds/revision1336/x264.exe) | MD5 (http://techouse.project357.com/builds/revision1336/x264.md5)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), unpatched, generic, fprofiled

________________________________________________________________________________

x264_x86_r1336_techouse (http://techouse.project357.com/builds/x264_x86_r1336_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x86_r1336_techouse.txt)
GCC 4.4.2 20091019 (x86.core2.Komisar), fprofiled, -march=core2

x264_x64_r1336_techouse (http://techouse.project357.com/builds/x264_x64_r1336_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x64_r1336_techouse.txt)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), fprofiled, -march=core2

Patches used:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1336048#post1336048)

[JEEB]

x264 r1336 64bit unpatched:
download (http://x264.fushizen.eu/files/revision1336/x264.exe) ; hash (http://x264.fushizen.eu/files/revision1336/x264.md5)

built on Nov 10 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, otherwise defaults

________________________________________________________________________________

x264 r1336 32bit
download (http://x264.fushizen.eu/files/1336/x264.exe) ; release notes

built on Nov 10 2009, gcc: 4.3.4 20090220 (prerelease) (x32.generic.Komisar)
fprofiled, -march=i686


x264 r1336 64bit
download (http://x264.fushizen.eu/files/1336_x64/x264.exe) ; release notes

built on Nov 10 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, -march=core2


patched with:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://www.mediafire.com/download.php?iyhjod2e2dn)


Lazy compiler posts lazy builds :3 Switched from the .jp server to the fushizen domain as the .jp one was acting weirdly (and because it will be taken offline in March or so).