Current Patches, Where to get them, How they affect speed/output

[MasterNobody]

x264 r1336 32/64 bit compiled with ICC 11.1: download (http://stashbox.org/697692/x264_1336_icc.rar); x264_icc_r1336.diff patch (http://stashbox.org/697652/x264_icc_r1336.diff)
"Qax" builds was compiled with /QaxSSE4.2 option but in reality they can be even slower than normal builds.

[XhmikosR]

Thanks for the patch... and the builds. :)

[Fr4nz]

x264 r1336 32/64 bit compiled with ICC 11.1: download (http://stashbox.org/697692/x264_1336_icc.rar); x264_icc_r1336.diff patch (http://stashbox.org/697652/x264_icc_r1336.diff)
"Qax" builds was compiled with /QaxSSE4.2 option but in reality they can be even slower than normal builds.

Thank you for your ICC builds MNB!

PS: Could you make an x264 build with these two patches?
* x264_win_zone_parse_fix_06.diff
* x264_hrd_pd_interlace.16_r1301.diff

You can get them here: http://forum.doom9.org/showpost.php?p=1342503&postcount=2589
Thanks!

[jpsdr]

Thanks MNB. Is it possible to have a link to the project ?
I personnaly want to try compiling with /Qx option, and not /Qax.

[MasterNobody]

Fr4nz
x264_win_zone_parse_fix_06.diff - not needed
x264_hrd_pd_interlace.16_r1301.diff - don't like it (it is still not stable enough)
So NO.

jpsdr
Link to the patch with all needed (except sources of x264 which you can get from x264 git) is in my previous post

[Fr4nz]

Fr4nz
x264_hrd_pd_interlace.16_r1301.diff - don't like it (it is still not stable enough)
So NO.


Well, actually Jeeb, Techouse and others have been using that patch without problems since many revisions ago...keep in mind, also, that hrd is needed for blu-ray compliancy.

If I'd were you I'd reconsider my NO... :)

[[ReX]]

x264_x64_r1336_unpatched (http://techouse.project357.com/builds/revision1336/x264.exe) | MD5 (http://techouse.project357.com/builds/revision1336/x264.md5)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), unpatched, generic, fprofiled

________________________________________________________________________________

x264_x86_r1336_techouse (http://techouse.project357.com/builds/x264_x86_r1336_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x86_r1336_techouse.txt)
GCC 4.4.2 20091019 (x86.core2.Komisar), fprofiled, -march=core2

x264_x64_r1336_techouse (http://techouse.project357.com/builds/x264_x64_r1336_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x64_r1336_techouse.txt)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), fprofiled, -march=core2

Patches used:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1336048#post1336048)

--version gives me (x86):
x264 0.77.1292M e381f6d
built by techouse on Oct 13 2009, gcc: 4.4.1 (x86.core2.Komisar)

[JEEB]

The nal-hrd patch "seems to work", but is still a hackwork as far as I've heard. I, too, would rather see a really working patch done, but currently it seems like not enough people are interested in it or really are in a desperate need of a nal-hrd patch.

Therefore I can understand MasterNobody's opinion as well.

Anyways, any new benchmarks between GCC'd and ICC'd builds, and also a question -- do I need MSVS to compile with ICC with this patch (as I'm seeing MSVS-project-like files)?

[kemuri-_9]

Anyways, any new benchmarks between GCC'd and ICC'd builds, and also a question -- do I need MSVS to compile with ICC with this patch (as I'm seeing MSVS-project-like files)?

this is for ICL not ICC (the difference being the former is for windows to replace MSVC's cl and the latter being for linux to replace gcc)
iirc, ICL can not even be installed without having MSVC installed first.

and i do think it's unfair for ICL to be supported and not ICC, seeing at how the latter is free for linux platforms (with some standard catches) while the former is only available as a limited trial.

Edit:
I do have to say i'm also heavily against readding bloated msvs sln/projs back to the repo which are only used for icl,
it would be more cleaner to add icc/icl to configure and then port configure to something natively windows supported, i.e. vbscript (a good portion of said porting i have done already)
and either use some windows port of make (msys's or gnuwin32's) or attempting to port the current GNU makefile to nmake (which will also be nontrivial).

[komisar]

Another small speed testing :)
Source: PlanetEarth_952x540_25fps.y4m 1000 frames with fades
"BM" -- MasterNobody ICC 11.1 builds
techouse build from here (http://forum.doom9.org/showthread.php?p=1343413#post1343413)
JEEB buld from here (http://forum.doom9.org/showthread.php?p=1343454#post1343454)

http://komisar.gin.by/test/speed.jpg

P.S. Can anybody make similar benchmark on AMD processor for comparison? Because results can differ there from my.

[[ReX]]

x264 r1336 32/64 bit compiled with ICC 11.1: download (http://stashbox.org/697692/x264_1336_icc.rar); x264_icc_r1336.diff patch (http://stashbox.org/697652/x264_icc_r1336.diff)
"Qax" builds was compiled with /QaxSSE4.2 option but in reality they can be even slower than normal builds.

I keep getting warnings and x264 is not compiled.

Which ICC command line did you use for libx264 and x264?
Maybe I'm missing some include, bin or whatever paths in MSVC? :confused:

[Boolsheet]

Another small speed testing :)
Hm, on my Core i7 920 MasterNobody's ICC builds are equal or slightly faster than the GCC builds from JEEB and techouse. Only did it over 64 bit executables:
x264_64_pf.exe : 28.22 fps
x264_64_pf_Qax.exe : 28.32 fps
x264.1336M.x64.JEEB : 27.30 fps
x264.1336M.x64.techouse: 28.04 fps
And it looks like the Qax optimization helps x264.

Options: -o NUL --preset slower --quiet --tune film matchpoint.y4m
Source was a downsized version (with FFmpeg's bicubic filter to 960x540) of the Match Point trailer (http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/Match%20Point) which is "chock full of fades".

[popper]

The nal-hrd patch "seems to work", but is still a hackwork as far as I've heard. I, too, would rather see a really working patch done, but currently it seems like not enough people are interested in it or really are in a desperate need of a nal-hrd patch.

Therefore I can understand MasterNobody's opinion as well.

Anyways, any new benchmarks between GCC'd and ICC'd builds, and also a question -- do I need MSVS to compile with ICC with this patch (as I'm seeing MSVS-project-like files)?

i wonder why we dont see a current LLVM 2.6
http://llvm.org/releases/download.html#2.6 compile too, is that to slow in operation today compared to the other two.

and has any x264/ffmpeg Dev considering going to their IRC Channel: irc.oftc.net #llvm
and helping patch that for better x264/ffmpeg LLVM speed ?

[JEEB]

x264 r1339 64bit unpatched:
download (http://x264.fushizen.eu/files/revision1339/x264.exe) ; hash (http://x264.fushizen.eu/files/revision1339/x264.md5)

built on Nov 15 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, otherwise defaults

________________________________________________________________________________

x264 r1339 32bit
download (http://x264.fushizen.eu/files/1339/x264.exe) ; release notes

built on Nov 15 2009, gcc: 4.3.4 20090220 (prerelease) (x32.generic.Komisar)
fprofiled, -march=i686


x264 r1339 64bit
download (http://x264.fushizen.eu/files/1339_x64/x264.exe) ; release notes

built on Nov 15 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, -march=core2


patched with:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://www.mediafire.com/download.php?iyhjod2e2dn)


Lazy compiler's lazy builds continue. Bugfixes and the flv output doing a comeback.

[aegisofrime]

Does r1339 fix the freezing issue?

[tph]

Does r1339 fix the freezing issue?
http://git.videolan.org/?p=x264.git;a=commit;h=e76a4adb1c8ee27748847656450c815f60d2fe6d

[schweinsz]

I found that the x264 only has the slice-level thread parallel. How much is the efficiency loss on Rate-distortion performance when 4 slices are used for HD contents generally?

[aegisofrime]

http://git.videolan.org/?p=x264.git;a=commit;h=e76a4adb1c8ee27748847656450c815f60d2fe6d

Thank you very much. Too bad I already started my 8 hour encoding job with 2 hours done. Oh well :/

Also, I have been looking for that page for quite a while, much appreciated :D

[burfadel]

Has anyone tested the speed of the x264.exe over at xvidvideo.ru in comparison to the other builds available? it, along with mplayerc and ffdshow, are apparently always compiled with the latest daily build of GCC 4.5.0

[nm]

I found that the x264 only has the slice-level thread parallel.

Only ancient x264 versions had slice-based multithreading. When frame-based threading model was introduced, support for multiple slices was dropped completely. It was only reintroduced a while ago, without connection to the threading model.

How much is the efficiency loss on Rate-distortion performance when 4 slices are used for HD contents generally?

I think this was discussed when multi-slice support was added back to x264 a few months ago. Try searching.

[LoRd_MuldeR]

I found that the x264 only has the slice-level thread parallel.

Wrong. It uses Frame-parallel encoding for multi-threading. Slices are supported, but not used for parallel encoding.

Some info on efficiency loss cased by multiple slices:
threads.txt (http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob;f=doc/threads.txt)

[techouse]

x264_x64_r1339_unpatched (http://techouse.project357.com/builds/revision1339/x264.exe) | MD5 (http://techouse.project357.com/builds/revision1339/x264.md5)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), unpatched, generic, fprofiled

________________________________________________________________________________

x264_x86_r1339_techouse (http://techouse.project357.com/builds/x264_x86_r1339_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x86_r1339_techouse.txt)
GCC 4.4.2 20091019 (x86.core2.Komisar), fprofiled, -march=core2

x264_x64_r1339_techouse (http://techouse.project357.com/builds/x264_x64_r1339_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x64_r1339_techouse.txt)
GCC 4.4.2 20091019 (x86_64.core2.Komisar), fprofiled, -march=core2

Patches used:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1339048#post1339048)

[JEEB]

x264 r1342 64bit unpatched:
download (http://x264.fushizen.eu/files/revision1342/x264.exe) ; hash (http://x264.fushizen.eu/files/revision1342/x264.md5)

built on Nov 16 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, otherwise defaults

________________________________________________________________________________

x264 r1342 32bit
download (http://x264.fushizen.eu/files/1342/x264.exe) ; release notes

built on Nov 16 2009, gcc: 4.3.4 20090220 (prerelease) (x32.generic.Komisar)
fprofiled, -march=i686


x264 r1342 64bit
download (http://x264.fushizen.eu/files/1342_x64/x264.exe) ; release notes

built on Nov 16 2009, gcc: 4.3.4 20090220 (prerelease) (x64.generic.Komisar)
fprofiled, -march=core2


patched with:

x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)
x264_hrd_pd_interlace.16_r1301.diff (http://www.mediafire.com/download.php?iyhjod2e2dn)


Lazy compiler's lazy builds continue. Weightp bugfixes.

[imk]

r1342M built with ICC.

Windows:
x264-r1342M-imk-win.7z (http://imk.cx/pc/x264/x264-r1342M-imk-win.7z)
win_build_info.txt (http://imk.cx/pc/x264/win_build_info.txt)


I was away for about a month and a half so I didn't have any builds in between then.

x264 has been updated quite a bit since my last build (r1271). Because of all the changes between the two, I think that before using this build for anything serious, run some smaller tests on a sample with both a GCC build and my ICC build and compare the results. I'll run some tests of my own later on. I expect it to match up, but test yourself just to be sure.

[rack04]

x264_x86_r1342M (http://www.mediafire.com/?mylzybygl5z)

Built by rack04 on November 16, 2009, 9:26:16 AM CST
GCC 4.3.4 (x86.core2.Komisar)
--extra-cflags="-march=core2"
make fprofiled

Patched with:

x264_hrd_pd_interlace.16_r1301.diff (http://forum.doom9.org/showthread.php?p=1336048#post1336048)
x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)

[Atak_Snajpera]

Our x264 guru says that GCC 4.x.x sucks but people still use it. Why ???

[Inventive Software]

Because people don't know how to read or know any better?! :confused:

[Dark Shikari]

Our x264 guru says that GCC 4.x.x sucks but people still use it. Why ???Because people eventually learn to ignore my whining and use it because it's faster ;)

[shon3i]

Because people eventually learn to ignore my whining and use it because it's faster ;)

It is faster, over five million... :D

http://www.youtube.com/watch?v=7Ntr-pw_6C0


On mine Athlon Phenom Techouse bulds are bit faster. Can somebody compile ICC for AMD processors?

[juGGaKNot]

ACC you mean ? haha

[MasterNobody]

x264 r1342M 32/64 bit ICC builds by imk (patched for AMD processors with SSE2): download (http://stashbox.org/702304/x264_1342M_icc_%5Bimk%5D.zip)

My own x264 r1342M 32/64 bit ICC builds: download (http://stashbox.org/702303/x264_1342M_icc.zip)

[shon3i]

Thanks MasterNobody, but i can't download your build propertly, download hang after few seconds. Anyway can you patch it with hrd aslo, just for testing.

[skystrife]

Our x264 guru says that GCC 4.x.x sucks but people still use it. Why ???

Unless it has changed recently, mingw64 can only use gcc 4 and above, so that would be why you don't see gcc 3.x, x86_64 builds (for windows).

For x86, lolidunno.

[XhmikosR]

I did a comparison between various ICC builds compiled with different settings (and a GCC build just for reference).

CPU: Intel Q6600 @ 3Ghz
OS: Windows 7 Pro 32bit
Test video: 2000 frames Big Buck Bunny 720p Lagarith
4 two-pass encodes for each build with the following command line:

x264 --preset slow --tune animation -p 1 -B 2500 --stats "stats.stats" --threads auto --thread-input -o NUL "test.avs"

x264 --preset slow --tune animation -p 2 -B 2500 --stats "stats.stats" --threads auto --thread-input -o NUL "test.avs"


x264.x86.r1342M.ICC11-QxSSSE3 -Qx profiled
encoded 2000 frames, 19.29 fps, 2525.29 kb/s
encoded 2000 frames, 19.46 fps, 2525.29 kb/s
encoded 2000 frames, 19.32 fps, 2525.29 kb/s
encoded 2000 frames, 19.46 fps, 2525.29 kb/s
encoded 2000 frames, 14.63 fps, 2502.33 kb/s
encoded 2000 frames, 14.80 fps, 2502.33 kb/s
encoded 2000 frames, 14.59 fps, 2502.33 kb/s
encoded 2000 frames, 14.82 fps, 2502.33 kb/s
average: 19.38/14.71

x264.x86.r1342M.ICC11-QaxSSSE3 -O3 profiled
encoded 2000 frames, 17.33 fps, 2525.29 kb/s
encoded 2000 frames, 17.80 fps, 2525.29 kb/s
encoded 2000 frames, 17.73 fps, 2525.29 kb/s
encoded 2000 frames, 17.77 fps, 2525.29 kb/s
encoded 2000 frames, 14.19 fps, 2502.33 kb/s
encoded 2000 frames, 14.38 fps, 2502.33 kb/s
encoded 2000 frames, 14.23 fps, 2502.33 kb/s
encoded 2000 frames, 14.37 fps, 2502.33 kb/s
average: 17.66/14.30

x264.x86.r1342M.ICC11-imk -O3 /QxSSE2 profiled
encoded 2000 frames, 19.31 fps, 2525.29 kb/s
encoded 2000 frames, 19.48 fps, 2525.29 kb/s
encoded 2000 frames, 19.39 fps, 2525.29 kb/s
encoded 2000 frames, 19.47 fps, 2525.29 kb/s
encoded 2000 frames, 14.56 fps, 2502.33 kb/s
encoded 2000 frames, 14.78 fps, 2502.33 kb/s
encoded 2000 frames, 14.54 fps, 2502.33 kb/s
encoded 2000 frames, 14.79 fps, 2502.33 kb/s
average: 19.41/14.67

x264.x86.r1342M.ICC11-MasterNobody
encoded 2000 frames, 17.54 fps, 2525.29 kb/s
encoded 2000 frames, 17.59 fps, 2525.29 kb/s
encoded 2000 frames, 17.56 fps, 2525.29 kb/s
encoded 2000 frames, 17.65 fps, 2525.29 kb/s
encoded 2000 frames, 14.14 fps, 2502.33 kb/s
encoded 2000 frames, 14.41 fps, 2502.33 kb/s
encoded 2000 frames, 14.18 fps, 2502.33 kb/s
encoded 2000 frames, 14.41 fps, 2502.33 kb/s
average: 17.59/14.29

x264.x86.r1342M.gcc.core2.mssse3 fprofiled
encoded 2000 frames, 17.86 fps, 2525.29 kb/s
encoded 2000 frames, 17.94 fps, 2525.29 kb/s
encoded 2000 frames, 17.95 fps, 2525.29 kb/s
encoded 2000 frames, 17.77 fps, 2525.29 kb/s
encoded 2000 frames, 14.08 fps, 2502.33 kb/s
encoded 2000 frames, 14.36 fps, 2502.33 kb/s
encoded 2000 frames, 14.48 fps, 2502.33 kb/s
encoded 2000 frames, 14.41 fps, 2502.33 kb/s
average: 17.88/14.33

So it seems the only real benefit in speed is when the Qx switch is used, although QxSSE2 and QxSSSE3 are almost the same, at least for my CPU. The disadvantage is that Qx builds won't run on AMD CPUs.
All the builds I used are (http://www.mediafire.com/?sharekey=3f33c77c2cf9ce25ab1eab3e9fa335cab0357954fbd64bc8) here. Can someone with a Core i7 make some tests?

[Trahald]

***snip*** is Alex' current hrd patch if someone wants to give it a try (someone that has access to applications to test the outputs validity.) Streams are accepted by scenarist (although i didnt try a mux)

[shon3i]

Yes i want to try, somebody compile it please :), If this passes can we expect soon to be merged into git?

[JEEB]

Have a build: clicky (http://x264.fushizen.eu/files/x264-20091119-nalhrd.exe)

I wonder why I'm still awake :3

Edit:

It's built with the win zone parsing fix and the nal-hrd patch linked by Trahald just now.
32bit, fprofiled, i686.

[Trahald]

Thanks. I just realized that the mail was private (gmail sorted in the same 'thread' as the mailing list posts), although im sure alex probably wouldnt mind (considering the patch is destined for open source) I removed the link from the other post. (until alex publiclly posts the patch) Anyone interested in testing use JEEBs build.

[Dark Shikari]

Thanks. I just realized that the mail was private (gmail sorted in the same 'thread' as the mailing list posts), although im sure alex probably wouldnt mind (considering the patch is destined for open source) I removed the link from the other post. (until alex publiclly posts the patch) Anyone interested in testing use JEEBs build.He's already posted the patch publicly AFAIK, and he wants it to be public, and the GPL requires it to be public, and... :p

[shon3i]

Ok i do a quick test, and use following cmd

--preset slow --tune film --bitrate 8306 --pass 2 --stats "" --keyint 24 --min-keyint 2 --slices 4 --vbv-maxrate 30000 --vbv-bufsize 30000 --sar 1:1 --level 4.1 --threads 12 --aud --nal-hrd --output fnf2.264 fnf2.avs

Stream is fine, it's passed by Elecard and MUIGenerator, but there is a huge problem, after loading in Elecard stream shows this

Stream info:
file name : I:\fnf2.264
file size : 43 276 602 bytes

video format : AVC/H.264
initial CBP removal relay : 0.76 sec
buffer size : 35 326 976 bits
declared bitrate : 35 326 976 bps
declared frame rate : 23.98 fps
bitrate type : VBR
padding : 0 bits


but i put in my commandline --vbv-maxrate 30000 --vbv-bufsize 30000, so that is not good at all. And Scenarist aslo refuse to import stream with following error Error "Content : Value of Cpb_size is not supported in the BD standard.", which is normal, because buffsize is over 30000, looks like hrd not follow VBV model.

[Underground78]

(until alex publiclly posts the patch)

http://mailman.videolan.org/pipermail/x264-devel/2009-November/006551.html

[jpsdr]

So it seems the only real benefit in speed is when the Qx switch is used, although QxSSE2 and QxSSSE3 are almost the same, at least for my CPU. The disadvantage is that Qx builds won't run on AMD CPUs.
Can someone with a Core i7 make some tests?

Hello.
I'm having an i7, but i would like to test the 2 followings :
- Qx SSE4.2 + O3
- Qx SSE4.2 + O2
If you can provide me these, i can make some tests.

Edit : In fact, if for both so, or if to long, doing only the following :
/O3 /QxSSE4.2 /Qopenmp /Qparallel (or -O3 -xSSE4.2 -openmp -parallel)
and, if possible, also :
/O2 /QxSSE4.2 /Qopenmp /Qparallel (or -O2 -xSSE4.2 -openmp -parallel)

[XhmikosR]

Hello.
I'm having an i7, but i would like to test the 2 followings :
- Qx SSE4.2 + O3
- Qx SSE4.2 + O2
If you can provide me these, i can make some tests.

Edit : In fact, if for both so, or if to long, doing only the following :
/O3 /QxSSE4.2 /Qopenmp /Qparallel (or -O3 -xSSE4.2 -openmp -parallel)
and, if possible, also :
/O2 /QxSSE4.2 /Qopenmp /Qparallel (or -O2 -xSSE4.2 -openmp -parallel)

I cannot fprofile with these CPU instructions since I don't have a CPU which supports them. So I only built a build with -Qx /QxSSE4.2 /Qopenmp /Qparallel but not profiled.

http://www.mediafire.com/?sharekey=3f33c77c2cf9ce25ab1eab3e9fa335cab0357954fbd64bc8

[shon3i]

@techouse. again you are too fast, and not read comments about hrd :) patch is still somehow broken :) http://forum.doom9.org/showthread.php?p=1345340#post1345340

[Trahald]

@techouse
yeah.. I only linked the patch so that a testing build could be made. I would say continue to use the old patch for now.

[moviefan]

I'm just wondering: what is the problem with the old hrd-patch?

[Trahald]

The output is likely fine. Its just that the code needs alot of cleanup. The new patch is fairly clean codewise, it just hasn't been publicly exposed as long so needs some bug killing.

[jpsdr]

I cannot fprofile with these CPU instructions since I don't have a CPU which supports them. So I only built a build with -Qx /QxSSE4.2 /Qopenmp /Qparallel but not profiled.


Ok, i'll make some test this evening, and report my result.
I'll probably doing test with 1000 frame 1080p lagarith
video, comparing SSE4.2 with standard nl build.
Wich optimisation have you used ? O2... O3...?

[shon3i]

The output is likely fine. Its just that the code needs alot of cleanup. The new patch is fairly clean codewise, it just hasn't been publicly exposed as long so needs some bug killing.
Isn't there a some limitation? when interlaced/pulldown is used or something, it think i found some comments by you or some other member, which point on that limitation? or i something overlooked.

[XhmikosR]

Ok, i'll make some test this evening, and report my result.
I'll probably doing test with 1000 frame 1080p lagarith
video, comparing SSE4.2 with standard nl build.
Wich optimisation have you used ? O2... O3...?

I used Qx (Full optimization). I don't know the real difference between this and -O3. So I added a build with -O3 too. The link is the same.