Current Patches, Where to get them, How they affect speed/output

[skystrife]

x264 r1115 (unpatched) (http://www.mediafire.com/?oy5yjtm1ohi) - Alternate Download (http://skystrife.com/x264/revision1115/x264.exe)
gcc 4.3.4 fprofiled build.

-------------------------

x264.1115M.x86.exe (http://www.mediafire.com/?diuz5nlnmzm) - Alternate Download (http://skystrife.com/x264/x264.1115M.x86.exe) / x264.1115M.x64.exe (http://www.mediafire.com/?zmanrjhgjzw) - Alternate Download (http://skystrife.com/x264/x264.1115M.x64.exe)
gcc 3.4.5 fprofiled build with -march=pentium2. / gcc 4.3.4 fprofiled build.

Patches used:

x264_hrd_pulldown.09_interlace.diff
x264_win_zone_parse_fix_05.diff

[squid_80]

Why do the x64 builds produce non-identical output to the x86 builds?

[Dark Shikari]

Why do the x64 builds produce non-identical output to the x86 builds?I'm going to guess miscompilation, since nothing else makes any sense. This doesn't surprise me given the thread earlier about GCC 4.x miscompiling x264 (reverting to GCC 3.x fixed it).

[kemuri-_9]

I'm going to guess miscompilation, since nothing else makes any sense. This doesn't surprise me given the thread earlier about GCC 4.x miscompiling x264 (reverting to GCC 3.x fixed it).

there's also the fpmath instruction difference that was pointed out a while back by BugMaster in the channel.

when AQ was active, there were several setting sets in which the x86 and x64 generate different binary output.

[LoRd_MuldeR]

Is x264 expected to give identical output for several encodes (with same settings and same source) at all ???

I thought it was mentioned before that there are some indeterministics in the code that make the output slightly different each run...

[Sagekilla]

Threads does make x264 -slightly- non deterministic I believe, but in most cases you can run an encode several dozen times and it will always come out the same.

I tried encoding the same source over and over with --threads auto (=3 threads for me) and the video came out bit-for-bit identical each time.

[LoRd_MuldeR]

I just ran three encodes with 6 threads (--threads auto) and I got three different results:

File Size MD5
test1.mkv 9.358.442 27b48fb76fddcf53183f9c2bb551504b
test2.mkv 9.358.953 46c67e8a40b58c6d78684d4e5b962eb0
test3.mkv 9.358.953 9101a2a8d893ab9689278c75b9aed4ac

Of course the settings were the same. The source is uncompressed YUV samples.

---[EDIT]---

Here some more encodes, this time with only one singly thread:

File Size MD5
test4.mkv 9.322.321 b24840bd6b744c92f01cb8c43246d2bc
test5.mkv 9.322.321 d99a654ad7c544e9aa9e283222ecef6c
test6.mkv 9.322.321 eed5a710ba529b901c433d65d168ab88

This time all files have got the same size, at least. But still not bit-identical.

---[EDIT_2]---

Okay, it seems the container has some random data that prevents proper MD5 comparison.

I now extracted the streams:

File Size MD5

test1.264 9.292.324 1708e8e3a66c551bd4182a5df01f299c
test2.264 9.292.835 338aea99dc8381f3e10606c8dd193b96
test3.264 9.292.835 338aea99dc8381f3e10606c8dd193b96

test4.264 9.256.262 88b8dd3f984f7a2b4e1e0aff9589dcdc
test5.264 9.256.262 88b8dd3f984f7a2b4e1e0aff9589dcdc
test6.264 9.256.262 88b8dd3f984f7a2b4e1e0aff9589dcdc

[squid_80]

I tested with --threads 1 and still get non-identical results. Also since --non-deterministic is a CLI option I would assume the default behaviour is deterministic/repeatable, but I guess Dark Shikari will have to confirm that.
(I am using avs2yuv piped to x264, so the source is identical.)

[Dark Shikari]

there's also the fpmath instruction difference that was pointed out a while back by BugMaster in the channel.

when AQ was active, there were several setting sets in which the x86 and x64 generate different binary output.Ah yes, that as well.

Per the above, x264 is supposed to be exactly deterministic when not using threads. If it isn't, in my experience, it's almost always the fault of some nondeterministic decoder/filter (e.g. using AddGrain in FFDshow).

[MasterNobody]

And with threads it is deterministic while you don't use VBV, --nr and --non-deterministic options.

[imk]

x264.r1115M.SSE2.x32.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSE2.x32.imk.exe)
x264.r1115M.SSE2.x32.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSE2.x32.mp4.imk.exe)
x264.r1115M.SSSE3.x32.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSSE3.x32.imk.exe)
x264.r1115M.SSSE3.x32.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSSE3.x32.mp4.imk.exe)

x264.r1115M.SSE2.x64.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSE2.x64.imk.exe)
x264.r1115M.SSE2.x64.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSE2.x64.mp4.imk.exe)
x264.r1115M.SSSE3.x64.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSSE3.x64.imk.exe)
x264.r1115M.SSSE3.x64.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1115M.SSSE3.x64.mp4.imk.exe)

x264.r1115.SSE2.linux.static.x64.imk.lzma (http://imk.cx/pc/x264/x264.r1115.SSE2.linux.static.x64.imk.lzma)
x264.r1115.SSSE3.linux.static.x64.imk.lzma (http://imk.cx/pc/x264/x264.r1115.SSSE3.linux.static.x64.imk.lzma)

I changed my build environment from just using the command prompt to using cygwin. It's a pretty different environment, so issues might arise from this.

The Linux binaries can be extracted with unlzma.

Older builds, build scripts, information, etc. can all be found here (http://imk.cx/pc/x264/).

[techouse]

What's the point of making linux binaries?!

[Dark Shikari]

What's the point of making linux binaries?!Most people don't have ICC?

[techouse]

AFAIK ICC for Linux is free for non-commercial use, but yeah I see your point.

@imk: Why not use gzip or bzip2 instead of lzma?

[imk]

lzma has better compression. :)

[LoRd_MuldeR]

lzma has better compression. :)

7-Zip?

[imk]

lzma is what 7-zip uses. I actually get better compression using lzma directly rather than 7z. I'm guessing there's some kind of overhead to 7z? You can open .lzma inside of 7-Zip, too.

[LoRd_MuldeR]

lzma is what 7-zip uses. I actually get better compression using lzma directly rather than 7z. I'm guessing there's some kind of overhead to 7z? You can open .lzma inside of 7-Zip, too.

I know. In fact 7-Zip is the reference implementation of LZMA. That's why I suggested using 7-Zip.

7-Zip archives are very common these days on all platforms, while "raw" .lzma files are not so much...

[imk]

Well, lzma just has a very nice feel to it on linux, just like bzip2.

lzma -9 file
unlzma file.lzma
tar cf - directory | lzma -9 > file.tar.lzma

;)

[Ranguvar]

Tar v1.20 and up has LZMA support built-in. Also, there's a new 'standard' extension for LZMA tar archives: .tar.xz.

[imk]

Tar v1.20 and up has LZMA support built-in. Also, there's a new 'standard' extension for LZMA tar archives: .tar.xz.

Yay, confusion in formats.

It looks like the lzma-utils git changed their tool name to xz to coincide with the extension and format change.

http://tukaani.org/xz/

It took me a second to find where lzma support was at in tar. It's not listed in the manpage, but the switch is -J.

I don't see any support for xz in tar yet, but there is a patch online to add support for it.

[Guest]

Gents, please stay on topic for the thread.

[gigah72]

is it just me or do others also have problem applying x264_hrd_pulldown.09_interlace.diff to latest revisions?

[LoRd_MuldeR]

is it just me or do others also have problem applying x264_hrd_pulldown.09_interlace.diff to latest revisions?

Not surperising with the major changes in r1117:
http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=3e4946f305317856ed79e0898f25f10859df22ed

Wait for an updated version of the patch...

[Trahald]

must be really recent. 1114 patched ok for me. i'll get on it.

[Trahald]

Ok.. fixed. attached is x264_hrd_pulldown.10_interlace.diff.txt

[imk]

x264.r1120M.SSE2.x32.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSE2.x32.imk.exe)
x264.r1120M.SSE2.x32.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSE2.x32.mp4.imk.exe)
x264.r1120M.SSSE3.x32.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSSE3.x32.imk.exe)
x264.r1120M.SSSE3.x32.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSSE3.x32.mp4.imk.exe)

x264.r1120M.SSE2.x64.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSE2.x64.imk.exe)
x264.r1120M.SSE2.x64.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSE2.x64.mp4.imk.exe)
x264.r1120M.SSSE3.x64.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSSE3.x64.imk.exe)
x264.r1120M.SSSE3.x64.mp4.imk.exe (http://imk.cx/pc/x264/x264.r1120M.SSSE3.x64.mp4.imk.exe)

x264.r1120.SSE2.linux.static.x264.imk.xz (http://imk.cx/pc/x264/x264.r1120.SSE2.linux.static.x264.imk.xz)
x264.r1120.SSSE3.linux.static.x264.imk.xz (http://imk.cx/pc/x264/x264.r1120.SSSE3.linux.static.x264.imk.xz)

Older builds, build scripts, information, etc. can all be found here (http://imk.cx/pc/x264/).


Edit:
Added links to the Linux binaries. These were built with a newer version of ICC (v11.0.081).
Tar v1.22 came out today supporting xz (-J flag). You'll need XZ Utils (http://tukaani.org/xz/) to extract.

[skystrife]

x264 r1120 (unpatched) (http://www.mediafire.com/?ocvwzcdyi0l) - Alternate Download (http://skystrife.com/x264/revision1120/x264.exe)
gcc 4.3.4 fprofiled build.

-------------------------

x264.1120M.x86.exe (http://www.mediafire.com/?bmd2nyxwmth) - Alternate Download (http://skystrife.com/x264/x264.1120M.x86.exe) / x264.1120M.x64.exe (http://www.mediafire.com/?jmmhd3ymr2z) - Alternate Download (http://skystrife.com/x264/x264.1120M.x64.exe)
gcc 3.4.5 fprofiled build with -march=pentium2. / gcc 4.3.4 fprofiled build.

Patches used:

x264_hrd_pulldown.10_interlace.diff
x264_win_zone_parse_fix_05.diff

[ACrowley]

I used the 1120 Build from techhouse

But i get unrecognised option `--nal-hrd' with this build ?

I was working before 1120 ? i changed nothing instead the x264 Build

It failed in 1st pass :
--pass 1 --bitrate 6500 --stats "F:\Tatort.HDTV\Tatort.Der.Tote.Chinese.HDTV.stats" --level 4.1 --keyint 24 --min-keyint 2 --bframes 3 --b-adapt 2 --weightb --direct auto --deblock -1:-1 --subme 2 --partitions none --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 16500 --vbv-maxrate 16500 --qcomp 0.5 --me dia --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output NUL "F:\Tatort.HDTV\Tatort.Der.Tote.Chinese.HDTV.avs" --mvrange 511 --aud --nal-hrd --sar 1:1

I think it has to do with the patched x264_hrd_pulldown.10_interlace.diff ? Has the Commande for the HRD Flag changed ?

EDIT:
mhh it works with x264 rev1120 from skystrifes. Only techhouse x264 1120 Build doenst recognize `--nal-hrd' anymore

[Trahald]

hmm.. i tried techouse's x86 1120 and its fine. didnt try the x64

[ACrowley]

hmm.. i tried techouse's x86 1120 and its fine. didnt try the x64

i use the x68 Build too it doesnt work here
I use megui with the standalone AVCHD Profile (litte bit modified)

EDIT.
jesus im a Idiot!

I used the x64 Build! So i got a error
Works fine ofcourse with x86 Build!
Sorry for this stupid post!

Other Question :
The x264 Builds from skystrife are also fully patched with all new Features as the Builds fromTechhouse are, correct ?
Maybe i should enable autoupdate in megui for x264 instead of manually downloading the techhouse Builds

[Sharktooth]

The x264 Builds from skystrife are also fully patched with all new Features as the Builds fromTechhouse are, correct ?
the skystrife's patched builds are the builds i upload in the megui auto-update server... so, yes, NAL-HRD patch is included as well as zones parse fix.
Maybe i should enable autoupdate in megui for x264 instead of manually downloading the techhouse Builds
that would be wise...

[techouse]

Mea culpa, mea maxima culpa. I forgot to add the hrd patch to my build initially but I fixed it now. Should be working now ;) Everyone that downloaded my r1120 builds prior to 22:00 CET, please re-download.

[video_magic]

Techouse
Your link to the info text at least on the nwgat mirror is wrong. It points to:
http://x264.nwgat.net/x264_x6x264_x64_r1120_techouse.txt

[techouse]

Techouse
Your link to the info text at least on the nwgat mirror is wrong. It points to:
http://x264.nwgat.net/x264_x6x264_x64_r1120_techouse.txt
I'm not the maintainer of that mirror. :(

[techouse]

I've noticed a very funny thing regarding the ICC builds. In the task manager it shows the ICC built x264.exe using 50% of the entire CPU (probably core no. 1) and svchost.exe using another 50% of the CPU (probably core no. 2). Both use the equal amount of RAM, so I guess the process is split. Another funny thing with this is, that x264.exe is run by the USER while svchost.exe is run by the SYSTEM.

As I am used to using GCC builds, where a single x264.exe run by the USER uses the whole CPU and has only 1 RAM pool, this leaves me wondered.

Here's a pic of the taskmanager processes.
http://www.shrani.si/t/3E/XG/3T2e4Utx/x264icc.jpg (http://www.shrani.si/f/3E/XG/3T2e4Utx/x264icc.png)

OS: Windows Vista Business x64 SP1
CPU: Intel Core 2 Duo E6600 @ 2.4 GHz
RAM: 2 GB

[imk]

This seems to be a very awkward thing with your system?

I can run my builds and svchost never raises in CPU usage. x264 will use 100% (using all cores on its own).

Grab Process Explorer (http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx). You can see what inside of the svchost is using CPU.

[techouse]

I used x264.r1120M.SSSE3.x64.mp4.imk.exe, avs2yuv-0.24 and an AVS with avisource(HuffYUV.avi) in that example.

[G_M_C]

I used x264.r1120M.SSSE3.x64.mp4.imk.exe, avs2yuv-0.24 and an AVS with avisource(HuffYUV.avi) in that example.

Avisynth + avs2yuv (one or both on svchost) and x264 on the other core ?

[...]
I can run my builds and svchost never raises in CPU usage. x264 will use 100% (using all cores on its own).
[...]


Works like that for me too on XP/32bits / SP3.

[techouse]

Seems it was some sort of false alarm, cause now everything works just fine (same setup as mentioned above)... The strange thing was, that even though x264.exe was using only 50% of the CPU it had 99% the exact FPS as a GCC built x264.exe.

http://www.shrani.si/t/1t/vQ/4Ymfcwlb/lolwut.jpg (http://www.shrani.si/f/1t/vQ/4Ymfcwlb/lolwut.png)http://www.shrani.si/t/3I/ew/1zxXLZGi/pe.jpg (http://www.shrani.si/f/3I/ew/1zxXLZGi/pe.png)http://www.shrani.si/t/2k/P7/1Vzr7DGt/thread.jpg (http://www.shrani.si/f/2k/P7/1Vzr7DGt/thread.png)

P.S.: The FPS remains the same as to when x264.exe was using only 50% of the CPU and svchost.exe was using the other 50%. Strange....

[squid_80]

Is it just me or is there a typo in the hrd_pulldown patch:
--pulldown <integer> Use 3:2 pulldown
- 32: TBT,BT,BTB,BT pattern
- 64: triple,double *recommended for 720p
Shouldn't it be TBT, BT, BTB, TB ?

[skystrife]

x264 r1123 (unpatched) (http://www.mediafire.com/?t0md1ywdonr) - Alternate Download (http://skystrife.com/x264/revision1123/x264.exe)
gcc 4.3.4 fprofiled build.

-------------------------

x264.1123M.x86.exe (http://www.mediafire.com/?okmit0qnmln) - Alternate Download (http://skystrife.com/x264/x264.1123M.x86.exe) / x264.1123M.x64.exe (http://www.mediafire.com/?qcyyt2tjg2n) - Alternate Download (http://skystrife.com/x264/x264.1123M.x64.exe)
gcc 3.4.5 fprofiled build with -march=pentium2. / gcc 4.3.4 fprofiled build.

Patches used:

x264_hrd_pulldown.10_interlace.diff
x264_win_zone_parse_fix_05.diff

[bob0r]

x264.nl: 08-03-09: x264 64bit is now auto mirrored, using (skystrife's) builds

skystrife's unpatched 64bit builds are now being automatically mirrored via a http2ftp script!

[skystrife]

x264 r1125 (unpatched) (http://www.mediafire.com/?nawjci5wjy2) - Alternate Download (http://skystrife.com/x264/revision1125/x264.exe)
gcc 4.3.4 fprofiled build.

-------------------------

x264.1125M.x86.exe (http://www.mediafire.com/?aigfyu1dtjf) - Alternate Download (http://skystrife.com/x264/x264.1125M.x86.exe) / x264.1125M.x64.exe (http://www.mediafire.com/?jm2nehiymzk) - Alternate Download (http://skystrife.com/x264/x264.1125M.x64.exe)
gcc 3.4.5 fprofiled build with -march=pentium2. / gcc 4.3.4 fprofiled build.

Patches used:

x264_hrd_pulldown.10_interlace.diff
x264_win_zone_parse_fix_05.diff

[techouse]

x264_x86_r1127_techouse (http://techouse.project357.com/builds/x264_x86_r1127_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x86_r1127_techouse.txt)
GCC 4.3.3, fprofiled, -march=core2

x264_x64_r1127_techouse (http://techouse.project357.com/builds/x264_x64_r1127_techouse.7z) | INFO (http://techouse.project357.com/nfo/x264_x64_r1127_techouse.txt)
GCC 4.3.4 20090220 (prerelease) (x64.generic.Komisar), fprofiled, -march=core2

Patches used:

x264_hrd_pulldown.10_interlace.diff
x264_win_zone_parse_fix_05.diff

[d0ORk]

Hi, I tried the 64bit version with my Vista 64bit but I'm always getting: x264 [error]: could not open input file 'x:\my.avs'

I works fine with the 32bit version. What could that be? :scared:

[ajp_anton]

Your avisynth is 32-bit.

[nm]

Install 64-bit AviSynth: http://members.optusnet.com.au/squid_80/

[kemuri-_9]

if you do any processing in that avisynth script that isn't supported by the available list of avisynth x64 plugins, you're gonna need to pipe from avs2yuv.

[skystrife]

x264 r1127 x64 (unpatched) (http://www.mediafire.com/?zgwnmwyknxz) - Alternate Download (http://skystrife.com/x264/revision1127/x264.exe)
gcc 4.3.4 fprofiled build.

-------------------------

x264.1127M.x86.exe (http://www.mediafire.com/?jzkwtyz2lu2) - Alternate Download (http://skystrife.com/x264/x264.1127M.x86.exe) / x264.1127M.x64.exe (http://www.mediafire.com/?htmdtmdmlm2) - Alternate Download (http://skystrife.com/x264/x264.1127M.x64.exe)
gcc 3.4.5 fprofiled build with -march=pentium2. / gcc 4.3.4 fprofiled build.

Patches used:

x264_hrd_pulldown.10_interlace.diff
x264_win_zone_parse_fix_05.diff