Current Patches, Where to get them, How they affect speed/output

[Dark Shikari]

i went through git revisions one by one starting from r958 to find which one caused the problem...

revision 953 caused the problem, meaning from r952 to r953, the problematic code occurs.Huh? That doesn't even make sense... r953? Are you sure you don't mean r963?

Anyways, this seems odd because I ran all of these patches through a huge slew of regression tests, and posted the patches in #x264 IRC, where another few people ran them through all of their own regression tests and confirmed that they worked flawlessly...

[kemuri-_9]

yeah, typo on my part.... (i blame the fact that it is 2am)
cherry picking from 962 to 964 also has no problems.

[Dark Shikari]

Indeed, there was a subtle test case in which it failed (a very rare one which almost never crops up... almost) and thus slipped past our regression tests.

Thanks for the report, fixed in r965.

[Soichiro]

I can confirm that there's an issue in r963. Horrible blocking shows up in the encode, which I've run 3 times now. It's not there in the source, I'm 100% certain, and it's not an avs filter.

Actually, I guess I'm late since it's fixed now. :p

[Ranguvar]

Home (http://sites.google.com/site/ranguvar13/x264-builds)

Direct download (http://sites.google.com/site/ranguvar13/x264-builds/rang_x264_r0965.7z?attredirects=0), Mirrors (http://www.rapidspread.com/file.jsp?id=hwfwpxvna5)

x264 r965 from Git (patched, fprofiled).
Compiled by Ranguvar on September 6th, 2008, with GCC 4.3.2.
DON'T think that because I used march=athlon it restricts the CPUs you can use.
It seems to improve performance (VERY slightly) for all CPUs.

Open this archive with the free, multi-platform tools 7-Zip or p7zip. Compressed with LZMA.
The src folder contains the patched source code.
The bin folder contains a binary executable, and a DLL for those apps that use them
(NOT for AviDemux - get those from LoRd_MuldeR).

Git: git://git.videolan.org/x264.git
Info, and source tarballs: http://www.videolan.org/developers/x264.html
Changelog: http://git.videolan.org/gitweb.cgi?p=x264.git
Vanilla builds: http://x264.nl/
Discussion: http://forum.doom9.org/forumdisplay.php?f=77
http://forum.doom9.org/showthread.php?t=130364


Applied patches (included, unchanged, in the patches folder):

patch -p1 -i ../x264diffs/x264_dll_alignment_fix.01.diff
patch -p1 -i ../x264diffs/x264_progress.indication_r957.diff
patch -p1 -i ../x264diffs/x264_hrd_pulldown.09_interlace.diff
patch -p1 -i ../x264diffs/x264_psy_rdo_0.6_r956.diff
patch -p1 -i ../x264diffs/x264_new_bframe_decision_04.7.diff


CLI used for build: ./configure --enable-shared --extra-cflags="-march=athlon -pipe"
make fprofiled VIDS="../enctests/deadline_cif.y4m"

Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
gtk: no
debug: no
gprof: no
PIC: no
shared: yes
visualize: no

[Comatose]

I can confirm that there's an issue in r963. Horrible blocking shows up in the encode, which I've run 3 times now. It's not there in the source, I'm 100% certain, and it's not an avs filter.

Actually, I guess I'm late since it's fixed now. :p
Yeah, umm, I've had two encodes go bad now =\ (r964)
Completely different source too, one anime and the other a 3D game D:

I shouldn't have downloaded a new compile D:

[Snowknight26]

Ranguvar, would be nice if you included pthreadGC2.dll in your build.

[Ranguvar]

I do. It's merged into the EXE. You don't need a separate one. If you just want it for something else, Google and download.

[techouse]

Ranguvar:
I guess you got GCC-4.3.2 from TDM's website and have overwritten all your old files with the new ones from TDM's zip/tar.gz packages. The problem is that the gcc-4.3.2-tdm-1-core.tar.gz package includes pthreadGC2.dll and pthreadGCE2.dll in its /bin folder, libpthreadGCE2.a, libpthreadGC2-static.a and libpthread.a in its /lib folder and pthread.h, sched.h and semaphore.h in its /include folder. You probably copied over those files to your MinGW /bin, your MinGW /include and your MinGW /lib folder folder.
I suggest you remove all these files, get the latest pthread cvs (cvs -d :pserver:anoncvs@sources.redhat.com:/cvs/pthreads-win32 checkout pthreads), compile (make clean GC-static) and install it (copy libpthreadGC2.a to your MinGW /lib folder and copy pthread.h, sched.h and semaphore.h to your MinGW /iclude folder). That way you will eliminate the need of x264.exe for an extra pthreadGC2.dll in C:\Windows\System32.
Trust me on this one, I had the same problem.

I also suggest you people update your GPAC libraries. Many have reported errors building it from CVS, however I have found a nasty trick that actually works:
1. download the old stable tarball from here http://downloads.sourceforge.net/gpac/gpac-0.4.4-rc2.tar.gz
2. once it is downloaded, extract it to your MSYS /home/username folder and cd into it (i.e. cd /home/username/gpac)
3. while inside the gpac folder get the latest GPAC CVS using the command cvs -z3 -d:pserver:anonymous@gpac.cvs.sourceforge.net:/cvsroot/gpac co -P gpac
4. run ./configure && make lib
5. copy /home/username/gpac/include/gpac to your MinGW /include folder

P.S.: If this doesn't work for you try it in your /home folder (i.e. /home/gpac).

[chriszxl]

now with 964 i get the same nice green with my mpeg2source sample and crf settings??
957 works fine.

EDIT:
coreavc in mpc decodes the broken scene, powerdvd in mpc just skipps over the broken scene...
maybe this does say something to someone?

EDIT2:
964 broken: http://www.mediafire.com/?i9h3tshdlcs
957 ok: http://www.mediafire.com/?1llhbmrsjdy

ya...me too...serious bug...

[gigah72]

ya...me too...serious bug...

965 (http://www.mediafire.com/?luzdgc2f03w) fixed this

[lucassp]

I'm trying to backup a DVD Video with x264 (the build from MeGUI) on Vista x64 but it crashes at the beginning. The avs script is playable in Media Player. Does x264 work with Vista x64?

[techouse]

x264_x86_r965_techouse (http://techouse.project357.com/builds/x264_x86_r965_techouse.7z)
Source: x264 r965 GIT (git://git.videolan.org/x264.git)

Applied patches (current versions):

x264_progress.indication_r957.diff

x264_psy_rdo.0.6_r957.diff

x264_hrd_pulldown.09_interlace.diff

x264_new_bframe_decision_04.7.diff


Please check http://forum.doom9.org/showthread.php?t=130364 and http://git.videolan.org/gitweb.cgi?p=x264.git;a=shortlog for more info

Compiled by techouse on September 6th 2008, 10:50:33 CEST with GCC-4.3.2 on Windows Vista Business SP-1 64-bit.

Commandline used: ./configure --extra-cflags="-march=core2 -pipe" && make fprofiled

Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
gtk: no
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[gav1577]

Indeed, there was a subtle test case in which it failed (a very rare one which almost never crops up... almost) and thus slipped past our regression tests.

Thanks for the report, fixed in r965.

Works perfect now thanks

[saint-francis]

I'm trying to backup a DVD Video with x264 (the build from MeGUI) on Vista x64 but it crashes at the beginning. The avs script is playable in Media Player. Does x264 work with Vista x64?

There is a new version on the update server which works.

[Sharktooth]

sure... i just updated it. damn, i did 3 test encodes with r964 and everything was ok... i was pretty sure that build was working fine...
well, 965 is up... so there should be no more problems.

[lucassp]

There is a new version on the update server which works.

Still no go :(

[Sharktooth]

? did yo update to r965 ?

[Ranguvar]

Ranguvar:
I guess you got GCC-4.3.2 from TDM's website and have overwritten all your old files with the new ones from TDM's zip/tar.gz packages. The problem is that the gcc-4.3.2-tdm-1-core.tar.gz package includes pthreadGC2.dll and pthreadGCE2.dll in its /bin folder, libpthreadGCE2.a, libpthreadGC2-static.a and libpthread.a in its /lib folder and pthread.h, sched.h and semaphore.h in its /include folder. You probably copied over those files to your MinGW /bin, your MinGW /include and your MinGW /lib folder folder.
I suggest you remove all these files, get the latest pthread cvs (cvs -d :pserver:anoncvs@sources.redhat.com:/cvs/pthreads-win32 checkout pthreads), compile (make clean GC-static) and install it (copy libpthreadGC2.a to your MinGW /lib folder and copy pthread.h, sched.h and semaphore.h to your MinGW /iclude folder). That way you will eliminate the need of x264.exe for an extra pthreadGC2.dll in C:\Windows\System32.
Trust me on this one, I had the same problem.

I also suggest you people update your GPAC libraries. Many have reported errors building it from CVS, however I have found a nasty trick that actually works:
1. download the old stable tarball from here http://downloads.sourceforge.net/gpac/gpac-0.4.4-rc2.tar.gz
2. once it is downloaded, extract it to your MSYS /home/username folder and cd into it (i.e. cd /home/username/gpac)
3. while inside the gpac folder get the latest GPAC CVS using the command cvs -z3 -d:pserver:anonymous@gpac.cvs.sourceforge.net:/cvsroot/gpac co -P gpac
4. run ./configure && make lib
5. copy /home/username/gpac/include/gpac to your MinGW /include folder

P.S.: If this doesn't work for you try it in your /home folder (i.e. /home/gpac).

Thanks very much :) The problem was that I somehow got that damn DLL into my PATH without knowing it, so I thought my build was good.

[kemuri-_9]

i don't even bother with 'installing' the gpac/pthread library/include files..
it's easy enough to add
--extra-cflags="-I../pthreads -I../gpac/include" --extra-ldflags="-L../pthreads -L../gpac/bin/gcc"
to x264 configure to tell it where the files are without having to install them
especially useful when building multiple versions
since i do -march athlon-xp and pentium2 builds

[komisar]

I updated my previous message about speed comparsion of GCC here: http://forum.doom9.org/showthread.php?p=1179832#post1179832

[965] My CLI and VFW VAQmod builds found here: http://komisar.gin.by/

[techouse]

I'm trying to backup a DVD Video with x264 (the build from MeGUI) on Vista x64 but it crashes at the beginning. The avs script is playable in Media Player. Does x264 work with Vista x64?
Huh? I've been encoding with x264 on a Vista Business x64 since the dawn of time and have never had any problems with it. Check your filters and AVS.
P.S.: I'm just guessing, but did you crop the file right?

[skystrife]

x264.965.modified.exe (http://www.mediafire.com/?aakqpgnnmaa) - Alternate Download (http://skystrife.com/x264/x264.965.modified.exe)

Patches used:

x264_psy_rdo_0.6_r956.diff
x264_new_bframe_decision_04.7.diff
x264_hrd_pulldown.09_interlace.diff
x264_progress.indication_r957.diff

gcc 3.4.5 fprofiled build with -march=pentium2.

[burfadel]

Huh? I've been encoding with x264 on a Vista Business x64 since the dawn of time and have never had any problems with it. Check your filters and AVS.
P.S.: I'm just guessing, but did you crop the file right?

x264 works fine on both Xp x64 and Vista x64

[DarkZell666]

I'm trying to backup a DVD Video with x264 (the build from MeGUI) on Vista x64 but it crashes at the beginning. The avs script is playable in Media Player. Does x264 work with Vista x64?

Post your script, and maybe open a separate thread for it :)
Make sure it contains a ConvertToYV12() somewhere (preferably at the end), otherwise x264 won't accept it.
If it does, do other builds behave the same way ?

[lucassp]

Post your script, and maybe open a separate thread for it :)
Make sure it contains a ConvertToYV12() somewhere (preferably at the end), otherwise x264 won't accept it.
If it does, do other builds behave the same way ?

DGDecode_mpeg2source("D:\Test\VIDEO_TS\file.d2v",info=3)
ColorMatrix(hints=true,interlaced=true)
#Load_Stdcall_Plugin("C:\Program Files (x86)\Encodings\megui\tools\yadif\yadif.dll")
#Yadif(order=-1)
#crop
#resize
#denoise
Trim(5000,7000)

[Ranguvar]

@komisar: I see you have a LOT of minor patches on your website. Would you mind explaining what they do, in fairly basic terms? Mostly the following:

k.38.cosmetic.diff
999.log_param.diff
x264_32x32samples_crash.r870.diff
k.20.x264_fix_stats_file_work.r877.diff
x264_multithreading_Nth_pass_ratecontrol.r870.diff
bm_x264_thread_pool.r870.diff
k.41.x264_log_file.01k.r928.diff


Thanks!

[kemuri-_9]

@komisar: I see you have a LOT of minor patches on your website. Would you mind explaining what they do, in fairly basic terms? Mostly the following:

k.38.cosmetic.diff
changes x264 signature in crf mode to use 4 digits after the decimal instead of just 1
also adds a long \r line to clear output from progress before printing stats <--- already implemented in r957.

999.log_param.diff
writes the x264 generated signature to the stderr/log file

k.20.x264_fix_stats_file_work.r877.diff
looks to be an overhaul of the stats file writing and includes a padding mechanism for vfw compatibility

bm_x264_thread_pool.r870.diff
implements http://en.wikipedia.org/wiki/Thread_pool_pattern

k.41.x264_log_file.01k.r928.diff
adds the log-file parameter to log output to a file
(imo easier to just use tee)

[Ranguvar]

Thanks, kemuri!

Thread pool pattern looks interesting. I'll try a build with and without it, and compare.

Any ideas on x264_multithreading_Nth_pass_ratecontrol.r870.diff?

[kemuri-_9]

Any ideas on x264_multithreading_Nth_pass_ratecontrol.r870.diff?
looked at it again and it's definitely complexing up the bitrate approximation/ratecontrol so it should be more accurate.
that is, bitrate aiming in multi-threading should be more accurate (though i don't usually have a problem with this in x264 as is)

[MasterNobody]

kemuri-_9
Try to make 2pass encoding of small sample (500 frames is enough) with --threads 128 (128 because more threads more visible is quality degradation of few frames after 30-frame) without and with this patch. Probably than you will see the difference at frames 31 and few others.

[kemuri-_9]

and why would i realistically do that insane number of threads besides seeing at how many resources are being wasted by threads that can't be executed on my processors (being held in wait state)?
and not to mention all the processing time overhead from the thread swapping since the scheduler will probably swap them frequently with that large of a number of them.

[lexor]

You are right, kemuri, most of us can't use that many threads. Still, I do know people who run their encodes on Sun's servers (when they are idling at their workplace), which do run 128 threads per box, so they would care about that.

[Ranguvar]

Oh yeah, and if you could explain x264_32x32samples_crash.r870.diff, forgot about that one. Thanks again.

Doing a few tests now...

[MasterNobody]

and why would i realistically do that insane number of threads besides seeing at how many resources are being wasted by threads that can't be executed on my processors (being held in wait state)?
and not to mention all the processing time overhead from the thread swapping since the scheduler will probably swap them frequently with that large of a number of them.
This is only the example where you can see the difference easily. This bug (incorrect bitrate prediction) also exist with only 2-3 threads but it is not so visible.
Oh yeah, and if you could explain x264_32x32samples_crash.r870.diff, forgot about that one. Thanks again.

Doing a few tests now...
It is simply fix the crash (division by zero) when width or height of the source is 32 (17-32) when you encode with B-frames (and don't use --no-b-adapt).

[Ranguvar]

Thanks.

But this is very annoying... the patches are a mess of dependencies. Still trying to get it working, but jeez.

EDIT: Alright, I admit defeat. Every damn patch wants to bring in its friends, or it'll whine at me. komisar, if you read this, please help give me a way to test/use...

I only want:

x264_dll_alignment_fix.01.diff (no problem with your patches)
999.log_param.diff
x264_progress.indication_r957.diff
x264_32x32samples_crash.r870.diff
x264_multithreading_Nth_pass_ratecontrol.r870.diff (maybe)
bm_x264_thread_pool.r870.diff (maybe)
x264_hrd_pulldown.09_interlace.diff
x264_psy_rdo_0.6_r956.diff
x264_new_bframe_decision_04.7.diff


However, there are tons of dependencies... I really don't want to use the VAQ2mod, or fix_stats_file_work, or your version of the new b-frame decision patch...

Thanks :)

[kemuri-_9]

some of these, like the 32x32sample and Npass ratecontrol patches,
are fixes and I think the developers should look into implementing them into the repository.
(granted they work as intended still)
the logfile patch would be a feature they could add in if they whimmed it.

[Dark Shikari]

Yes, I agree; such fixes should not be used in custom builds but rather should just be committed if useful or not committed if not useful.

[Gabriel_Bouvigne]

looked at it again and it's definitely complexing up the bitrate approximation/ratecontrol so it should be more accurate.
that is, bitrate aiming in multi-threading should be more accurate (though i don't usually have a problem with this in x264 as is)
Could someone please point me at this patch?

[Underground78]

http://komisar.gin.by/x.patch/last.used/x264_multithreading_Nth_pass_ratecontrol.r870.diff

[komisar]

My used patches:
# Write to log encoding settings
999.log_param.diff (http://komisar.gin.by/x.patch/last.used/999.log_param.diff) (wo dep)

# Simply fix the crash (division by zero) when width or height of the source is 32 (17-32) when you encode with B-frames (and don't use --no-b-adapt)
x264_32x32samples_crash.r870.diff (http://komisar.gin.by/x.patch/last.used/x264_32x32samples_crash.r870.diff) (wo dep)

# Fix incorrect Nth-pass ratecontrol work with multithreading
x264_multithreading_Nth_pass_ratecontrol.r870.diff (http://komisar.gin.by/x.patch/last.used/x264_multithreading_Nth_pass_ratecontrol.r870.diff) (wo dep)

# More accurate calculation of i_frame, more correct workaround for VFW Nth-pass ratecontrol
k.20.x264_fix_stats_file_work.r877.diff (http://komisar.gin.by/x.patch/last.used/k.20.x264_fix_stats_file_work.r877.diff) (wo dep)

# Modified thread pool patch
bm_x264_thread_pool.r870.diff (http://komisar.gin.by/x.patch/last.used/bm_x264_thread_pool.r870.diff) (dep: "fix_stats_file_work", "multithreading_Nth_pass_ratecontrol")

# Write encoder outputs to file
k.41.x264_log_file.01k.r928.diff (http://komisar.gin.by/x.patch/last.used/k.41.x264_log_file.01k.r928.diff) (wo dep)

# Change precision of CRF-value in encoder settings
k.38.cosmetic.diff (http://komisar.gin.by/x.patch/last.used/k.38.cosmetic.diff) (wo dep)

# Add some addition task to profiling
999.profiled.01.diff (http://komisar.gin.by/x.patch/last.used/999.profiled.01.diff) (wo dep)

# Workaround of incorrect profiling of VFW (gcc 4.4)
k.vfw.01.dummy-log.diff (http://komisar.gin.by/x.patch/last.used/k.vfw.01.dummy-log.diff) (wo dep)

# Helper for make profiled build of VFW
k.vfw.02.profile.diff (http://komisar.gin.by/x.patch/last.used/k.vfw.02.profile.diff) (wo dep)

# Add ability to call external gui for VFW
k.vfw.03.external_config.01.diff (http://komisar.gin.by/x.patch/last.used/k.vfw.03.external_config.01.diff) (wo dep)
Ranguvar
x264_psy_rdo_0.6_r956.diff
x264_new_bframe_decision_04.7.diff
x264_hrd_pulldown.09_interlace.diff
Some as posted in this thread. I only make this compatible with my patched build.

P.S. I add prefix "k.XX...." because this patches (in original) not clear apply and I make them more compatible. Or I make this patch by myself.

P.P.S. Original version of some patches "by MasterNobody/BugMaster" found here: [x264-devel] BugMaster's patches (fix various problems) (http://mailman.videolan.org/pipermail/x264-devel/2008-January/003921.html)

And mirror of MasterNobody/BugMaster patches (next message) found here: http://komisar.gin.by/x.patch/BugMaster/20080908/

[MasterNobody]

Here are updated versions of my patches (which I use to build x264vfw): http://stashbox.org/208351/bm_x264_patch_collection.r965.zip
In this archive you will find independent versions of some patches (for example, x264_thread_pool.r965.diff)

List of patches in collection:
01_x264_32x32samples_crash.r965.diff
01_x264_cosmetic.r965.diff
01_x264_debug_defines.r965.diff
01_x264_fix_stats_file_work.r965.diff
01_x264_multithreading_bug_check.r965.diff
01_x264_multithreading_Nth_pass_ratecontrol.r965.diff
02_bm_x264_error_memoryleaks.02.r965.diff
03_bm_x264_thread_pool.r965.diff
04_bm_x264_vaqmod.01.r965.diff
05_bm_x264_psy_rdo_0.6.r965.diff
06_bm_x264_new_bframes_decision.04.7.r965.diff

Independent\
x264_32x32samples_crash.r965.diff
x264_cosmetic.r965.diff
x264_debug_defines.r965.diff
x264_error_memoryleaks.02.r965.diff
x264_fix_stats_file_work.r965.diff
x264_multithreading_bug_check.r965.diff
x264_multithreading_Nth_pass_ratecontrol.r965.diff
x264_thread_pool.r965.diff
x264_vaqmod.01.r965.diff

[Gabriel_Bouvigne]

The 32x32 fix seems really strange, especially the "pmb = -imb;" line. Did you intended to have a negative number of P MB ?

[Ranguvar]

Thanks very much, BugMaster - that may be what I'm looking for. I'll test ASAP.

Will all your updated patches be here always? http://komisar.gin.by/x.patch/BugMaster/

[stanjr]

I am compiling/encoding on a 64-bit Ubuntu quad-core machine.

Using x264_new_bframe_decision_04.7.diff causes the following error:
libx264.c: In function 'X264_init':
libx264.c: 165: error: X264_param_t has no member named 'b_bframe_adaptive'
(after compiling x264 and then when compiling mplayer to use x264 with mencoder)

Have I done something wrong?

[recover]

I just wanted to let you know that there is an updated version of the progress indication patch here (http://forum.doom9.org/showthread.php?p=1117683#post1117683).
The difference is only that the code is now enclosed in #ifdef _WIN32 so that it can compile on other OS.
I guess you have to fix this updated version the same way you've done with the previous version as well.

[Sharktooth]

it's useless for him since he's on ubuntu

[stanjr]

What about the 'b_bframe_adaptive' error?

[Sharktooth]

was the patching successfull?

[stanjr]

Yeah, patching x264 was successful and it compiled fine. It was only when I then got to compiling mplayer that I got the error. If you need more info, I can try to get what you need when I get home later.