x264 development

[celtic_druid]

"extend zones to support (some) encoding parameters in addition to ratecontrol."

[bob0r]

gcc -o x264.exe x264.o matroska.o muxers.o libx264.a -lpthreadGC2 -lwsock32 -lgpac_static -lwinmm -lvfw32 -s -fprofile-generate
libx264.a(ratecontrol.o):ratecontrol.c:(.text+0x4077): undefined reference to `strtok_r'
libx264.a(ratecontrol.o):ratecontrol.c:(.text+0x422d): undefined reference to `strtok_r'
make[1]: *** [x264.exe] Error 1
make[1]: Leaving directory `/home/user/x264'
make: *** [fprofiled] Error 2

Any clue anyone? (i see Cef compiled it)

[Sharktooth]

are you sure you did a correct svn checkout?
i cant check what's going on at the moment, but it seems you could have some old files on your local copy (or maybe a typo in the sources??!? check the changes on the SVN)

[bob0r]

Trax has the same issue, only x264 changed, nothing on my system.

[Sharktooth]

try updating glibc. if it still fails then HAVE_STRTOK_R / NO_STRTOK_R must be implemented ...

[Selur]

"extend zones to support (some) encoding parameters in addition to ratecontrol."
some additional infos would be nice, since --longhelp doesn't show any additional infos :)

[bob0r]

try updating glibc. if it still fails then HAVE_STRTOK_R / NO_STRTOK_R must be implemented ...

Might you help me with telling me what to update?
lib and include files? If so which ones... i dont want to mess up my current system :)

Compiling whines about some port thingy....

[Sharktooth]

the GNU C Library... http://www.gnu.org/software/libc/

[bob0r]

Temp solution found here:
http://forum.doom9.org/showthread.php?p=1024677#post1024677

Note, i will not update x264.nl with a patched version, ill wait for this to be fixed in SVN.

Thanks for the help!

[MetalPhreak]

"extend zones to support (some) encoding parameters in addition to ratecontrol."
some additional infos would be nice, since --longhelp doesn't show any additional infos :)

I'd also like to know what parameters can be used with zones, usage examples?

[akupenguin]

8x8dct, b-bias, b-pyramid, bime, brdo, chrome-me, dct-decimate, deblock, direct, fast-pskip, me, merange, mixed-ref, nr, partitions, ref, scenecut, subme, trellis.

example:
x264 in.avs -o out.264 --crf 20 --deblock -2:-2 --zones 1000,2000,deblock=0:0/2000,3000,nodeblock


Regarding strtok_r: I just remembered there's a reason strtok_r is necessary and strtok doesn't work. It's not just making x264 as a whole re-entrant, but even within one call to zones it depends on strtok_r being re-entrant. So if you have multiple zones and any of them use the new parameters, parsing will fail on windows. I'm not interested in re-implementing strtok (not that it's hard, just I don't care), so it will stay broken until someone submits a patch.

[Selur]

@akupenguin: any plans to extend zones, that one can force an IDR Frame at it's beginning? (sometimes one might want cutpoints at specific frames)

[Cef]

I found a small bug in ratecontrol.c, h->rc->zones[i].param are allocated with regular malloc and freed with x264_free. It probably goes unnoticed on most platforms, but it's actually crashing in my build.

[Trahald]

Regarding strtok_r: I just remembered there's a reason strtok_r is necessary and strtok doesn't work. It's not just making x264 as a whole re-entrant, but even within one call to zones it depends on strtok_r being re-entrant. So if you have multiple zones and any of them use the new parameters, parsing will fail on windows. I'm not interested in re-implementing strtok (not that it's hard, just I don't care), so it will stay broken until someone submits a patch.

im just using
#ifdef _MSC_VER
#define strtok_r strtok_s
#endif

in ratecontrol.h

from what i read the problem is solely on a gnu compile. (i compile in msvc)

[TheRyuu]

I've built x264 using a newer GCC 4.2.1.

The reason I don't like GCC 3.4.x is because when you use -O3, it turns on -frename-registers (or whatever it's called :p). That option is useless (IMO) (but it can be useful on x86_64), not to mention the newer GCC versions are better at optimizations.

You can get it here (rev667b x86). (http://www.esnips.com/doc/27a5b2cc-5b04-4d82-bb7a-ae6a200e1d39/x264_x86_r667b)
(if you want to try it :))

It was compiled with the rev. 667b source code from Cef's build site. I'm finding that this version is faster then the current Cef builds. (if only by a little bit, 34 sec. compared to 36 sec. on a fast 1:30 file with a fast 1 pass encode, probably a greater speed difference would be noticeable on a larger encode, but I'll take any more speed I can get no matter how little on x264).

It was also compiled with march=i686 (for all you people who don't have an sse cpu) and with mtune=pentium3 (which would enable sse for people who have it) which pretty much make the build all around compatible with almost any cpu and allows newer cpu's to take advantage of sse.

Like I said, I just like building things with the newer GCC since I find things faster and sometimes more stable.

I can't see any downsides to using the newer GCC 4.2.1 (thanks to cc979 ;)) I find it more stable, optimizations are better (no -frename-registers), etc..

Thanks and Enjoy, cya :)

P.S.
Cflags were:
-ffast-math -fomit-frame-pointer -march=i686 -mtune=pentium3 -mfpmath=387 -and any other standard flags
additional ldflags:
-Wl,-O1 -Wl,--sort-common

[akupenguin]

The reason I don't like GCC 3.4.x is because when you use -O3, it turns on -frename-registers (or whatever it's called :p). That option is useless (IMO) (but it can be useful on x86_64), not to mention the newer GCC versions are better at optimizations.
Why should you care whether any given option is enabled? The only question is which compiler generates faster code.
Anyway, if it really bothers you, then the obvious solution is -O3 -fno-rename-registers.

It was also compiled with march=i686 (for all you people who don't have an sse cpu) and with mtune=pentium3 (which would enable sse for people who have it) which pretty much make the build all around compatible with almost any cpu and allows newer cpu's to take advantage of sse.
Either the binary contains sse or it doesn't. gcc doesn't generate runtime cpu detection, so any binary that contains sse won't run on pre-sse computers. -march determines what instructions are available, -mtune only determines scheduling.

[TheRyuu]

Why should you care whether any given option is enabled? The only question is which compiler generates faster code.
Anyway, if it really bothers you, then the obvious solution is -O3 -fno-rename-registers.

I care what I compile my programs with, don't you?
I also care because -frename-registers is pretty much useless on anything but x86_64, and it bloats code IMO (would be about the same speed, sometimes there might be a slight gain, but still, IMO, not worth it and could be counter productive too)
It was eventually taken out of -O3 also although thats besides the point. :p

The real purpose of the post wasn't to tell my personal opinions on compiler options though, it was to show that a newer GCC is better at optimizing x264 then the older (mingw 3.4.4) one.

Either the binary contains sse or it doesn't. gcc doesn't generate runtime cpu detection, so any binary that contains sse won't run on pre-sse computers. -march determines what instructions are available, -mtune only determines scheduling.

Thats what I meant I guess. It was build for i686 but is optimized for the pentium3, correct?
That should allow someone with an i686 cpu to run it, but also allow someone with a pentium3 or above to use the extra optimizations, right? (I don't fully understand how -march+-mtune really work together yet I guess)

Just sharing some experience building it thats all. :)

[akupenguin]

It was build for i686 but is optimized for the pentium3, correct?
That should allow someone with an i686 cpu to run it, but also allow someone with a pentium3 or above to use the extra optimizations, right?
Right.
A given instruction takes different numbers of cycles on different cpus. A given computation can be expressed many ways, using different combinations of instructions.
Of all the alternatives that would run on the cpu specified by -march, gcc picks the alternative that's fastest for the cpu specified by -mtune.

Example of how -march can help: -march=pentium4 goes through contortions to avoid register-to-register mov instructions, because mov is really slow on pentium4. So if it needs a value in 2 registers, it loads it twice from memory, whereas most other cpus would load it once and then mov to the other register.

The point I was correcting is: -march=i686 -mtune=pentium3 doesn't use sse because i686 can't run sse, even if sse would be the fastest way to implement something on pentium3.

Nitpicking your cflags: -mfpmath=387 is redundant. Not only is it the default, it's also the only possibility given that sse is disallowed.

[burfadel]

I've built x264 using a newer GCC 4.2.1.

The reason I don't like GCC 3.4.x is because when you use -O3, it turns on -frename-registers (or whatever it's called :p). That option is useless (IMO) (but it can be useful on x86_64), not to mention the newer GCC versions are better at optimizations.

You can get it here (rev667b x86). (http://www.esnips.com/doc/27a5b2cc-5b04-4d82-bb7a-ae6a200e1d39/x264_x86_r667b)
(if you want to try it :))

It was compiled with the rev. 667b source code from Cef's build site. I'm finding that this version is faster then the current Cef builds. (if only by a little bit, 34 sec. compared to 36 sec. on a fast 1:30 file with a fast 1 pass encode, probably a greater speed difference would be noticeable on a larger encode, but I'll take any more speed I can get no matter how little on x264).

It was also compiled with march=i686 (for all you people who don't have an sse cpu) and with mtune=pentium3 (which would enable sse for people who have it) which pretty much make the build all around compatible with almost any cpu and allows newer cpu's to take advantage of sse.

Like I said, I just like building things with the newer GCC since I find things faster and sometimes more stable.

I can't see any downsides to using the newer GCC 4.2.1 (thanks to cc979 ;)) I find it more stable, optimizations are better (no -frename-registers), etc..

Thanks and Enjoy, cya :)

P.S.
Cflags were:
-ffast-math -fomit-frame-pointer -march=i686 -mtune=pentium3 -mfpmath=387 -and any other standard flags
additional ldflags:
-Wl,-O1 -Wl,--sort-common

I tried it, it was slightly slower on my computer by a few percent than the 667b build made by Cef. I have a Core 2 Duo E6600.

I believe Cef optimised the builds for the Core 2 Duo himself, it just goes to show different compiling optimisations work better for different CPU types :)

[TheRyuu]

Right.
A given instruction takes different numbers of cycles on different cpus. A given computation can be expressed many ways, using different combinations of instructions.
Of all the alternatives that would run on the cpu specified by -march, gcc picks the alternative that's fastest for the cpu specified by -mtune.

Example of how -march can help: -march=pentium4 goes through contortions to avoid register-to-register mov instructions, because mov is really slow on pentium4. So if it needs a value in 2 registers, it loads it twice from memory, whereas most other cpus would load it once and then mov to the other register.

The point I was correcting is: -march=i686 -mtune=pentium3 doesn't use sse because i686 can't run sse, even if sse would be the fastest way to implement something on pentium3.

Nitpicking your cflags: -mfpmath=387 is redundant. Not only is it the default, it's also the only possibility given that sse is disallowed.

Thanks.
Adding sse wouldn't really matter anyway since most of it is in asm though right?

Whatever though.
New build here. (http://www.esnips.com/doc/a1d6e08a-0dc8-4baa-8d9b-ee0f3a1cee07/x264_x86_667b_2)

[sampath]

is there a bug in recent vfw build of cefs. It gives error messeage : second pass has more frames than first pass 331>328 .but i didn't change frames at passes?$$

x86 vfw r677 on virtualdub1.6.19

i didn' encounter this on the snow_xmas vfw build

[akupenguin]

Symptom of the fact that vfw is incompatible with B-frames or x264's threads, whichever of those you're using. No, it can't be fixed.

[daWsOn_s]

Hi I'm now trying the new 667 Cef release and I've discovered an acceleration in enconding of about 40% , is that general?

[Terranigma]

Cef, do you plan on updating to rev.671, or do you not find the current crop of changes significant enough?

[morph166955]

the only changes since r665 have to do with the configure script and a few small build options, nothing that should change the binary in any way (theoretically). So what I won't speak for Cef, I doubt hes going to update it since there really is no reason based on the changes that were done.

[Sharktooth]

yep, 667 and 671 bins would be identical. however this is x264 dev thread where ppl should talk about development, not about bugs, not about vfw (that's no longer mantained) and not about builds.
thanks.

[rbt01]

can anybody give me a suggestion, i can't use the long option, such as : --pass 1 --bitrate 2000

[JarrettH]

maybe look at the megui x264 profiles for command line examples

[Vesi]

I have downloaded megui from sourceforge and got all the updates, i want to do xvid rips and x264, so my question is that i have read this thread http://forum.doom9.org/showthread.php?t=89979.
and i'm using HP 3200+ (2,20 GHz) with 1GB DDR memory. do i need those pathes and file which made by sharktooth?

[Sharktooth]

no. x264 and xvid are already included in megui updates.

[Vesi]

no. x264 and xvid are already included in megui updates.

Thanks alot Sharktooth. I really appreciate your work.
@ one thing more what i was thinking that H264 give the highest quality.would it be a good idea to rip with A/R 720*320 for 1CD 650MB with good quality source? thanks

[akupenguin]

Initial review of the instructions to be included in SSE5 (http://developer.amd.com/sse5.jsp). Note that I'm only discussing the functionality, I have no idea of the speeds of the instructions yet. There were some SSSE3 instructions that looked interesting, and reduced the instruction count of some functions, but when run on a Core2 turned out to be much slower than the equivalent multiple SSE2 instructions.

First, discard all the floating-point instructions, because video codecs don't use floating-point.

PCMOV: generally useful anywhere the scalar version of the algorithm has branches, e.g. deblocking.
PCOM*: might save a MOV or a negation over PCMP*, if one wants a predicate other than EQ or LE.
PPERM: zigzag scan in 2 instructions. (finally equivalent to altivec)
PHADD*: useful in all the pixel comparison functions, especially satd. (note that these aren't the same as SSSE3's PHADD*)
PMADC*: SSD and SSIM.
PMAC*: DCT (but not HCT).
PSHLB: I have occasionally wanted this. the per-element shift amount isn't important though, only that there wasn't previously any byte shift instruction.

[SpAwN_gUy]

apparently.. i've found whole bunch of patches at Cef's place on x264.nl...

can anybody give an advice.. which should be applied, and which are not so usefull?
(i'm going to supreme quality ;) ... speed does not matter)

[Audionut]

can anybody give an advice.. which should be applied, and which are not so usefull?
(i'm going to supreme quality ;) ... speed does not matter)


Current Patches, Where to get them, How they affect speed/output
http://forum.doom9.org/showthread.php?t=130364

[LigH]

Excuse my non-developer based question...

A user in the german doom9/Gleitz board spotted news about the evision 683, implementing quant_2x2_dc for Altivec; despite the fact that Altivec is irrellevant for (intel compatible) PC owners, he just wondered why he never saw any "2x2" option in MeGUI...

Could you briefly explain how far the distance is between quant_NxN[_dc] functions and macroblock partitions (with or without DCT)?

[Manao]

In intra macroblock, once the 4x4 DCTs are made, the DC of each of the DCT are retransformed with another DCT. For luma, there are 16 blocks so it's another 4x4 DCT. For chroma, there are only 4 4x4 blocks, so it's a 2x2 DCT. It's done only on DCs coefficients.

[LigH]

This abbreviation is too short to search for, sorry...

DCT = Discrete Cosine Transform;
DC = ?

[Manao]

The DC is the DCT coefficient that represents the average of the untransformed block. It is usually opposed to the ACs, which are the remaining coefficients (those that represent the variations in the block).

If I am not mistaken, the names come from electricity (Alternative Current / Direct Current).

[Guest]

This abbreviation is too short to search for, sorry... Use google and put site:forum.doom9.org after the search term(s). E.g.:

dc coefficients site:forum.doom9.org

@Manao

You are not mistaken.

[LigH]

Wonderful... Indeed, I remember AC/DC, just didn't expect such a similarity. ;) - Now I understand quite well.

BTW, I was quite surprised by the rating of AltiVec in the german Wikipedia article ("ignored and underestimated"). Nice to see that a few developers try to use all means of optimizations, even if only few people may get an advantage.

Fun and success! :cool:

[mroz]

Actually you can get useful info from just searching on dc coefficients (at least with Google), which is odd as I thought it discarded two character words from the search string.

[Adub]

No, just the common ones like "to" or "of".

[Inventive Software]

Feature request that'll get me shot at.... implement interlaced and esa and interlaced and direct=temporal, as per the errors I got when I tried to encode interlaced to interlaced with x264.

[Dark Shikari]

Feature request that'll get me shot at.... implement interlaced and esa and interlaced and direct=temporal, as per the errors I got when I tried to encode interlaced to interlaced with x264.I could easily implement ESA and Interlaced simply by having it swap to the standard (and mindbogglingly slow) ESA algorithm instead of the SEA algorithm that x264 currently uses :)

[Inventive Software]

You could keep the current one for progressive input, and switch it when the user flags interlaced. :) I just thought I'd mention it as it flagged up when I encoded interlaced content.

It's just as well it doesn't work as I'd like it to... interlaced decoding seems to be broken with ffdshow and interlaced x264 encodes. :(

[nm]

It's just as well it doesn't work as I'd like it to... interlaced decoding seems to be broken with ffdshow and interlaced x264 encodes. :(
You'll need to use mod-32 height. That's a limitation in libavcodec's decoder.

[Inventive Software]

Bugger... did wonder what was wrong with it. Thanks.

EDIT: Just done some quick calcs, and I am using mod-32 height... 720x576. You sure that's not mod-32 width as well as height?

[Sagekilla]

Bugger... did wonder what was wrong with it. Thanks.

EDIT: Just done some quick calcs, and I am using mod-32 height... 720x576. You sure that's not mod-32 width as well as height?

720 / 32 = 22.5, it's the width that's the problem.

Edit: Whoa, just realized it's not necessarily mod32. 720 / 16 = 45, works fine.

[Inventive Software]

Sussed that one, and I tested with a binary of ffplay, and it's ffdshow that's the problem.

EDIT: It's also my idiotness that didn't update ffdshow from rev 1620 to rev 1656! Cos that fixed the problem! Doh! :o

[akupenguin]

Just so that anyone reading doesn't get funny ideas: width does not need to be mod32.