x264 development

[Chengbin]

Can we get a "roadmap" of where x264 development is heading? The last 1 year(ish) had been almost all performance improvements. Right now it seems like it is adding features. Can we get a very general idea of what x264's development is going, like Intel's CPU roadmap?

[nakTT]

Can we get a "roadmap" of where x264 development is heading? The last 1 year(ish) had been almost all performance improvements. Right now it seems like it is adding features. Can we get a very general idea of what x264's development is going, like Intel's CPU roadmap?
I agree with you. Even a general idea would be helpful to end user like me.

:thanks:

[PlazzTT]

There was such a chart a few years (less?) ago. With info on which features are being developed, how difficult they are to program, etc. An updated version would be interesting.

[G_M_C]

Can we get a "roadmap" of where x264 development is heading? The last 1 year(ish) had been almost all performance improvements. Right now it seems like it is adding features. Can we get a very general idea of what x264's development is going, like Intel's CPU roadmap?

Performance imrovements ? Maybe, i've seen many quality-improvements too; --Subme x, --aqmode x, --mbtree etc. And now the lookahead VBV.

My feeling is that there are more quality improvements, and that the speedimprovements weren't the intended goal, but are to the credit of the patch writers :)

[Chengbin]

Performance imrovements ? Maybe, i've seen many quality-improvements too; --Subme x, --aqmode x, --mbtree etc. And now the lookahead VBV.

My feeling is that there are more quality improvements, and that the speedimprovements weren't the intended goal, but are to the credit of the patch writers :)

I meant "last year" as before subme, aqmode, mbtree.

[burfadel]

With 'Direct Mode' and CRF, I realise that for the most part x264 will use 'Spatial' when in CRF mode since typically information was not forward reaching. With the rc-lookahead, is it possible that 'Direct Mode' can make use of 'Temporal' in 'Auto' mode, since I believe the required information from future frames with the inclusion of rc-lookahead is now available?

[akupenguin]

Unrelated to rc-lookahead. The information needed to decide between spatial and temporal has always been available if you're doing any kind of ratecontrol or frametype decision, at least if lowres would be good enough. It's still not implemented.

[Revgen]

when will the x264 in megui update be updated? it is v1183 at the moment, 1210 is the latest version on x264.nl

Kurtnoise is now handling MeGUI updates. You can find a link to his server in the MeGUI thread.

[me7]

Any news on weightp, that is supposed to help fades with MBTree?

[microchip8]

Any news on weightp, that is supposed to help fades with MBTree?

weightp will be committed when it's GSOC author is done - within days or so. Stop asking

[Zachs]

weightp progress here - http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog;h=refs/heads/gsoc

[LoRd_MuldeR]

weightp progress here - http://repo.or.cz/w/x264/x264-p-frames.git?a=shortlog;h=refs/heads/gsoc

How can I check this out? It always fails via HTTP and doesn't work at all via GIT protocol.

http://pastie.org/593644

(I know that I can download the snapshot as ZIP file, but then the configure script will complain that it's not a valid GIT repository)

[ACoolie]

You're using the wrong urls. s/w/r/ in the HTTP url: http://repo.or.cz/r/x264/x264-p-frames.git
and Git: git://repo.or.cz/x264/x264-p-frames.git

[LoRd_MuldeR]

You're using the wrong urls. s/w/r/ in the HTTP url: http://repo.or.cz/r/x264/x264-p-frames.git
and Git: git://repo.or.cz/x264/x264-p-frames.git

Ah :thanks:

[LoRd_MuldeR]

I was getting a crash (reproducible!) with libx264 r1232 on a specific sample clip, using the following settings:

cabac=1 ref=8 deblock=1:-1:-1 analyse=0x3:0x133 me=umh subme=10 psy=1 psy_rd=1.00:0.15 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 chroma_qp_offset=-3 threads=6 nr=0 decimate=1 mbaff=0 bframes=5 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 wpredb=1 keyint=500 keyint_min=25 scenecut=40 rc_lookahead=60 rc=crf mbtree=1 crf=21.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00

The backtrace goes like this:
*********** EXCEPTION **************
Registers:
EAX: 00000012 EBX: 075F1750 ECX: 00000000 EDX: 00000001 ESI: 00000000
EDI: 0022A984 ESP: 002180B0 EBP: 00000002 EIP: 04C22F07 EFlags: 00010246

Exception Code: EXCEPTION_ACCESS_VIOLATION (C0000005)
Exception Flags: 00000000

Origin:
D:\Avidemux 2.5\libx264-72.dll(x264_quant_4x4_trellis+0x72F07) [0x04C22F07]
*********** EXCEPTION **************

*********** BACKTRACE **************
Frame 0: D:\Avidemux 2.5\libADM_core.dll(Z16exceptionHandlerP17_EXCEPTION_RECORDPvP8_CONTEXTS1_+0x530E) [0x6510530E]
Frame 1: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EC4A) [0x7D61EC4A]
Frame 2: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EC1B) [0x7D61EC1B]
Frame 3: C:\WINDOWS\system32\ntdll.dll(KiUserExceptionDispatcher+0x1EA56) [0x7D61EA56]
*********** BACKTRACE **************

However I am unable to re-produce the problem with a debug build of libx264! Also x264.exe did not crash on the sample. Any ideas ??? :eek:

(BTW: There's no such problem with my build of libx264 r1222, so it must be something between r1222 and r1232)

[IgorC]

x264 supports ARM now. :eek:

It's important news considering that now ARM is expanding from embedded/mobile systems to laptop sector. Also ARM is extremely
power efficient that makes its systems runs without power supply during many hours or even days while Intel can't.

I'm working with some basic application based on ARM7 (maybe Cortex in 1 or 2 years) as part of student project. In future I'm considering to move to some multimedia approach.
It's great to count with x264 and CoreAVC 2.0 supporting ARM.

[LoRd_MuldeR]

Okay, I was able to track down my crash (http://forum.doom9.org/showpost.php?p=1318401&postcount=765) to the "-march=core2" compiler option!

With that option set, the resulting libx264 will crash. Tried both builds of GCC 4.4.1 , Komisar's and TDM's. Both with the same result:

Crash with "-march=core2" and no crash without it. Also "-march=pentium2" does not trigger the crash. However my previous builds (r1222 and older) compiled with "-march=core2" are working fine :confused:

I suspect something between r1222 and r1232 did cause GCC/MinGW to break x264 with "-march=core2" for some reason. Any ideas ???

[juGGaKNot]

the 1222 march core2 also crashed for 2-3 people using my bat, some did not crash but the fps was double on some frames ( faster, music lasted longer )

and youtube converter did not work on them ( before worked )

-march=i686 all problems solved.

[LoRd_MuldeR]

Ha! It's getting more curious:

There's no crash with GCC 4.3.3 and there also is no crash with GCC 4.4.1 after I re-added the Graphite Loop Transformations (-floop-interchange -floop-strip-mine -floop-block).

Damn, I just removed the Graphite optimizations on suggestion (http://forum.doom9.org/showpost.php?p=1317722&postcount=36) of Komisar :D


Also this warning just appeared on the TDM web-site:

The 4.4.1-tdm-1 release is known to have a bug which causes drastically increased CPU usage in programs compiled with it. You are urged to use a previous release until this bug is fixed.

:eek:

[juGGaKNot]

i guess 4.3.3 for life haha

aa forgot about it, 1222 ( i think ) also crashed when i set bframes to 0 than to 3

try this out.

[lexor]

Seeing the new commits for the GSoC ARM patch makes me curious. Why does x264 need ARM optimizations? Why would you run encoder on an ARM device, I see why you would want to play back on it, but why encode? I just don't see a use case.

[Dark Shikari]

Seeing the new commits for the GSoC ARM patch makes me curious. Why does x264 need ARM optimizations? Why would you run encoder on an ARM device, I see why you would want to play back on it, but why encode? I just don't see a use case.Video recording? Video calls/conferencing? Personal video broadcast (a'la Justin.tv)?

See the blog post (http://x264dev.multimedia.cx/?p=142).

[LoRd_MuldeR]

Seeing the new commits for the GSoC ARM patch makes me curious. Why does x264 need ARM optimizations? Why would you run encoder on an ARM device, I see why you would want to play back on it, but why encode? I just don't see a use case.

As far as I know, we will see (Sub)Notebooks and/or Netbooks with ARM architecture on the market very soon (if they aren't available yet).

Given that you own such a device and want to encode a video with x264, but there's no "heavyweight" x86 machine available at your location, then you'll be very happy for ARM optimizations!

So I see no reason to not have those optimizations, as long as somebody is skilled and willing to do it. They certainly won't hurt :)

[CruNcher]

Think about being able to use x264 to save your tv programm with a itchy small device that fits in your pocket and you can cary it to wherever you want no big boxes standing anywhere anymore :) (a dream) and with a pico projector integrated you can watch it on every wall in reach (though that will become IP problematic because you not allowed to show the content in public most of the time hehe)

[rack04]

Ha! It's getting more curious:

There's no crash with GCC 4.3.3 and there also is no crash with GCC 4.4.1 after I re-added the Graphite Loop Transformations (-floop-interchange -floop-strip-mine -floop-block).

Damn, I just removed the Graphite optimizations on suggestion (http://forum.doom9.org/showpost.php?p=1317722&postcount=36) of Komisar :D


Also this warning just appeared on the TDM web-site:

The 4.4.1-tdm-1 release is known to have a bug which causes drastically increased CPU usage in programs compiled with it. You are urged to use a previous release until this bug is fixed.

:eek:

Does it crash with this build?

x264 core:72 r1232M x86

Download (http://www.megaupload.com/?d=RF89ZJL8)

Built by rack04 on August 25, 2009, 12:52:28 PM CST
$ gcc --version
gcc.exe (GCC) 4.3.4 20090526 (prerelease) (x86.core2.Komisar)
$ ./configure --extra-cflags="-march=core2"
Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no
$ make fprofiled VIDS="bigbuckbunny.avs LosslessTouhou.avs riverbed.1920x1080.yuv"

Patched with:

x264_hrd_pd_interlace.16.fix.1232.diff (http://pastebin.org/11898)
x264_win_zone_parse_fix_06.diff (http://kemuri9.net/dev/x264/patches/x264_win_zone_parse_fix_06.diff)

[DeeGee]

Yes, quite a lot of new hardware is coming/has already came with ARM Cortex A8/A9 based processors. I myself have been eyeing the rumored Nokia N900 (RX-51/Rover) as my next tech toy/phone :)

[lexor]

I'm not convinced that's a use case. Why would you choose to encode on that as consumer. And we can't install x264 (or anything else for that matter) on video cameras, so again I don't see a point.

[Dark Shikari]

I'm not convinced that's a use case. Why would you choose to encode on that as consumer. And we can't install x264 (or anything else for that matter) on video cameras, so again I don't see a point.Whoever said that this patch was for the consumer?

[popper]

I'm not convinced that's a use case. Why would you choose to encode on that as consumer. And we can't install x264 (or anything else for that matter) on video cameras, so again I don't see a point.

actually You Can, (well this developer version doesnt have a cam right now,and the netbook when its ready may or not but they can take a usb cam) at a higher price right now, or if your a developer you may get a unit under consession if your doing something of real value to them in the OSS space perhaps.

http://bbrv.blogspot.com/2009/08/efikamx-on-sale-now.html
http://www.genesi-usa.com/products/openclient
http://www.genesi-usa.com/products/netbook not available yet.

theres no report of the Efika dev manager (neko) or anyone else with a board (markos, freevec etc),
compiling x264 Cortex A8 and running D1 H@L3.1 Encoder tests on that, and comparing to its internal Cortex A8 Encoder (can that even do High@? anything) quality so far.

but it may be tryed soon as their latest firmware catchs up with their HW, as 264 NEON is ready made for them ,and they DO know about the x264 NEON additions.

[LoRd_MuldeR]

Two questions regarding r1233 (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=ee95ca7970140e8b99af233b03828feb250f0af0):

In case of Core2 optimized builds: Would it be helpful to pass "-mssse3" rather than the "-msse" set by default? Or is that redundant to the "-march=core2" switch?
Is it sufficient to pass "-mfpmath=387" in --extra-cflags to make a build that will run on a CPU that only supports MMX, but no SSE? (like the good old Pentium2)

[J_Darnley]

Is it sufficient to pass "-mfpmath=387" in --extra-cflags to make a build that will run on a CPU that only supports MMX, but no SSE? (like the good old Pentium2)

No, because of the changes made to encoder/encoder.c

[LoRd_MuldeR]

No, because of the changes made to encoder/encoder.c

What "changes" are you referring to please?

I saw that there's a new error message that will pop up on machines without SSE support, if x264 was compiled with SSE enabled.

But what I'm trying to get is a build that doesn't have SSE enabled in a "hardcoded" way. So what do I need to do?

(I don't want to completely disable all ASM stuff, because at least MMX optimized code should stay enabled in the Pentium2 build)

[J_Darnley]

What "changes" are you referring to please?

You posted the link.

static int x264_validate_parameters( x264_t *h )
{
#ifdef HAVE_MMX
- if( !(x264_cpu_detect() & X264_CPU_MMXEXT) )
+ if( !(x264_cpu_detect() & X264_CPU_SSE) )
{
- x264_log( h, X264_LOG_ERROR, "your cpu does not support MMXEXT, but x264 was compiled with asm support\n");
+ x264_log( h, X264_LOG_ERROR, "your cpu does not support SSE1, but x264 was compiled with asm support\n");
x264_log( h, X264_LOG_ERROR, "to run x264, recompile without asm support (configure --disable-asm)\n");
return -1;
}


[EDIT] The message isn't completely new. To allow non-sse machines to run x264, just revert all the changes in 1233.
[EDIT2] If you are going to continue providing builds like this then watch out for actual SSE instructions being added.

[LoRd_MuldeR]

Sorry, please see my edit in the previous post...

[J_Darnley]

LOL, see my edits.

[LoRd_MuldeR]

[EDIT2] If you are going to continue providing builds like this then watch out for actual SSE instructions being added.

Well, from all that I understand, x264 does its own runtime CPU detection. So it won't call any SSE optimized ASM functions, unless SSE is explicitly supported by the CPU.

AFAIK only MMX instructions will be used in a "hardcoded" way, if MMX was supported by the machine x264 was compiled on. But that's not a problem for the Pentium2 processor.

However if the C compiler uses SSE instructions, this may result in code which won't run on Non-SSE processors. And we are not talking about the ASM code here anymore!

Starting with r1233 x264 does tell the C compiler to use SSE. All I wanted to know: Is passing "-mfpmath=387" in the --extra-cflags sufficient to avoid that? :confused:

(BTW: What about my second question: Is "-mssse2" redundant, if "-march=core" was set ???)

[J_Darnley]

When non-MMX support was dropped, this was because of an inlined function that (I think) was only disabled with --disable-asm. This hasn't happened for SSE, yet, because the commit is just plain C but it is faster when done with SSE instructions.

[EDIT] I have no idea about 'Is "-mssse2" redundant, if "-march=core" was set?' so I only responded to the bit I could answer.

[LoRd_MuldeR]

When non-MMX support was dropped, this was because of an inlined function that (I think) was only disabled with --disable-asm. This hasn't happened for SSE, yet, because the commit is just plain C but it is faster when done with SSE instructions.

Yes, I see. But this didn't answer my question. Or did I miss the point? ^^

Unless they start using SSE code in Inline-Assembly we should be on the safe side by disabling SSE code generation in the C compiler. But still I wonder whether "-mfpmath=387" is sufficient for that :confused:

I'd simply test it, if I only had a Pentium2 machine here. But I don't have one anymore...

[J_Darnley]

We need to stop editing our posts. :)

I meant: No, MMX is now used unless you configure with --disable-asm. See the (old) error message:
your cpu does not support MMXEXT, but x264 was compiled with asm support to run x264, recompile without asm support (configure --disable-asm)

That is a necessary restriction because MMX is explicitly used in one function. But with SSE, the restriction is only required because of the CFLAGS which tell the compiler to use use SSE. If you tell the compiler to use the "old" instructions then there is no need for the "no SSE" error.

[LoRd_MuldeR]

I meant: No, MMX is now used unless you configure with --disable-asm. See the (old) error message:
your cpu does not support MMXEXT, but x264 was compiled with asm support to run x264, recompile without asm support (configure --disable-asm)

That code is only enabled when HAVE_MMX is defined. And that is only defined when the machine where x264 was compiled on did support MMX.

If x264 was compiled on a Non-MMX machine, it will run on all Non-MMX machines just fine. No error message will appear...

That is a necessary restriction because MMX is explicitly used in one function. But with SSE, the restriction is only required because of the CFLAGS which tell the compiler to use use SSE. If you tell the compiler to use the "old" instructions then there is no need for the "no SSE" error.

But I suspect the error message will appear anyway on a Pentium2 CPU, even when I told the C compiler to not use SSE!

That's because the code in question is enabled (HAVE_MMX is defined), but the return value of x264_cpu_detect() won't have X264_CPU_SSE set for obvious reasons :eek:

[J_Darnley]

Configure doesn't check to see if MMX can be run, HAVE_MMX is defined if you don't disable asm and you are on x86 or x64.

If you disable asm, x264 will run on all x86 CPUs.

At present, the "no SEE" error will appear if x264 was build with asm and x264 detects your CPU lacks SSE. Previously it was "no MMX" if your CPU lacked MMX.

If for some masochistic reason you want to run x264 on a Pentium 2, you had better revert the changes in rev 1233 and compile. Then abort your encode, because your only encode is still going and start again.

[LoRd_MuldeR]

Maybe time to rename the HAVE_MMX symbol to ENABLE_ASM or something like that...

[kemuri-_9]

Maybe time to rename the HAVE_MMX symbol to ENABLE_ASM or something like that...

or not since the HAVE_(__ASM_FLAG__) is hardware specific:

PPC has no such flag,
x86/x86_64 uses HAVE_MMX
ARM uses HAVE_ARMV6, HAVE_ARMV6T2, and HAVE_NEON

you can't realistically use something as vague as 'ENABLE_ASM' to cover all these situations

[LoRd_MuldeR]

You are right. But "HAVE_MMX" just isn't meaningful any more. It's meaning now changed to "HAVE_SSE1" and probably may change again in the future...

[kemuri-_9]

make a patch to change all instances of HAVE_MMX to HAVE_SSE within the code base if you're that paranoid about it.
and maybe it'll get committed

[akupenguin]

HAVE_MMX hasn't meant MMX to the exclusion of SSE since r634 (http://git.videolan.org/?p=x264.git;a=commitdiff;h=8b37cc6aa5e23bf4a529b79745f73dedab1fa4ff), and it isn't going to change now.

[Dark Shikari]

We really don't care about Athlon K6-2s and Pentium 2s. Seriously.

[LoRd_MuldeR]

We really don't care about Athlon K6-2s and Pentium 2s. Seriously.

Okay, but with "--disable-asm" (lib)x264 will still run on those, right?

[Dark Shikari]

Okay, but with "--disable-asm" (lib)x264 will still run on those, right?Of course.

[LoRd_MuldeR]

i8x8/i4x4 never got analysed when fast_intra was toggled and RD was off; up to a 2-3% quality improvement in non-RD mode.
With this bug dating back to r369, this is probably the second-oldest bug ever fixed in x264.

So do I understand right that this bug did effect "--subme 5" and lower only?