x264 development

[Dark Shikari]

i8x8/i4x4 never got analysed when fast_intra was toggled and RD was off; up to a 2-3% quality improvement in non-RD mode.
With this bug dating back to r369, this is probably the second-oldest bug ever fixed in x264.

So do I understand right that this bug did effect "--subme 5" and lower only?Subme 6 and lower, as 7 enables RD on B-frames.

[LoRd_MuldeR]

Subme 6 and lower, as 7 enables RD on B-frames.

Ah, okay :)

[Daemon404]

Since ARM support has been added recently, I was wondering, have any tests been done to compare speed and/or power draw on ARM procs, in comparison to similar (as in price/speed/mobile) x86 procs (e.g. atom)?

[aegisofrime]

Since ARM support has been added recently, I was wondering, have any tests been done to compare speed and/or power draw on ARM procs, in comparison to similar (as in price/speed/mobile) x86 procs (e.g. atom)?

I'm interested to know as well.

[Dark Shikari]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.

[popper]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.

as a more general baseline on a full real devs/consumer PCB, and evolving Firmware/SW
(remember "Efika" and 'Efika MX' Are two totally seperate PCB, one a PPC G3 ,discontinued, one the arm i.MX515 )

Matt Sealey,Developer Relations,Product Development Analyst
for Genesi said of their 'Efika MX' Open Client, Freescale i.MX515 (ARM Cortex-A8/NEON 800MHz) board

"BTW Open Client power consumption running Citrix is 5W maximum, tested using a Kill-A-Watt.

I've not been able to try out any video decoding yet, and again this is before any advanced power management has been enabled."

"the chip is running at top speed and the backlight is turned on full.. You can expect some very decent power savings across the board once the software is more mature."

http://www.powerdeveloper.org/forums/viewtopic.php?t=1557&postdays=0&postorder=asc&start=0

regarding their codecs for it, he also said this on the blog
"You can play video from anything, depending on the speed of the device. USB should be fine for DVD-quality video. You might hit limits on an SDHC card (15MB/s or thereabouts) for HD content.

We did not release accelerated codecs yet but we are working on it."

there are OC several tests on that thread and linked off it, so you can get a good feel for their full load averages etc,
the only real No's you should care about as its doing real work, its odd, but no one seems to have tryed using the
included wireless 11N streaming/down/uploading while collecting the power No's so far...

for instance Peter Czanik (czp) Jul 13, 2009 said

"I just measured the power consumption of my iMX515 based EFIKA system - display device is included. It also has 4x more RAM and 2x the MHz of the original EFIKA, but no HDD, just a larger SD card. Power consumption is 9W with the bundled LCD turned on. A 2.5" USB HDD add one more Watt and an USB Ethernet adapter another one. When the LCD turns off, it spares 2-3W.
"

[Daemon404]

Results are pretty much what I figured. Anyway, is there anything publicly available that uses A9 yet?

[nakTT]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.
I also getting more exited with these ARMs thingy going on, especially the upcoming Cortex-A9 MPCore.

Just out of curiosity to get a more clear picture, may I know the clock speed of all the processors in question? (Core 2, ARM Cortex-A9 [from which company?], the Pentium 4).

[popper]

I also getting more exited with these ARMs thingy going on, especially the upcoming Cortex-A9 MPCore.

Just out of curiosity to get a more clear picture, may I know the clock speed of all the processors in question? (Core 2, ARM Cortex-A9 [from which company?], the Pentium 4).

in reply to a imagination/POWERVR VXE320 and VXE360 cores Encoder IP question apparently "Raquel and Bill owners of genesi said... the graphic cores found in the i.mx51 were licensed from AMD not Imagination. We do have a source license and the code from AMD so you can expect the performance to be very good." make of that what you will, until some real Numbers become available at least.

as for the Freescale Cortex-A9, that comes in both a single and dual packages, but theres NO sign what so ever of any retail PCBs on the horizon if thats what your hoping for and after, so no real life clock data for you , but just for interest you might find this http://arm.com/products/CPUs/ARMCortex-A9_MPCore.html a good starting point.

its interesting from a sometime in the future NEON MP x264 POV IF these initial freescale A8/NEON take off on mass...

[Blue_MiSfit]

Hmm... I wonder if a whole mess of tiny embedded systems would be more power / space efficient than my 2U 8 core encoders? :)

~MiSfit

[nakTT]

in reply to a imagination/POWERVR VXE320 and VXE360 cores Encoder IP question apparently "Raquel and Bill owners of genesi said... the graphic cores found in the i.mx51 were licensed from AMD not Imagination. We do have a source license and the code from AMD so you can expect the performance to be very good." make of that what you will, until some real Numbers become available at least.

as for the Freescale Cortex-A9, that comes in both a single and dual packages, but theres NO sign what so ever of any retail PCBs on the horizon if thats what your hoping for and after, so no real life clock data for you , but just for interest you might find this http://arm.com/products/CPUs/ARMCortex-A9_MPCore.html a good starting point.

its interesting from a sometime in the future NEON MP x264 POV IF these initial freescale A8/NEON take off on mass...
Many thanks for you reply. As of now I only have eyes for Cortex-A9 MPCore or at least A8. You seems to know quite a lot about ARM processor. This is kind of guy I'm looking for when comes to advice and real-life info.

As for the retail PCB unavailability, I think I can settle for the upcoming ARM SmartBook instead (rumour to be released sometime in the near future) though I still prefer it to be in the 'OpenRD's form factor' instead of a SmartBook.

Any idea (dispense with the manufacturer's marketing talk) what can we expect from Cortex-A9 MPCore from Qualcomm 1.5 GHz part, TI 1.0 GHz part (rumour to be more power efficient compared to Qualcomm's 1.5 GHz part clock-for-clock) and Freescale, Marvell, Samsung and the like? Please share your thought.


:thanks:

[nakTT]

Hmm... I wonder if a whole mess of tiny embedded systems would be more power / space efficient than my 2U 8 core encoders? :)

~MiSfit
I think they are, in term of power/performance/space. But I maybe wrong.

[popper]

Hmm... I wonder if a whole mess of tiny embedded systems would be more power / space efficient than my 2U 8 core encoders? :)

~MiSfit

if you can write a (message based ? Realtime Multitasking [Amigs OS style] data passing) patch to make x264 Effectively use these mass of clustered CPU/SOC/SIMD over a gigE+ tcp:IP/UDP links.

or start a well run community based 'Bounty program' Original AROS bounty style http://www.power2people.org/projects.html http://aros-exec.org/ to collect financing for missing and/or interesting quality based extensions such as adding Generic multi platform clustered Encoding to the x264 codebase and GUI support applications etc, and so kick start them to get more professional coders interested in contributing.

then you might stand a reasonable chance of better using the under used generic LAN/WAN connected device resources, both now, and in the future, OR not as the case may be, again make of this really Old idea what you will ;)

[Dark Shikari]

I also getting more exited with these ARMs thingy going on, especially the upcoming Cortex-A9 MPCore.

Just out of curiosity to get a more clear picture, may I know the clock speed of all the processors in question? (Core 2, ARM Cortex-A9 [from which company?], the Pentium 4).Clock speed doesn't matter for those numbers, and it's rather easy to look up what clock speeds the P4/Core 2 were/are available in, but as far as I recall the Cortex chips are currently available around ~600mhz and may scale up to about 1Ghz.

[nakTT]

Clock speed doesn't matter for those numbers, and it's rather easy to look up what clock speeds the P4/Core 2 were/are available in, but as far as I recall the Cortex chips are currently available around ~600mhz and may scale up to about 1Ghz.
Sorry, totally my mistake. I have missed your "per clock" word while reading your previous post. My bad.


From a test done a while back on H.264 decoding with all assembly code turned off
I'm no programmer, perhaps you could explain a little bit what is the significance of "assembly code turned off/on" so that people like me can better understand things.

:thanks:

[Dark Shikari]

I'm no programmer, perhaps you could explain a little bit what is the significance of "assembly code turned off/on" so that people like me can better understand things.

:thanks:It means we're measuring (counting in whatever inefficiencies of GCC and whatnot) the performance per clock in ordinary situations without using the SIMD unit.

If you don't know what SIMD is, look it up.

[Astrophizz]

Assembly code turned off: slow (relative to regular performance), relies more heavily on compiler for speed.
Assembly code turned on: blazing fast.

Edit: doh i got beat!

[nakTT]

It means we're measuring (counting in whatever inefficiencies of GCC and whatnot) the performance per clock in ordinary situations without using the SIMD unit.

If you don't know what SIMD is, look it up.
I see. In other word we can expect the "H.264 Decoding" number will somewhat get improved overtime (per clock). Am I understand you correctly?

How about encoding? Any info/figure that you can share from your previous testing?

:thanks:

[Dark Shikari]

I see. In other word we can expect the "H.264 Decoding" number will somewhat get improved overtime (per clock). Am I understand you correctly?Sure, if ARM improves their chip designs, the chips' performance will improve.

[nakTT]

Sure, if ARM improves their chip designs, the chips' performance will improve.
Actually what I meant is the improvement from better coding.

As of now we can even encode using x264 on ARM based system?

[popper]

i do wonder if theres any plans by anyone to make x264 FPGA ready one day,Dark Shikari, or if its even possible as the code stands today for effective translation, now that might be very interesting, nakTT that means a version of x264 routines translated to and built into hardware rather than software running on some cpu/simd hardware as today.

the old The H.264 Scalable Video Codec (SVC) is still a very popular read
http://www.dspdesignline.com/howto/206902266

[Dark Shikari]

Actually what I meant is the improvement from better coding.

As of now we can even encode using x264 on ARM based system?Did you even read the past two pages of this thread? ;)

[nakTT]

Did you even read the past two pages of this thread? ;)
No I'm not. :D

Sorry, will try to read it.

:thanks:

[Kurtnoise]

The most useful patch for today (http://pastebin.com/f737a88e8)...:D

[Dark Shikari]

The most useful patch for today (http://pastebin.com/f737a88e8)...:DWhy? It works fine as-is.

[Kurtnoise]

yes but I was just curious by the -Aall during compilation...so

[vucloutr]

Does Multi-Slice encoding support (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4) affect multithreading in any way ? If yes, how ?

[Dark Shikari]

Does Multi-Slice encoding support (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4) affect multithreading in any way ? no .

[kemuri-_9]

yes but I was just curious by the -Aall during compilation...so

the short options can either be followed immediately by the value or have a space between:

-m0 can also be done as -m 0, both are accepted.

[G_M_C]

Any news on weightp, that is supposed to help fades with MBTree?

weightp will be committed when it's GSOC author is done - within days or so. Stop asking


Since GSoC was/has finished on 17 august I wondered if there is any news on the status on weightp atm ?

[J_Darnley]

Lurk more.
http://mailman.videolan.org/pipermail/x264-devel/2009-September/006324.html

[G_M_C]

Lurk more.
http://mailman.videolan.org/pipermail/x264-devel/2009-September/006324.html

Thx :)

I'm no developer, so i'm not usually lurking on the development pages, so i missed that post. I'm "just a user", but an interested one, that why i hope to expiriment with this patch (new addition/new feature might be a better description) when it's commited.

[microchip8]

Thx :)

I'm no developer, so i'm not usually lurking on the development pages, so i missed that post. I'm "just a user", but an interested one, that why i hope to expiriment with this patch (new addition/new feature might be a better description) when it's commited.

If you don't want to lurk on these pages, then do as I do. Subscribe to x264 devel mailing list and get your notifications for free and on time ;)

[iwod]

A Cortex A8 uses about 0.2 watts, while an Atom with chipset uses about 15 watts. They're hardly comparable.

From a test done a while back on H.264 decoding with all assembly code turned off, if a Core 2 has a speed of "1.0" per clock, while a Cortex A9 has a speed of about "0.4" per clock, a Pentium 4 has "0.38", and a Cortex A8 has about "0.28", if I recall the numbers correctly. I don't know if the gap closes or increases with the addition of NEON, but the Atom has much much worse performance than the Pentium 4, so I would expect the Cortex A9 to trash the Atom clock-for-clock; even the A8 should beat it.

Arh... That is some very Sad numbers.
When i saw Intel do have a very clear path to 10nm. I thought that was the end of ARM. Since within 2 - 4 years time they could scale Atom into current ARM size with nearly the same power envelop. And since Hardware is Cheap, Software is expensive, no one will be border to port anything over to ARM.

However these numbers looks like there is still a huge gap.

Core2Duo have 4 times the Maximum Frequency then Cortex A9.
Core2Duo uses around 50 - 100 times the Max Power of Cortex A9's Max power.
Core2Duo has , i guess nearly 10 times the Die Size. ( Properly much less with out the Cache though )

Now i wonder if Core2Duo with MMX+SSE would bring higher performance increase then Cortex A9 + Neon.

Looks like ARM still have some fighting chance.

[Sagekilla]

Sure, a Core 2 can make the Cortex A8/A9 look like crap performance wise. Dark just said that.

But Atom is not the same chip as Core 2. It's based off that architecture, but they removed a lot from it to get it to be power efficient (IIRC, Out-of-order execution, fewer execution units, instruction emulation, among other things).

Sure, if they can shrink down a Core 2 and adjust the core just right so they don't gimp it too much, it could be a threat to Cortex.

IMO, they'd have better results trying to optimize their process for ultra low voltage processors and then doing some stripping of more power hungry features. Latest ULV processors dissipate 5.5 - 10 W, atom averages 2.5 - 8. Those ones are full fledged Pernyns, mind you.

[popper]

Arh... That is some very Sad numbers.
When i saw Intel do have a very clear path to 10nm. I thought that was the end of ARM. Since within 2 - 4 years time they could scale Atom into current ARM size with nearly the same power envelop. And since Hardware is Cheap, Software is expensive, no one will be border to port anything over to ARM.

However these numbers looks like there is still a huge gap.

Core2Duo have 4 times the Maximum Frequency then Cortex A9.
Core2Duo uses around 50 - 100 times the Max Power of Cortex A9's Max power.
Core2Duo has , i guess nearly 10 times the Die Size. ( Properly much less with out the Cache though )

Now i wonder if Core2Duo with MMX+SSE would bring higher performance increase then Cortex A9 + Neon.

Looks like ARM still have some fighting chance.

OC they have now commited to a 2gig speed for the A9, although no ones going to produce anything substantial until the last Quarter it seems, and we still dont know the generic configuration of the dual-NEON™ as yet...
and one of the reasons i asked about any existing or planed x264 IP for FPGA/ASIC and DSP related use sooner or later....

http://www.arm.com/news/25922.html
16 September 2009

ARM Announces 2GHz Capable Cortex-A9 Dual Core Processor Implementation
...
"The Cortex-A9 power-optimized hard macro implementation delivers its peak performance of 4000 DMIPS while consuming less than 250mW per CPU when selected from typical silicon.

The hard macro implementations include ARM AMBA®-compliant high performance system components to maximize data traffic speed and minimize power consumption and silicon area. Each Cortex-A9 hard macro implementation also includes the CoreSight™ Program Trace Macrocell (PTM) which provides full visibility into the processor’s instruction flow, enabling the software community to develop code for optimal performance.
..."

"...
Both ARM dual core Cortex-A9 hard macros will share a common seven-power domain, dual-NEON™ technology configuration supporting SMP (symmetrical multiprocessing) operating systems with up to 8MB of Level2 cache memory and will be delivered with all scripts, vectors and libraries required to integrate the macro directly within any SoC device.
..."

"implementations are available for license today with delivery in the fourth quarter of 2009. ARM’s 40G physical IP platform is also available today at designstart.arm.com."

http://www.arm.com/products/physicalip/designstart.html
"...
DesignStart
Welcome to DesignStart, the most comprehensive online IP access portal in the industry. DesignStart offers a full compliment of Free Physical IP libraries and extensive front end views or models of Fee based IP supporting industry leading EDA tools, providing a unique try before you buy design environment. In addition, we offer selected foundry sponsored Processor Design Kits.
..."


http://search.arm.com/search?q=high+profile+h.264&site=FullSiteSearch&client=ARM_Search&output=xml_no_dtd&sort=date%3AD%3AL%3Ad1&ie=UTF-8&oe=UTF-8&proxystylesheet=ARM_Search

[Fr4nz]

I saw that the upcoming weighted p-frame prediction uses k-means; my question to DS is: are you using the old Lloyd's algorithm or something better (in terms of speed and result quality), like k-means++ (http://theory.stanford.edu/~sergei/papers/kMeansPP-soda.pdf)? If not, consider that paper, it should also be easy to implement as the Lloyd's algo...

[Dark Shikari]

I saw that the upcoming weighted p-frame prediction uses k-means; my question to DS is: are you using the usual Lloyd's algorithm or something really better (in terms of speed and result quality), like k-means++ (http://theory.stanford.edu/~sergei/papers/kMeansPP-soda.pdf)? If not, consider that paper...Yes, we use k-means++.

[LeonLanford]

Hi, I want to ask a little, I just updated from "x264 core 75 r1251M e553a4c" to "x264 core 75 r1259M dd026f2".

I use crf for encoding and I noticed that with the same old crf the size became quite lot smaller(and the quality degrades also).

Is the crf settings got changed? I usually use crf 26, I want to know how much the crf changed so I can predict what crf I can use now.(is it 23?)

Hope someone can answer,
Thanks

[Dark Shikari]

There has been no significant change in CRF in the past 8 revisions.

[wyti]

And what about this (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commitdiff;h=411ee507328890fb1fde96b37b6eef854d27101a) ? it should slightly reduce bitrate at the same CRF.

btw any news on planning release date of weighted P ? (weeks ? months ?) it's seems amazing and i really want to test it with Mb-tree (fades make me sick :P)

[LeonLanford]

There has been no significant change in CRF in the past 8 revisions.

Uh I'm using the same script and same file but I got about 30% size reduction and quality degrades also..

the old one

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 5 frames
Muxing mode : Container profile=Unknown@3.0
Codec ID : V_MPEG4/ISO/AVC
Duration : 2mn 59s
Bit rate : 289 Kbps
Width : 512 pixels
Height : 384 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.049
Stream size : 6.19 MiB (85%)
Writing library : x264 core 75 r1251M e553a4c
Encoding settings : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=6 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=4 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=27.0 / qcomp=0.90 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:0.50


the new one

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.0
Format settings, CABAC : Yes
Format settings, ReFrames : 5 frames
Muxing mode : Container profile=Unknown@3.0
Codec ID : V_MPEG4/ISO/AVC
Duration : 2mn 59s
Bit rate : 209 Kbps
Width : 512 pixels
Height : 384 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.036
Stream size : 4.49 MiB (81%)
Writing library : x264 core 75 r1259M dd026f2
Encoding settings : cabac=1 / ref=5 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=6 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=4 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=27.0 / qcomp=0.90 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:0.50

[Dark Shikari]

Source, exact commandline, and full x264 output (output stats) for both?

[LeonLanford]

Source, exact commandline, and full x264 output (output stats) for both?

I'm using avidemux, I don't know how to generate output stats or exact commandline. I only use custom job script to encode.
The source is chi's new address ep 68 (http://www.nyaatorrents.org/?page=download&tid=85708).
I also use other source and get the same reduction.

[Dark Shikari]

Does it occur if you use x264 itself natively instead of Avidemux?

[LeonLanford]

Does it occur if you use x264 itself natively instead of Avidemux?

I don't know how to use x264 without avidemux.

Anyway I just want to ask about if the crf scale got changed or not, but it seems it's avidemux problem? I'll just do some experiments to get my ideal crf again then.

Thanks for the reply..

[LoRd_MuldeR]

Anyway I just want to ask about if the crf scale got changed or not, but it seems it's avidemux problem?

Nope, the CRF scale didn't change noteworthy between r1251 and r1259. Also I see no such behavior with Avidemux :confused:

Just encoded the "Black Pearl" sample with Avidemux r5341 (Gruntster's build) with both, libx264 r1251 and libx264 r1259 (my own builds), using a CRF of 22.

As you can see the difference in total size is marginal:

22.09.2009 17:04 18.771.812 BlackPerl.x264-r1251.crf-22.avi
22.09.2009 16:57 18.679.516 BlackPerl.x264-r1259.crf-22.avi

That's a difference of less than 0.5% and therefore should be inside the expected fluctuations ;)


I don't know how to use x264 without avidemux.

http://forum.doom9.org/forumdisplay.php?f=78 :sly:

[Chengbin]

What does the upcoming weight p feature do?

Does it improve quality beside improving fades?

[burfadel]

I also take it weighted B frames will work with mb-tree once weight p comes in?

[Dark Shikari]

I also take it weighted B frames will work with mb-tree once weight p comes in?Weighted B-frames work just fine with MB-tree right now.