x264 and Intel's Media Engine / Sandy Bridge

[royia]

As we all know by now, Sandy Bridge will have Media Engine built in with video Encoding capabilities:
http://www.anandtech.com/show/3922/intels-sandy-bridge-architecture-exposed

and:
http://www.anandtech.com/show/3922/intels-sandy-bridge-architecture-exposed/6

In light of this thread:
http://forum.doom9.org/showthread.php?t=155234

Do you think the Media Engine is doomed as well?
Or could it be used to accelerate, at least to some level, x.264?

What about the AVX instructions, do they hold any promise?

[MatMaul]

What about the AVX instructions, do they hold any promise?
If I understood correctly it adds wider floating point instructions which is useless for x264.

[burfadel]

The real question is AMD's SSE5 instruction set (well, collection), and FMA4.

[royia]

You say the Media Engine will not accelerate any sub routine in x.264?

[mariush]

According to DS which posted something about it on doom9+1, that "engine" is similar to an already existing Broadcom encoder chip and Intel may just have licensed it from them and included it in their chips - if this is the case it's nothing really innovative.
The actual performance would also be barely enough to encode an 1080p in real time (40-50fps) , using some relatively poor quality settings ... x264 is already much better optimized and gives much more speed on better quality compared to than this "engine".

Just see this quote:


The video encode engine is a brand new addition to Sandy Bridge. Intel is being light on the details of the encoder but we saw a demo where Intel took a ~3 minute 1080p 30Mbps source video and transcoded it to a 640 x 360 iPhone video format. The total process took 14 seconds and completed at a rate of roughly 400 frames per second.


Run a similar test with x264 and see how fast you can encode the clip using preset --fast or --veryfast.. Encoding something to 640x480 (though personally I really think it was 480x360 as video was for iPod/iPhone but whatever) is not hard.

[royia]

Could you link to the discussion?
Thanks.

Because when I encode 720x576 using x.264 I don't get any more than 20fps.

[LoRd_MuldeR]

As always, when those "fps" numbers are presented, they mean absolutely nothing, if they aren't put into a relation to quality and bitrate!

If that so called "Media Engine" encodes at 50 fps, but delivers quality like x264 with preset "fast" or "veryfast", then it will be of very limited use for us :rolleyes:

And, as always with "hardware" encoders, I would expected that the "speed -vs- quality" options will be restricted, if existent at all.

Moreover, if they simply flange-mounted an "encoder chip" to their CPU, you must think of it as a "black box", which you can either use (as a replacement for a software encoder like x264) or not use at all.

Hence it probably can't be used to accelerate (or take over) only a specific "sub routine" of x264. Anyway, the thing is called "x264" not "x.264" ;)

[mariush]

T:\Winfast>x264_new --bitrate 2000 --preset veryfast -o out.mp4 "Video0914-0100(
TV35).mpg"
ffms [info]: 768x576p 1:1 @ 25/1 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.1
[46.3%] 626/1351 frames, 117.82 fps, 2056.03 kb/s, eta 0:00:06


On a Q6600 overclocked at 3 Ghz. With medium, it goes down to about 30 fps.


roya: No point linking to the other forum, as it's just a general discussion about the new features in that processor, not something particular about this video engine.

Otherwise, I agree with Mulder, it won't be something exposed like SSE or MMX instructions... I believe it will probably be something like DXVA and programmers will probably just be able to configure some parameters exposed through the video driver. It won't be something like Cuda or OpenCL, which allows you to offload some parts or all the computing in the processors.

[Snowknight26]

By the way, it's either H.264 or x264. Since there's no middle ground, hopefully you can distinguish the two (and start using the correct one! :O).

[iwod]

Just when i saw the hardware encoder inside Sandy Bridge, i was thinking this Encoder would be great for Mobile Devices. On iPhone the quality would be unnoticeable given the source is HD unless it is very poor encoder.

Users would not have to worry about anything, just drag whatever they have and itunes would automatically transcode those into compatible format @ very fast rate. I dont know which encoder they used, but the one from Power VR would get you more then 1000fps encoding, that is a hour long movie would take less then 2 Min!!!!!!!!!!

[Blue_MiSfit]

unless it is very poor encoder.


It will be. But the results will probably be "good enough" for most casuals to transcode their pirated episodes of True Blood for their PMPs, plus it will be built into the bloody CPU.... so yeah I see this (or similar technology) being fairly successful in coming years.

Derek

[iwod]

Well Depending on your view on it, i wont classify it as poor ( yet ), it is only x264 is so good.

I just read this here
http://lee.hdgreetings.com/2010/09/intel-cpu-vs-nvidia-gpu-video-transcoding.html

It seems a delicate hardware encoder aren't much faster then x264 fastest pure CPU solution. I was expecting something like Power VR
http://www.imgtec.com/powervr/powervr-vxe.asp

Which encodes at 1000+ fps...

[Audionut]

i wont classify it as poor ( yet ), it is only x264 is so good.

Just because something is good, doesn't mean the other isn't poor.

Which encodes at 1000+ fps...

x264 --preset ultrafast --crf 40 -o h:\test.mkv h:\test.avi
ffms [info]: 640x360p 1:1 @ 30000/1001 fps (vfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Baseline, level 3.0
x264 [info]: frame I:29 Avg QP:38.00 size: 6631
x264 [info]: frame P:7032 Avg QP:41.15 size: 833
x264 [info]: mb I I16..4: 100.0% 0.0% 0.0%
x264 [info]: mb P I16..4: 8.9% 0.0% 0.0% P16..4: 17.6% 0.0% 0.0% 0.0% 0.0% skip:73.5%
x264 [info]: coded y,uvDC,uvAC intra: 10.7% 33.3% 8.0% inter: 2.8% 4.5% 0.2%
x264 [info]: i16 v,h,dc,p: 36% 28% 23% 12%
x264 [info]: i8c dc,h,v,p: 67% 16% 14% 3%
x264 [info]: kb/s:205.43

aborted at input frame 7067, output frame 7061
encoded 7061 frames, 606.77 fps, 205.48 kb/s

Q6600 @ 3.2

If I was to drop the resolution abit more and clock it @3.6, I reckon I could hit 1000fps too. :cool:

[iwod]

Wow... sorry i dont have such powerful hardware. If that is the case then what is the point of a dedicated Hardware Encoder on a fast CPU like Sandy Bridge?

[Audionut]

If that is the case then what is the point of a dedicated Hardware Encoder on a fast CPU like Sandy Bridge?

It will make Intel money.

[iwod]

Doesn't make any sense, How does Intel make money from have a Hardware Encoder? No one will buy Sandy Bridge because of a Hardware encoder if Sandy Bridge in itself is already performing much better then the Hardware Encoder. Both in terms of quality and speed.

People Buy Sandy Bridge because it is a Great Chip all around.

So if the Hardware encoder are not faster nor produce better quality then a software encoder, what is the point ?

[Audionut]

what is the point ?

Money. Bragging rights.

"Hey dude, my new Intel computer encodes my new Dora episodes at wicked speeds."

AMD has something just as useless. Now Intel does too.

And I would think that there would be plenty of people who would base some of the purchasing decision on the fact that it has a 'dedicated hardware encoder'.

Honestly, There's a great many things in the world where you think, what's the point.

[royia]

I guess too naive to think it would be flexible enough to allow some acceleration of at least the sub routines all H.264 encoders share.
Someone should just fax AMD / Intel the perfect SIMD instruction set and hardware block to give us x264 in good quality mode with speed over 120fps :-).

[aegisofrime]

Money. Bragging rights.

"Hey dude, my new Intel computer encodes my new Dora episodes at wicked speeds."

AMD has something just as useless. Now Intel does too.

And I would think that there would be plenty of people who would base some of the purchasing decision on the fact that it has a 'dedicated hardware encoder'.

Honestly, There's a great many things in the world where you think, what's the point.

If you are talking about AMD's AVIVIO encoder... Now that's something that is REALLY useless.

[Sharktooth]

Avivo is different. it uses GPU acceleration... it's not just some silicon stuffed into the CPU.

[iwod]

Hey dude, my new Intel computer encodes my new Dora episodes at wicked speeds.

That is exactly my point, my new Intel Computer would encode my new Dora episodes WITH or WITHOUT the hardware encoder at wicked speeds!!!!!

I seriously hope the final version of Sandy Bridge hardware encoder will be much faster then what is currently being tested.

[aegisofrime]

Avivo is different. it uses GPU acceleration... it's not just some silicon stuffed into the CPU.

From what I read and from my own experience with my Radeon 4870, it doesn't use the GPU. It's just a H.264 encoder that uses the CPU, the reason why it's somewhat fast is because it turns off all quality features.

There are plenty of reviews of it, and you will find that the quality it produces is downright awful, perhaps the worst of all H.264 encoders out there.

[Audionut]

Avivo is different. it uses GPU acceleration...

No it doesn't.

That is exactly my point, my new Intel Computer would encode my new Dora episodes WITH or WITHOUT the hardware encoder at wicked speeds!!!!!

Yeah, most people on this board know that. But with the right marketing, I'm sure there are thousands of people who buy it thinking it's the best and fastest encoder based on the media encoder component.

[jpl]

I think the intended benefit of Media Engine is reduced power consumption. It may be the same speed as the CPU running flat out at max speed, but the hardware encoder will use much less power.

This would be most beneficial for mobile devices when running on battery.

[Sharktooth]

No it doesn't.
proof please.
http://www.dinoxpc.com/articolo/schedevideo_avivo_affair_images/utilizzo_gpu.jpg
from this SS avivo WORKS using the GPU (GPU usage ~20% on a 4870).
your statement is simply not true: http://en.wikipedia.org/wiki/ATI_Avivo#ATI_Avivo_Video_Converter

Edit: Btw, a software solution is different than a hardware solution. SandyBridge's media engine is crappy and you have it in your CPU. No possibilities to improve it unless a new silicon... Avivo is a software using GPU HW and can be improved anytime.

[mariush]

I remember reading somewhere (don't ask me, in the driver release notes or somewhere) AVIVO only uses GPU to accelerate DECODING of some content - MPEG 2 in particular and possibly VC-1 too. It does not use the GPU at the actual encoding process.

[Sharktooth]

that's on unsupported GPUs. the screenshot is self explanatory. you wont see any GPU usage from decoding a video source since decoding uses a dedicated silicon (UVD) not the GPU.
if you want to try the latest xcode (avivo converter) get it here: http://www.atizone.it/download?object=file&objid=8450
it needs catalyst 10.9 and a recent ati card though.

[mariush]

Well, with my 4850 (which should be supported, don't you agree?), the only place I can see some GPU acceleration is when transcoding a VOB (720x480) to the MPEG-2 profile:

http://savedonthe.net/image/679/gpucoding.png

You can see the gray line was at 0% all the time I tried MP4 1080p to MP4 720p, to MPEG2 720p and then went up to 15% gpu and increased gpu processor frequency when I started the VOB to MPEG-2 process.

This video converter is super crappy anyway - it reloads the video each time you go from one tab to another and each time you move the slider (I have about 20 Haali splitter icons in the system tray now - i'll have to kill the ccc process to get rid of them ) and is overall not so optimized.

[Sharktooth]

i didnt say it was any good... ;)
however i dont know it 4850 is supported and i dont even know if the software makes choices depending on the CPU model/frequencies.
what i know is it is developed using stream sdk and "sometimes" it uses GPU acceleration and those cases were different with different xcode versions.

[Caroliano]

There is one thing that this hardware encoder will definitely bet by far x264: fps and quality per watt. Maybe a good thing for laptops. Other than that, seems like a little waste of 3mm^2 of silicon.

[Sharktooth]

encoding with a laptop... anyone that buys a laptop for encoding is insane...
OTOH x264 could deliver better performance and quality on a laptop too.

[Audionut]

proof please.

No, it's not in hardware since there is a "cracked" avivio encoder that runs on EVERY video card...


from this SS avivo WORKS using the GPU (GPU usage ~20% on a 4870).

the transcode acceleration will not be ready by that time. ATI has committed to delivering the transcode acceleration by the end of this year,


Nice to see that in 5 years they've managed to get the GPU usage to 20%.

And here we have speculation that it is from decoding, not encoding. Can you prove that 20% is being used to encode!

[Sharktooth]

as i said it's developed using stream sdk. also, UVD usage (that's the silicon used for decoding) is not measured in the GPU usage since it's a standalone unit.

[Caroliano]

encoding with a laptop... anyone that buys a laptop for encoding is insane...
OTOH x264 could deliver better performance and quality on a laptop too.
You don't need to "buy it for encoding" to encode in it. And I said performance and quality PER WATT. In a laptop this can mean how many hours you can encode on one charge, as a benchmark.

[Didée]

Some days ago...

CPU: Intel i7-860
GPU: GeForce GT240

Source: 1280x720@50fps, Avi/Huffyuv


CUDA @ default / --bitrate 4150: 94 fps

x264_x64 --preset superfast: 101 fps

Results: LINK (http://www.mediafire.com/?d6xb7m46ua4l6iv) (~13MB, MediaFire)
With GPU/Hardware encoding becoming more popular, the world will be flooded with bad quality videos.

[iwod]

encoding with a laptop... anyone that buys a laptop for encoding is insane...
OTOH x264 could deliver better performance and quality on a laptop too.

Well, you can now finally drag all kinds of Video into iPhone and the Hardware encoder will do the transcoding for you at decent speed ( i wouldn't say very fast unless it is 1000fps + ) .

Even at 450 fps, 120 Min Video would still takes 6 min..... not including the time to transfer that file to iPhone........

[deekey777]

From what I read and from my own experience with my Radeon 4870, it doesn't use the GPU. It's just a H.264 encoder that uses the CPU, the reason why it's somewhat fast is because it turns off all quality features.
Try this one:
http://www.cyberlink.com/downloads/trials/mediaespresso/download_en_US.html
Your HD4870 will encode videos to H.264 (eg for PS3) or MPEG2. And if you enable video decoding over GPU, your HD4870 will encode with UVD clocks.

There are plenty of reviews of it, and you will find that the quality it produces is downright awful, perhaps the worst of all H.264 encoders out there.
I've played with the MSE several times: You get the best result, when MSE lets the graphics card decode the video and the CPU encode it. For me (HD4850&Q6600) it was the best way: Same speed and much better quality.

(But MSE is still piece of s...)

[bsoft]

(Sorry to reopen a dormant thread, but SB is shipping soon so this will affect a lot of people)

Anyone who's comparing the performance of the Intel Sandy Bridge media engine to x264 is missing the point. The SB media engine is not really about permanent transcoding.

You're going to get better results with a software encoder. Always. Hardware is harder to implement and there are real cost/power constraints that x264 doesn't have to face because it can always just use more time.

SB's H.264 encoder is good where low CPU usage is more important than high bitrate efficiency. Want to stream your StarCraft II matches without stealing CPU time from the game? Want to stream to your DLNA TV or WiDi device without killing your battery? Want to watch a recorded program on your smartphone or tablet without sucking down 100W from the wall?

For these uses, how fast the SB streaming engine is doesn't matter - as long as it's at least realtime. Hell, in the LAN cases how efficient it is doesn't matter much either, since you can always throw more bitrate at the problem.

No matter how good x264 is you can't get away from the fact that it keeps the CPU busy. Busy CPUs mean crappy battery life. Busy CPUs mean that there are less cycles for other tasks.

Does the prospect of being able to screencast to my TV with near-zero CPU load interest me? You bet it does.

[Dark Shikari]

(Sorry to reopen a dormant thread, but SB is shipping soon so this will affect a lot of people)AFAIK, the Sandy Bridge shipping in January will not have the transcoding engine. You're going to be waiting quite a while.

[aegisofrime]

AFAIK, the Sandy Bridge shipping in January will not have the transcoding engine. You're going to be waiting quite a while.

May I ask where you heard that from? I have not seen anything saying that on the usual tech news sites.

[saint-francis]

DS have you have the ability to see results of x264 on the new SB? Is there going to be any gain? I'm not really sure what the point of this up coming SB is.

[LoRd_MuldeR]

DS have you have the ability to see results of x264 on the new SB? Is there going to be any gain? I'm not really sure what the point of this up coming SB is.

AVX (Advanced Vector Extensions) should help x264, I guess - if new AVX-optimized assembly is written.

But AFAIK the first Sandy Bridge generation will only support AVX with 256-Bit registers, rather than the full 512-Bit. Still that's twice the size of the SSE registers.

Also you'll need Windows 7 with SP-1 to be able to use AVX. Or some recent Linux kernel ;)

[poisondeathray]

AVX (Advanced Vector Extensions) should help x264, I guess - if new AVX-optimized assembly is written.

But AFAIK the first Sandy Bridge generation will only support AVX with 256-Bit registers, rather than the full 512-Bit. Still that's twice the size of the SSE registers.

Also you'll need Windows 7 with SP-1 to be able to use AVX. Or some recent Linux kernel ;)

I think D.S. said AVX wasn't going to be useful for x264



What about 256bit AVX? Each module can process only one at time (AFAIK) and x264 will surely support these new instructions (I hope eheh)

Float-only, thus a useless pile of tripe.


http://doom10.org/index.php?topic=514.0

[LoRd_MuldeR]

Didn't know that AVX is FP-only. That's a pity...

[deadrats]

I think D.S. said AVX wasn't going to be useful for x264.


if i had a dollar for every time his darkness has said something that didn't make any sense i would be a rich man by now.

he says avx is "Float-only, thus a useless pile of tripe" yet what he fails to mention is that he could convert the code to floating point, there's nothing that says it must be integer based.

as a very simple example if you have the following code snippet:

for ( a = 1; a < 100001; a++ )
for ( b = 1; b < 100001; b++ )
{
ab = a * b;
}

and use the following variable declaration:

int a, b, ab;

you cause the above to be executed on the alu (integer unit), if however you do this:

float a, b, ab;

it's executed using the floating point registers.

depending on the compiler you can even do something like this:

_m128i a, b, ab;

and perform a scalar calculation using the sse registers (there's a bit more code required than just that, but you get the idea).

yes, it would be a lot of work to rewrite the code to take advantage of the new avx registers and it's contingent on gcc supporting the required assembler instructions (he could always spend the dough and buy a copy of intel's compiler, though he would also need a copy of visual c++), but there's nothing inherently integer based about the code (other than that's they way he wants it) and there's nothing really standing in his way from changing it to take advantage of the sandy bridge's capabilities. (<--in all fairness, if he did do this he would need to maintain 2 versions of x264, one for cpu's that support avx and one for those that don't and he may not be willing to do that).

[Dark Shikari]

he says avx is "Float-only, thus a useless pile of tripe" yet what he fails to mention is that he could convert the code to floating point, there's nothing that says it must be integer based.Of course. You could convert it to floating point, and it would be at least 2 times slower, even with AVX, because you can only fit 8 floating point values in an AVX register, compared to 16 pixel values in an SSE register. It might be worthwhile if the AVX registers were 512-bit or 1024-bit, but at only 256-bit, it's not enough to justify a switch. There's also the additional problem of floating point instructions generally being slower than integer instructions, but even ignoring that, it's probably not worth it.

Anyways, since you're such a genius, write me a 16x16 SAD function that uses AVX and floating point input and runs in under 35 clock cycles (the speed of the SSE implementation on a Core i7). Here's the code for the C (with uint8_t converted to float):

static float sad_16x16( float *pix1, int stride_pix1, float *pix2, int stride_pix2 )
{
float sum = 0;
for( int y = 0; y < 16; y++ )
{
for( int x = 0; x < 16; x++ )
sum += fabs( pix1[x] - pix2[x] );
pix1 += stride_pix1;
pix2 += stride_pix2;
}
return sum;
}

I'll make it easy on you; you can assume the inputs are aligned (they aren't in reality).

[Sharktooth]

pwned...
FP math is ALWAYS slower than INT math, unless your have a CPU with ridiculously big FP registers.
also FP math leads to precision problems over time... unless you use a ridiculously high FP precision...
that said, INT math is a way better solution.

[deadrats]

Anyways, since you're such a genius, write me a 16x16 SAD function that uses AVX and floating point input and runs in under 35 clock cycles (the speed of the SSE implementation on a Core i7). Here's the code for the C (with uint8_t converted to float)

i just realized you're using 8 bit int's, all this time i thought you were using 32 bit int's, now the various objections you have raised make sense.

as for the homework assignment, i will gladly admit that i can't do it, but i will throw you a bone and use an excuse that you are fond of: company x hasn't sufficiently documented technology y, so it's their fault not mine.

simply replace x with intel and y with avx.

and here's some more excuses: i don't own an avx enabled cpu and i don't have a compiler that supports that instruction set (i don't think gcc supports it yet).

but i'm man enough to admit that like most people i don't know how to code with avx instructions...yet.

[Dark Shikari]

i just realized you're using 8 bit int's, all this time i thought you were using 32 bit int's, now the various objections you have raised make sense.Exactly -- that's the real power of integer SIMD -- the fact that you can get more throughput by using smaller data types.

Pixels are only 8-bit, and transform intermediates (as well as DCT coefficients) are only 16-bit. x264 makes very minimal use of anything larger than 16-bit.

Ironically, with integer SIMD, the variety of instructions available for 32-bit is actually rather lacking. For 16-bit, for example, you have pmulhw, pmullw, pmulhrsw, and pmaddwd for multiplication, providing a pretty good variety of instructions. For 32-bit, you basically only have pmulld and pmuldq -- the former of which is slow and SSE4-only, and the latter of which only does two multiplies, hardly justifying SIMD at all.

[deadrats]

Exactly -- that's the real power of integer SIMD -- the fact that you can get more throughput by using smaller data types.

Pixels are only 8-bit, and transform intermediates (as well as DCT coefficients) are only 16-bit. x264 makes very minimal use of anything larger than 16-bit.

Ironically, with integer SIMD, the variety of instructions available for 32-bit is actually rather lacking. For 16-bit, for example, you have pmulhw, pmullw, pmulhrsw, and pmaddwd for multiplication, providing a pretty good variety of instructions. For 32-bit, you basically only have pmulld and pmuldq -- the former of which is slow and SSE4-only, and the latter of which only does two multiplies, hardly justifying SIMD at all.

you know, it seems i owe you an apology, all this time i was reading various posts you made as well as some of the things you had written in your "diary of an x264 developer" and a lot of what you said struck me as absurd. now it all makes sense.

on the topic of gpu accelerated encoding, is one of the reasons you have claimed that for any given number of threads a cpu will be faster because with the cpu you can use 8 bit and 16 bit int's and with cuda (and it's brethren) you have to use 32 bit int's minimum?

is it also safe to assume that bulldozer, with it's 2 128 bit alu's per core, will be THE cpu to get for x264 encoding?

2 more quick questions: you recently signed a licensing agreement with pegasys and reading some of the press releases it seems that you created a parallel "commercial friendly" license under which you licenses x264 llc (that is the name of the commercial variant, is it not?). does this not violate the spirit, if not the letter, of the gpl?

i know many companies consider the gpl an "infectious" license, but doesn't the gpl explicitly forbid taking gpl'd code and making closed source? does it not also require that any derivative work also be gpl'd?

by creating a parallel licensing scheme haven't you a) created a derivative that's not gpl'd, b) opened the door for companies to create derivatives that are not gpl'd, c) opened the door for companies to close source the x264 code they license from you, d) and perhaps most importantly open the door for a company to make some simple changes and try and claim copyright to that, a claim that they could use to prevent you from making similar changes to the gpl'd version of x264?

lastly, i'm wondering what ide do you use during the development of x264, i'm assuming you use gcc to build the executables but do you use a front end like code blocks or dev-c++? also what, if any optimization options do you use? do you target any specific architecture, simply use -O3, a combination?

thanks.