parallelizing the x264 on the ps3

[nithins]

Hi,

I am thinking of parallelizing the x264(or portions of it) as a university project on the cell processor. I have access to a small cluster(8 nodes) of PS3s.

I am a complete noob when it comes to video encoding. Does anyone have an idea if it is possible to do this in a span of 4 months by a single person. And also could you please point me to some useful resources that could help me get started off.

Thanks a lot.

- N

[BlackSharkfr]

i've seen a project like that in this very forum called x264-farm
http://forum.doom9.org/showthread.php?t=117889

[akupenguin]

x264farm can use a cluster, but without additional code you just have a cluster of reasonably slow PPCs. There are several projects trying to make x264 use the Cell's real capacity, but none have gotten very far.

[Adub]

Probably not possible for a video noob to do it in less than 4 months.

[Atak_Snajpera]

There are several projects trying to make x264 use the Cell's real capacity, but none have gotten very far.

Theoretical question. Does it make sens at all? Could we expect huge speed up?

[Dark Shikari]

Theoretical question. Does it make sens at all? Could we expect huge speed up?Certainly, because x264 is not at all optimized for Cell architecture (one main core, 7 Altivec-only cores).

Realistically, I doubt it'll happen, just because so much of the code would have to be drastically reworked to take full advantage of the Cell, to the point where it would result in a fork of the project.

[buzzqw]

well my ps3 will be glad to test this encoding :) and me too

just to view how fast can be...

BHH

[RaynQuist]

Certainly, because x264 is not at all optimized for Cell architecture (one main core, 7 Altivec-only cores).project.

I think the question was will an optimized x264 running on Cell be significantly faster than our current x86 processors. Cell is probably faster than a Core 2 Duo, but against a Core 2 Quad I'm not too sure.

[Atak_Snajpera]

Cell is probably faster than a Core 2 Duo, but against a Core 2 Quad I'm not too sure.

I don't think so :) Real power is in vector units not in main core.

[odditory]

This would be a great project if you had the ability to freeze time, or develop in a relevance vacuum. Unfortunately for this idea, Intel CPU's are roadmapped only to continue getting more and more parallelized, and very aggressively so.

Fast forward to the hypothetical point in the future where x264 is optimized for the Cell processor (years?) - how can you hope to be relevant against the latest x264-for-Intel build that chews through frames at a ridiculously fast rate on multi-socket Intel desktop boards (not even necessarily server boards) with 8 or 16 cores per CPU? Myself and others already encode on dual-cpu quadcore Xeon systems, so 8 core encoding is *already* here, in fact some shops are encoding on quad-CPU quadcore Xeon boards (16 cores) - sure, expensive now, but will only get cheaper with time.

I don't imagine it'll be forever before someone comes up with the ability to grid process (farm encode) with x264.exe either - and I'm not just talking about the guy on the forum here that's struggling with a homebrew solution. In 18 to 36 months or more, let alone today, no one will think that 4 or 8 clunky PS3's networked together is more efficient than a single Intel box with 16 or 32 sum cores.

Even as x264.exe stands today at its current build, it scales VERY well to newer/faster/more cores - almost too well (ie the only encoder that will let me hit 100% utilization all 8 cores of my systems).

Compound all this by the fact that the life cycle for game consoles and computers (and the chips made for them) are shorter than ever - its not like the old days where you had 4-10+ years worth of product lifetime to be able to keep optimizing code. In other words people aren't going to care about the PS3 more than a couple more years before the next big thing diverts their attention anyway.

The time to start this x264 on Cell project was 2-3 years ago, if at all.

That said, I do admire the idea simply for the sake of someone having wondered "what if".

[LoRd_MuldeR]

@odditory
1. The Cell Processor is not limited to the PS3, it's also used in other products (e.g. IBM Blade Servers)
2. The Cell Processor is still under development, there will be future (even more powerful) versions
3. Nobody can prevent you from using your PS3 for x264 encoding, even when it's "outdated" for gaming

[odditory]

@odditory
1. The Cell Processor is not limited to the PS3, it's also used in other products (e.g. IBM Blade Servers)


What a surprise - its used in the servers of the company that makes them - that's about the only company that uses them besides Sony, and Sony only uses them because they practically got them free.


2. The Cell Processor is still under development, there will be future (even more powerful) versions

I'm all for competition to Intel and AMD, but realistically will it happen? Signs point to no. Problem is, the rest of the computer industry has voted with their feet and moved away (far away) from anything to do with PPC/IBM - even Apple finally gave up. I wouldn't hold my breath waiting for any miracle CPU's coming out of IBM any time soon unless it's some IBM exec's pet project that ignores ROI dynamics and wants to anger shareholders.


3. Nobody can prevent you from using your PS3 for x264 encoding, even when it's "outdated" for gaming

True, nothing can prevent me from using a PS3 for x264 encoding, except for a total lack of practicality. Point is, by the time an x264 version for Cell could be implemented/optimized, will anyone WANT to use the PS3 (or a cluster of them) for encoding, given more powerful alternatives at lower pricepoints will exist at that point in the future. Why do you prevent yourself from encoding on the old Pentium II system you have in the garage (so to speak)? Nothing prevents you, except the fact it doesn't make any sense to do so.

[708145]

Certainly, because x264 is not at all optimized for Cell architecture (one main core, 7 Altivec-only cores).

Realistically, I doubt it'll happen, just because so much of the code would have to be drastically reworked to take full advantage of the Cell, to the point where it would result in a fork of the project.

Right, the result would be far from x264 as it is now.
The main showstopper is the small LocalStore of 256KB. Try to partition your code and data into such small chunks without killing efficiency!

I think there is a reason why no project so far made significant progress.
I nonetheless wish good luck to anyone trying to do it for "academic" or "hobby" or just plain "curiosity" reasons.

bis besser,
T0B1A5

[CruNcher]

http://extranet.ateme.com/download.php?file=649 ;)
http://extranet.ateme.com/download.php?file=802

[R3Z]

Surely if Cell is to take off on other platforms then someone like mainconcept or sonic will chime in with support.