Looking for multi-stream SD H.264 encoding hardware

[MuTPu4]

Hello,

I am working on software component for realtime encoding of a bunch of synthesized video streams. Full requirements on target system component are as follows:

Input: 50+ uncompressed (YUV or RGB) video streams of SD quality (640x480, 25fps). May be represented as memory buffers, socket streams or whatever.

Output: AVC-encoded (preferably main profile) video streams of low (like x264 ultrafast preset) or medium (like x264 fast preset) quality. Target bandwidth is about 512 - 1024 kbps. Low latency (< 500 ms) is preferred but may be considered as a trade-off for other requirements. Output representation may vary but RTP streaming seems to be the most convenient way.

I have developed a reference implementation spawning slave VLC process for each stream with x264 as encoder. Quality and other requirements are roughly fulfilled so the main concern is an overall system cost and reliability. I haven't done all necessary testing yet but it seems we need something like one or more multi-processor Xeon PCs (roughly estimating 2 low quality streams per 1 core). Reliability of VLC is also questionable.

I am wondering if there is a hardware solution meeting my specification which may reduce cost or improve other properties of VLC-based solution. The best solution I can imagine is a bunch of PCI/PCIE cards capable of multi-stream SD encoding. I have found feature-rich encoding servers from HaiVision, Hivionics, ATEME, AdvancedDigital, TANDBERG, Elecard etc capable of multi-stream encoding but all of them seems heavy-weight and far to expensive.

Can somebody give me a hint on searching cost-effective solution for my problem?

Thanks.

[Dark Shikari]

x264 can easily do 10-20 SD streams on a single Core i7. 2-4 Core i7s are far, far cheaper than any decent hardware solution.

Remember the rule of hardware; there are four types:

1. Cheap, featureless, terrible compression.
2. Expensive, mediocre features, slightly less terrible compression.
3. Really expensive, good features, mediocre compression.
4. Really expensive, good features, pretty good compression.

;)

Do also remember that most hardware solutions have much worse compression than x264, even on "veryfast", let alone "fast" or "medium". Furthermore, low latency is often a feature that cheaper systems won't have, while x264 has a large set of low latency-streaming features (http://x264dev.multimedia.cx/?p=249).

[MuTPu4]

x264 can easily do 10-20 SD streams on a single Core i7. 2-4 Core i7s are far, far cheaper than any decent hardware solution.

Thank you for your encouraging answer. I am going to continue evaluation of x264-based solution on dedicated i7-950 machie and if it is capable of handling of more then 10 simultaneous streams it would be the most promising solution.

[Blue_MiSfit]

Instead of an i7 950, consider a dual Xeon 5520 box. This is basically the cheapest way to get 2 Core i7 CPUs for 8 physical and 16 logical cores. Plus, you can but hordes of RAM in it so you can run 20 streams in real-time without worrying about memory usage! A 64 bit OS is obviously mandatory, in any case :) You can get a box like this for under 2 grand...

~MiSfit

[mariush]

299.99$ x 1 http://www.newegg.com/Product/Product.aspx?Item=N82E16813151085
199.99$ x 4 http://www.newegg.com/Product/Product.aspx?Item=N82E16819105235
29.99$ x 4 http://www.newegg.com/Product/Product.aspx?Item=N82E16835114078
109.99$ x 8 http://www.newegg.com/Product/Product.aspx?Item=N82E16820134936
279.99$ x 1 http://www.newegg.com/Product/Product.aspx?Item=N82E16811165196
154.99$ x 1 http://www.newegg.com/Product/Product.aspx?Item=N82E16817815003
------

2535 $ for 4 x Quad Core 2.1 Ghz, 16 GB of memory. Add hard drives.
So you have 16 cores that can probably do 3-4 simultaneous 640x480 SD encodings per core for a total of about 40 and the motherboard gives you 3 gigabit network cards so you could stream uncompressed videos directly.
Will probably beat any hardware solution and dedicated hardware equipment.

ps. this particular model of motherboard has an extension module that you can use to add 4 additional processors for a total of 8, so you'd have 8 x quad core processors. you'll need 2 x 850w power supplies though, or at least one 1700w power supply according to the reviews on the product page.


ps/disclaimer. I did not check if the motherboard is compatible with the processors and the memory I selected. What I wrote is just a suggestion, it's your duty to check the motherboard manual and see what's supported or not.

[atl]

199.99$ x 4 http://www.newegg.com/Product/Product.aspx?Item=N82E16819105235

This processor will not work with mentioned Motherboard.
You need 8300 series Opteron for 4 x socket Mobos, but those processors are highly priced.

Advising to stick on 8 (2 x CPU) core Xeon or 12 core (2 x CPU) Opteron

[mariush]

hmmm... that's why i added the disclaimer there, I knew the prices were a bit cheap... I remember when I bought a xeon processor for a dual cpu motherboard and was about 600$.... the processors in the 8xxx series are about 1500$ and up: http://ncix.com/products/index.php?minorcatid=106&subminorcatid=172

So i guess you could use for example this motherboard:

http://ncix.com/products/index.php?sku=36353&vpn=MBD-H8DA3-2-O&manufacture=SuperMicro

http://www.supermicro.com/Aplus/motherboard/Opteron2000/MCP55/H8DA3-2.cfm
http://www.supermicro.com/a_images/products/Aplus/MB/H8DA3-2_spec.jpg

at around 560 Canadian$ with two of these http://ncix.com/products/index.php?sku=36004&vpn=OS2350PAL4BGH&manufacture=AMD at 339 Canadian $ each

again disclaimer/nitpick/whatever:

i'm picking these because they're both available at a well known online store and I assume the motherboard supports the cpu because there's no cpu support list on the supermicro website, though it says the MB supports all 2000 series quad cores and six cores (with a bios update

also, i don't have any preference for amd vs intel, it's just that i started with amd in my previous post so i keep going with it

later edit.. ok, let's add intel alternative..

384.99$ x 1 http://www.newegg.com/Product/Product.aspx?Item=N82E16813131373
384.99$ x 2 http://www.newegg.com/Product/Product.aspx?Item=N82E16819117185

(yes, they're the same price. of course, you have to hunt for coolers as the processors don't come with it)

[Blue_MiSfit]

My advice for this kind of solution would be to buy something from Dell / HP. They are priced very competitively, are well supported, and "just work" out of the box. Building a server with off the shelf components requires extremely careful scrutinizing of the components to ensure perfect compatibility etc..

I'd say stick with Intel CPUs. The i7 is a real x264 monster.

~MiSfit

[kolak]

My advice for this kind of solution would be to buy something from Dell / HP. They are priced very competitively, are well supported, and "just work" out of the box. Building a server with off the shelf components requires extremely careful scrutinizing of the components to ensure perfect compatibility etc..

I'd say stick with Intel CPUs. The i7 is a real x264 monster.

~MiSfit

Yep- HP workstations are powerful, well designed (good cooling system), very stable and just work :)


Andrew

[atl]

http://supermicro.com/products/system/1U/6016/SYS-6016T-NTF.cfm

[Blue_MiSfit]

Supermicro is indeed an excellent alternate vendor. They scale into 4 (and more) socket systems well also, but that gets pricey ;)

~MiSfit

[TEB]

Im looking at something similar at work where i have 1 HD-SDI stream input and i wanna try x264 as a RT encoder for 1080i. My biggest consern is how to do this with audio in synch and output the lot as a Transportstream in mapping it to a UDP/Multicast stream. Anyone have a suggestion on how to design this?

br TE

[Blue_MiSfit]

I Imagine the difficult part will be getting the HD-SDI feed somehow into ffmpeg or a similar tool - which I know nothing of :)

Once you get it into there (probably as a stream of uncompressed audio and video), you should easily be able to specify encoding settings, and store the output in a transport stream, or pipe the output to some kind of streaming software.

~MiSfit

[TEB]

Good point, but i was thinking of using a osperey hd-sdi card or similar and piping audio and video to the encoder..

[kolak]

Good point, but i was thinking of using a osperey hd-sdi card or similar and piping audio and video to the encoder..

BlackMagic will be cheaper and easier to implement. There is company called Medialooks which does custom filters for BlackMagic cards.

Andrew

[TEB]

Awesome, ill look into em..
Its just a little experience to feed my curiosity on h264 quality (x264 vs Tandberg and Motorola encoders i have at work)..

[MuTPu4]

Thank you everybody for your advices.
We will closely consider multi-processor Xeon 5520 systems and similar solutions.
For now I have done some naive tests to estimate multi-stream capabilities of Core i7-950 in comparison with my Athlon X2 7850. I haven't manually configured x264 much but I assume that this is enough for rough estimation:

profile main
preset ultrafast/veryfast/faster/fast
bframes 1
keyint 25
keyint_min 5

I have used two synthesized video fragments one with highly-dynamic and one with lowly-dynamic content which comprise the interval of the expected number of simultaneous streams:

preset/CPU Core i7-950 Athlon X2 7850 Speed gain
ultrafast (22, 40) (8, 11 ) (2.8, 3.6)
veryfast (18, 30) (5, 8 ) (3.6, 3.8)
faster (9, 15) (2.5, 4.5) (3.6, 3.3)
fast (6, 10) (1.5, 3.5) (4, 2.9)

Multi-stream capacity scales almost linearly which gives me hope that one quad-processor or two dual-processor systems may be capable of encoding up to 50 streams simultaneously with acceptable quality (around faster preset after some tuning).
I have also tried CUDA encoders with slightly surprising results:

encoder/GPU GeForce GTX295 GeForce GTX260
NVCuVEnc (16, 20 ) (6.5, 8.5)
Badaboom (4.5, 6.5) (4.5, 6.5)

Final quality is around veryfast-faster presets and encoding speed is comparable with x264 on Core i7. Also Badaboom does not scale with better GPU. I assume CUDA encoders are not intended for my low-bitrate settings and target quality.

[popper]

Im looking at something similar at work where i have 1 HD-SDI stream input and i wanna try x264 as a RT encoder for 1080i. My biggest concern is how to do this with audio in synch and output the lot as a Transportstream in mapping it to a UDP/Multicast stream. Anyone have a suggestion on how to design this?

br TE

well x264 will be getting Audio, and Transport Stream capabilitys soon, it depends when <kierank> over on Freennode #x264dev gets around to adding TS to it OC, perhaps by Q2, you should get over there,hang out, and give them a helping hand if you know this stuff.

how Does this generic HD-SDI stream output ?, some form of V4Lv2 !, then perhaps someone should take the time as a side project way to get into, and understand the main x264 code, and add that V4Lv2 and related updated RFC's as another input for x264, given its perhaps soon ready for a mass of industial uses too sooner or later.

http://www.spinics.net/lists/linux-media/msg09797.html
http://www.linuxtv.org/downloads/video4linux/API/V4L2_API/spec-single/v4l2.html

http://www.linuxtv.org/hg/v4l-dvb

http://x264dev.multimedia.cx/?p=270
"01/17/2010 (10:40 pm)
What’s coming next in x264 development
Filed under: development, low latency, ratecontrol, speed, streaming, x264 ::
As seen in the previous post, a whole category of x264 improvements has now been completed and committed.

So, many have asked–what’s next? Many major features have, from the perspective of users, seemingly come out of nowhere, so hopefully this post will give a better idea as to what’s coming next.

Big thanks to everyone who’s been helping with a lot of the changes outside of the core encoder.

This, by the way, is one of the easiest ways to get involved in x264 development; while learning how the encoder internals work is not necessarily that difficult, understanding enough to contribute useful changes often takes a lot of effort, especially since by and large the existing developers have already eliminated most of the low-hanging fruit.

By comparison, one can work on x264cli or parts of the libx264 outside the analysis sections with significantly less domain knowledge. This doesn’t mean the work is any less difficult–only that it has a lower barrier to entry.
.... more snip
"

[Blue_MiSfit]

CUDA quality similar to --preset veryfast???? Last I heard, CUDA encoders were beyond terrible. I would call performance like this "not so terrible"!

I think the CUDA stuff is useful for quick transcodes for portable devices etc, but not much else. I'd be interested to see some test encodes, if you can post them

~MiSfit

[TEB]

Hi. Evedently more is happening in x264 that i even could dream of..
If someone as u say are interested in implementing DVB-TS support internally, AAC and Mpeg1-L2 support and would be the start on making the worlds first opensource RT H264 encoder.
If we could implement some kind of v4l input support and UDP multicast read support and multicast output support that would be even better ;)

[nm]

If someone as u say are interested in implementing DVB-TS support internally, AAC and Mpeg1-L2 support and would be the start on making the worlds first opensource RT H264 encoder.
If we could implement some kind of v4l input support and UDP multicast read support and multicast output support that would be even better ;)
Well, VLC already does most of these things.

[MuTPu4]

CUDA quality similar to --preset veryfast???? Last I heard, CUDA encoders were beyond terrible. I would call performance like this "not so terrible"!
Bear in mind that I am using low-bandwidth (640 kbps) and low-latency settings.


I think the CUDA stuff is useful for quick transcodes for portable devices etc, but not much else. I'd be interested to see some test encodes, if you can post them

I am not sure if I may post sample images maybe I could reproduce my results later on some other content.

[Dark Shikari]

Bear in mind that I am using low-bandwidth (640 kbps) and low-latency settings.Why would you use low-latency settings? You're not doing low-latency streaming as far as I can tell.

Furthermore, does Badaboom even have a comparable low latency mode? I don't recall that it supported very small VBV or zero input delay, which would make such a comparison rather unfair.