Someone help me with RAIDs?

[gia257]

Hello, I don't know a lot about hardware or electricity. If anyone could confirm or correct what I think I would really appreciate it.

I got a "new" computer that uses GA-EP43-DS3L mainboard. It has six SATA 1 ports and one IDE port, besides it's got a PCIe x16 port which I use with a HD4850 video card, four PCIe x1 slots and two PCI, all these are empty.

It has a SATAII 250GB Seagate hard drive.

I have so far recorded old pc games with fraps, sometimes it drops frames but its not often at least not noticeable. However I tried a newer game recently, and frames dropped left and right. Besides this problem I wanted to capture tv and my game consoles in "perfect" quality. So I found and read about RAIDs.

To my setup I want to add two video capture cards, one is Sweetspot which I'll use with my old consoles/tv, It says it reuqires a PCI v2.1 port (no idea if my mobo's PCI is 2.1...). And the other is MonsterX which requires a PCIe x1.

I read my mobo doesn't have a bundled raid, so I must buy a card for that. However I do not understand what to look for when buying. I know I want raid 0, and that besides that I want a separate disk to keep my OS and non-video-related files.

Now I dont get if you are supposed to plug the hard drives to your raid card then the card transmits the data through their PCI connection, or if you connect each part to the motherboard or whatever they do. I'm not sure if I can have a disk separate from the RAID and connected to the motherboard either.

Anyway I see cards for PCI, PCIe x1, PCIe x4, x8, x16. I only have x1 available though. I read that slot has 640MBps so I'm assuming that's the max bandwidth the raid would have.

Gamecube's maximum video is 480p, or 704*480 at 60fps, let's say 24 bit color? that would be 58MBps without audio. PCIe x1 is ok with that.

The pc videogame I wanted to record is 1280*1024, I have windows set to 32bit color and fraps I set on 60fps, so I guess I need 300MBps again without audio, but fraps is not lossless so I need less than that. Still ok for PCIe x1.

So am I correct that a PCIe x1 raid card suits my needs? Or am I missing something and x16 is the only one that would meet the speed I want (or maybe PCI is enough).

I also read a single SATA1 7200rpm drive has a writing speed of 60-70MBps and SATAII doubles it, so to meet the 300MBps I'd need five drives on the raid since my mobo only supports SATA1.

Or does the SATA thing is handled by the card so if I buy all SATAII I'd get SATAII speeds regardless of the mobo? (In that case I'd need 3 drives I guess).

What does external and internal connectors, channels, lanes etc mean? I have read some cards don't support this or that, which card supports five (three?) big drives (1.5TB?) on raid 0 using a PCIe x1? or am I doing it wrong?

And now I recall fans and power supplies... I see two fans, one to the side of the case and another on the back, they are the bundled fans, are these enough for this considering I got the video card too? (right now the bundled program says 42 celsius on the GPU, I can't find a program to check the CPU, it is an E7200 C2Duo). Or do I need two more fans put anywhere there's room? :P

Will all this kill my power supply?
I got a RP-500-PCAR, the specs say:
1 x Main connector (20+4Pin)
1 x 12V(P4)
6 x peripheral
3 x SATA
2 x Floppy
1 x PCI-E
Do I need something much bigger so I don't damage any part?

The pc is connected to a regulator: http://www.digssa.com.mx/productdetails.asp?id=5, 1006i, the pc on the Bypass slot (2 slots) and the screen on the AVR Outlet (4 slots), no actual reason for doing so, just guessed the one with fewer is a better slot. I wanted to also connect the consoles to it (with a 150W transformer). Will this thing not be enough for whatever it does if I add all that stuff? Would the wall socket be enough? :D

Thank you very much for any insight.

[TEB]

Hi. I can understand that diving into the HW domain can be a little daunting but fear not ;)

1. Most motherboards today has a onboard raid controller of sorts. They are basically SW/CPU driven and allows u good raid 1 or 0 performance, and terrible writing performance on raid5 or 6.
U could get a promise fasttrack or 3ware controller but it will costs u 200-300$.
Alternativly u can do software raid which also worksdepending on the cpu usage ure codec uses with the capture program.

My suggestion is to buy 2-3 500gb SATA drives, put them in sw raid 0 and keep ure existing drive as OS drive.

[Adub]

[begin linux plug]

zfs ftw!!!

[/end linux plug]

[gia257]

@1. I know my motherboard doesnt have raid, or is "software raid" a program, does it have a name?
200-300$ sounds good if it satisfies 300MBps capture speed, thanks for the brand names, I'll check them, but I bet they have 10 cards under the same brand so I won't know which to choose

@linux but the pc games are for windows

Any more insight on any of my doubts is always welcome :D

Btw, I checked my pc again and its three fans, one on the cpu, and goes to the side of the case, one on the video card and points down, and another on the back of the case.

And my current hard disk is 7200.10 250GB Seagate Barrracuda. Are there a special brand of hard drive for capturing video? I know it has to be a big disk (1TB) but I read some brands are best at something and not the other.

[von_Runkel]

I have recently done some research on SATA disks in various raid combinations on systems like yours. The goal was to achieve maximum read and write throughput for uncompressed video. In my particular case, this has been my bottleneck for years. My setup is a dual quad-core with an onboard Intel raid-controller with four SATA ports. I use the hardware striping (raid 0) with three 500 GB disks and the fourth port is used for system and boot.

However, the term hardware raid is seldom valid these days since true hardware raid cards (where the raid controller handles all logic and completely offloads the CPU) are insanely expensive and you find them in high-end servers and dedicated storage installations, so called SAN (Storage Area Network).

The mainstream raid controllers installed as separate cards, or - as in my workstation - embedded on the motherboard uses a lot of CPU cycles when disks are accessed. This has been a trend for some years because normally you have a huge surplus of cpu cycles in a dual or quad core machine. It is unusual that one need both full CPU power AND disk speed at the same time, unless you need to do capture and encoding in the same pass. All medium sized/priced file servers I have administered the last three or so years utilize 30-50% CPU in a raid 5 setup when max throughput is reached in write operations. As TEB said raid 5 or 6 is very slow and CPU intensive in write operations.

The raid cards that I tested was Adaptec 2820SA 8-port, LSI Logic SAS3442 4+4 port and Silicon Image 3124 4 port. All are PCI-X 64-bit cards.

All three cards use way to much cpu to be called "hardware" raid controllers. Especially Silicon Image 3124 in a raid 5 setup with 4 disks hogged so much CPU that the test machine (dual 2.8 GHz Dell 2850 server) almost froze!

Even in a raid 0 config the cards consumes cpu cycles on mass. The LSI Logic card is now installed in a production server with two raid 0 volumes, two 500 GB disks each. When I saturate the disks with R/W from the network (it has two NICs), the CPU goes up to 50%. If I chose software raid 0 (striping in Windows) on the same machine, the results are the same circa 50% CPU and almost identical transfer rates.

Transfer rates are by nature very difficult to measure because it is very hard to tell were the bottlenecks are. It can be the disk, the controller, the data bus or anything else in the chain. If you want to benchmark different disk setups, be sure to use the same benchmarking software, otherwise the results are useless. I use IOmeter which is freeware and has tons of features (http://www.iometer.org/doc/downloads.html)

When you study HD data sheets, transfer rates of 300 MB/s are often mentioned. Do not believe in those figures, they are just the theorethical transfer rate for the SATA II bus. 50-70 MB/s are more realistic for a 500 GB disk and 80-90 MB/s for a 1 TB disk (higher density=higher speed).

My suggestions are almost the same as TEBs AND the reservation for that you do not mention how CPU intensive your capture codec is!

Three 1TB disks in a software stripe volume. Connect the disks to the mobo SATA connectors and set up a 3 TB volume in disk manager (a very easy task). This will give you plenty of headroom for your captures. Connect the existing 250 GB to a fourth SATA port and use it for system, boot and software. Be sure to enable write cache on the striped disks. Last but not least - as you surely know - a striped volume has no fault tolerance at all, if one disk fails, the entire volume fail.

Your mobo DO support SATA II,
http://www.gigabyte.com.tw/Products/Motherboard/Products_Overview.aspx?ProductID=2847

That does not mean (as you indicated) doubled transfer rates. One of the benefits of SATA II is better "traffic" handling. It can queue more commands and make smart decisions of how to serve the command queue in the shortest time. This will save you CPU cycles.

There is (to my knowledge) no special hard drives for capturing video. Some HD manufacturers have special "Raid editions" They are not faster than other disks but designed for 24/7 uptime servers and sometimes comes with extended warranty. And a higher price tag.

It is almost impossible to recommend a special brand or model. All I can say is that I am very happy with my newest investment for my entertainment computer, two 1 TB Seagate Barracudas. Impressive transfer rates to be a budget drive. Be sure though to remove the tiny jumper that disables SATA II on the drive...

[gia257]

Cool, I'd use 1TB disks I guess, I'd need the capacity anyway. What was your write speed on your tests? Did you reach 350MBps?

I checked the manufacturer's site before, it says SATA it doesn't say SATA II. My hard disk is SATA II anyway.

I usually want to only capture, using huffyuv at most, or plain raw. But also want to capture 3d games with fraps which uses its own codec, I imagine the games are cpu intensive. I have decided I need 350MBps actual bandwidth for writing to disk and at least 4TB of capacity on the raid, hopefully 8TB. I still dont know how to achieve that :(

[turbojet]

You might want to check out this matrix chart (http://www.tomshardware.com/charts/raid-matrix-charts/Average-Read-Transfer-Performance,218.html) when deciding on number of HDD"s to use. You will need quite a bit to get up to 350 MBps read or write.

Also if you are planning on doing software raid I have noticed XP with any SP bottlenecks RAID0 while Vista/Windows 7 don't. For instance 3x Seagate 7200.12 r+w on same RAID0 is ~60 MBps on XP SP3 x86 and XP SP2 x64. On the same hardware Vista SP2 and Win7 RC1 r+w is ~100 MBps. One thing that's commonly looked at as a downside of software RAID0 is when you format you lose the RAID but in the case of moving betwen XP/Vista/Win7 this is not true, all OS's can see the same RAID0. However there is no RAID5 support in software, only RAID0 and RAID1. In this case one thing I often suggest is buying 3-5 large drives, creating a small partition on one drive for OS, if you need another OS partition create on another drive. Then use the spare for software RAID0. It saves the price of having a dedicated OS drive and this way with 3 drives gives you roughly the same throughput as you would get from having an OS+3 drives.

If you aren't going to use RAID5 or don't need to use the raid between linux/mac and windows I don't really see a need to buy a raid card in your situation. It's possible to do a software raid between linux/mac and windows but you need to create it in linux/mac and I know linux does and as far as I know mac loses raid info when OS is wiped.

[gia257]

Thanks but you missed the point. I only want a minimum of 350MBps write speed for 4TB of disk under windows.

Those charts don't say much, you linked to the read transfer, write transfer doesn't reach 300MBps, I don't see what did they use for the raid anyway, from the specs popup that for some reason doesn't seem to update when you click on another, looks like those charts are outdated (I keep seeing <500GB drives).

[MfA]

Windows 2K, XP Pro and Vista Ultimate/Enterprise have something called Dynamic Disks, which supports striping which which is essentially software RAID 0.

[turbojet]

It shows that in RAID0 adding another drive doesn't produce 100% throughput of the drive.
2x RAID0 about 175%
3x RAID0 about 240%
4x RAID0 about 275%
5x RAID0 about 300%
6x RAID0 about 320%
compared to a single drive in an OS software and soft/hard raid card that von_Runkel explains.

Here's a single drive throughput chart (http://www.tomshardware.com/charts/2009-3.5-desktop-hard-drive-charts/h2benchw-3.12-Avg-Write-Throughput,1013.html) which is accurate for low level transfers with tools such as h2benchw but in your situation with windows you should reduce it by about 5-10% for OS overhead.

In order to reach 350 MB\s with the fastest SATA HDD out atm, 7200.12 500 GB you'd need probably 8-10 HDD"s in a RAID0. I've done some tests with these drives and 3x RAID0 is about 230 MB\s, adding a fourth resulted in 260 MB\s with HDTune that takes windows overhead into account.

However 300 MB\s seems like a really high bitrate for 1280x1024 at 60 fps video. On the fraps site they have a 1280x720 at 60 fps example and its 7.5 MB\s. I can't imagine 1280x1024 resulting in much higher bitrates and any modern single HDD should be capable of doing it.

If you really need 4TB what I'd suggest is buying 4x 1TB HDD"s, setup a software RAID0 and see how it goes. If it's still not capable of doing what you want add another 1TB HDD.

[gia257]

Maybe you are correct about fraps, if you notice up there I did a calculation on how I ended up wth 350MBps, and I was considering a lossless recording which fraps isn't, still that's a very high compression. Probably it compresses that well on certain games but others don't.

Im still at a loss of what hardware to buy :(

[Shinigami-Sama]

you sure you weren't just looking at 350mbps? not MBps?

[Blue_MiSfit]

I'm almost sure he's talking about 350mbps... 350 MBps is more than you need for 1080p uncompressed YUY2. FRAPS is much much much much much smaller (350mbps == 43.75 MBps)

I think a single fast drive would be fast enough (something like the WD Caviar Black 1TB), or a 2 disk software RAID 0 would be more than enough. Either using your motherboard's RAID logic (very basic), or Windows' "Dynamic Disks" would be fine.

No need for hardware RAID in this case. Also, I think it's insane to not buy 1 or 1.5tb drives, as they're stupid cheap these days. Why waste time with low density drives like 500gb???

I ran into issues with FRAPS when capturing due to insufficient CPU power (on a Q6600 @ 3 GHz ;)) I was trying to get 1080p at 60fps while playing GRID, and couldn't get decent performance...

~MiSfit

[Shinigami-Sama]

my WD 1TB black caviers sustains 70MB/s disk to disk
three of them should be just over 300$ with shipping(96$ CAD @ ncix)
if should get at least 160MB/s
you're set for pretty much anything

[TEB]

U could get 10x SSD disks and you would be golden ;)
But seriously, my suggestion is:

9650SE-4LPML



http://3ware.com/products/serial_ata2-9650.asp
and 4x SATAII Disk depending on size and run it in RAID0