mac & kona3 & G-Tech 4-Port, PCIe x4 RAID Controller & Quadro FX 4800 for Mac

[smok3]

- The idea is to setup fast/simple/efficient standalone editing stations, they need to provide prores HD editing and uncompressed SD stuff.
- The ability to use tape ingest is also crucial (ingest and frame accurate playout to machines like sony betacam IMX and similar (thats expected from kona).
- The ability for an editor to select editing suite (adobe production premium or final cut pro studio or later maybe autodesk smoke)
- The ability to preview anamorphic SD and HD version while editing (in both adobe and apple suites) (expected from kona)
- The ability to accelerate stuff in CS5, expected from nvidia and kona3.

questions:
a. will
kona3 & G-Tech 4-Port, PCIe x4 RAID Controller & Quadro FX 4800
play fine together in a single system
b. what would be a simple cost/effective backup strategy (local or network? if any of this, what kind? etc)
c. how to share files between workstations (the existing two stations are running smoke linux systems) - must be simple to administer, must be fast
d. objective video signal measurement? external, software based, what? (together with quicktime gamma weirdness)
e. file based workflow for final stuff (should kick-in at the same day when we finnaly realize tapes are boring).

[Blue_MiSfit]

A) They certainly should. I'd look into copying whatever turnkey NLE vendors are using :)

B) Backup is a tricky question. How much storage do you need?

C) Setting up a simple iSCSI SAN is what I would suggest. Again, how much storage do you need?

D) I'd suggest something like the BlackMagic Ultrascope. We have tons of them at work, and are a fantastic, much cheaper alternative to a hardware scope.

E) My company is all file based. Tapes are only used for ingest into a variety of mezzanine formats. ProRes is a good mezz format for most, depending on your application. DNxHD is preferred IMO, due to better performance outside of Mac OS X.

[smok3]

Blue_MiSfit, thanks for replay

a. turnkeys seems to be based on ATi (possibly due to apple color? we almost certainly wont be using color, so not big deal, but still not sure if i can stuff everything in there...), i did a lot of reading about AJA-io to put at least that outside, but it doesnt look like a good replacement for kona3, basically everything is then tied to prores as it seems.

b. infinite, say each lil project is between 40->100 gigs (i was thinking about large quantities of small usb2/fw 800 500 gigs drives, one drive will be full in about a month, so thats like at least 5 drives per month, seems costly...) and that still can't really be called a backup.

c. can iscsi box use ubuntu and typical ssh/ftp/www servers? (and possibly netatalk here as well), since this is temporary storage, i'd say about 2 tera will do for a while (goes in hand with a tricky backup i guess)
-----
edit: seems complicated and it's just not abstract enough, i think fast 10g router + ftp/ssh/netatalk ubuntu box would be much easier and much more abstract (any other box can access it via one of those protocols).
edit2: basically anything that needs daily administration is worse than tape workflow imho (seems true for apple xsan2 as well) while for example my current ubuntu box is sshing and ftping with 70 days of uptime right now (and besides initial setup i did not touch it since)

e. i do expect at least 4 formats to happen during the chain, for example 1. shoot to AVC (or xdcam), 2. edit in prores, 3. broadcast playback box uses XY + 4. typical mpeg4 compression for broadcasting, so basically/possibly i can beaf-up the ISCSI box and send edited files there (posibly some sort of uncompressed stuff), what do you think?

(basically i would try to build an island around a messy input/output options)

[Blue_MiSfit]

I'd suggest you think MUCH bigger in terms of storage. I always think in terms of terabytes now. No point getting anything less than a 2TB drive, honestly!

A) Hmm.. I was thinking more in terms of which RAID cards turnkey workstations use. I know the Mac Pro can sometimes be finicky with various combinations of PCIe devices. Kona 3 is a fantastic card. If you can afford it, get it! The only alternative IMO is the BlackMagic Decklink / MultiBridge cards (I like the latter). I have the most FCP experience with the AJA stuff though. VTRExchange isn't half bad either for quick captures. Since you're on mac though, what's wrong with being tied to ProRes? :Devil:

B) Here's how I think about storage:
* The cheapest option is always bare 3.5" SATA drives. This, combined with eSATA hot-swap docks gives you a really, really affordable way to expand storage. It's important you keep track of which projects are on each disk! Label them, and keep them safe!
* Moving to any kind of NAS / SAN will be AT LEAST 4x more expensive than the raw storage. For example, I can buy 1PB of 3.5" HDDs for about $75k. The cheapest equivalent in NAS storage (from Digiliant who uses high density 4u supermicro chassis) costs about $300k. The difference wasn't worth it to me! Note that at this price point, you're basically addressing "pods" of storage, and not a unified file system, so it's not really much better than bare drives.
* High performance "true SANs" or uber huge clustered file system NAS's are 8-15x the cost of raw storage (I've seen $600k to $1.5M+ for 1PB).

Conclusion: Bare drives are awesome, but I never work with uncompressed media. Everything I need (bandwidth wise) fits nicely into the performance envelope of a modern SATA drive. Your story may be different!

It does sound like you need a decent pool of online, reasonably high performance storage (enough for 2 simultaneous 10 bit 4:2:2 1080i uncompressed streams I'd bet). I think a nice cluster of 2.5" 500GB drives would handle this very nicely, connected over 10 gigabit ethernet. I'd use OpenSolaris or FreeNAS. With either you get the ZFS file system, and a nice iSCSI target, plus backwards compatibility with NFS, CIFS/SAMBA, or AFP.



Derek

[smok3]

eSATA hot-swap docks

are you actually using one of those on a esata-unfriendly mac? if yes which one? does it actually hotswap without reboot? or should i consider fw800 hot-swap dock of some sort..?

I'd use OpenSolaris or FreeNAS. With either you get the ZFS file system, and a nice iSCSI target, plus backwards compatibility with NFS, CIFS/SAMBA, or AFP.
and will turn me into geeky network sysadmin with big glasses?

[Blue_MiSfit]

I'm using these docks all the time! They're AMAZING! Get the Thermaltake BlacX, with a single drive slot. It works flawlessly with Mac OS X using a proper controller card that has good drivers and fully supports hot swap.

Vanilla Silicon Image 3132 chipset based cards have worked well for me, but you might want to go sonnet or one of the "mac" companies to get something guaranteed to work with 10.6 etc... :)

FreeNAS is super easy. You do all the configuration via a web interface (like a router). OpenSolaris isn't very difficult either, but it's more like a pure desktop / server OS. The good news is, there's excellent documentation and support, much like x264 / avisynth etc :)

Derek

[smok3]

docks: so thats either esata, which will need another internal slot taken, dont like it, or usb2 which will be slow..., are there any fw800 versions of this docks out there? (i'am afraid to ask google again)

Bare drives are awesome, but I never work with uncompressed media. Everything I need (bandwidth wise) fits nicely into the performance envelope of a modern SATA drive. Your story may be different!
yes, i don't see how uncompressed HD workflow would look like here, i guess the workflow could be offline (prores) and online type, at the end only offline is backuped and the next incartaion of the project will simply have slightly lower quality (seems like a good educational project as well then ;P)

edit: what would be the feature of iscsi to go over AFP? (as main protocol)

[Blue_MiSfit]

I think you have enough slots.

1) GPU
2) RAID
3) eSATA
4) KONA 3

As far as uncompressed HD goes.. why bother? :) ProRes is great!

iSCSI is nice because you can get better performance than AFP / CIFS / NFS, and you can create a separate physical network for storage-only.

Derek

[cacepi]

edit: what would be the feature of iscsi to go over AFP? (as main protocol)
Your two workstations are running Linux, so avoid AFP like the plague. AFP runs like a dog on Linux/*BSD, if at all.

Really, just stay away from AFP. You're asking for a world of trouble. Stick with NFS and a dedicated server.

If you're sharing files amongst Macs, I have one word for you: Xsan. It was made specifically for this. Use it, love it, rinse, repeat.

[smok3]

the machines that stay are two smoke systems (red hat), 1x pc editing station with dedicated network for storage (part of a bigger picture) and two new macs, the red hats are basically tape workflow, meaning backups are done to digital betacams, ect, anyway its a huge mess and i can see now that networking stuff is way over my head....

[foxyshadis]

FreeNAS is super easy. You do all the configuration via a web interface (like a router). OpenSolaris isn't very difficult either, but it's more like a pure desktop / server OS. The good news is, there's excellent documentation and support, much like x264 / avisynth etc :)

Derek

btw, unless you're running OpenSolaris for other reasons (and there are good ones), NexentaStor would be better, if slightly more immature than FreeNAS. Support's there and you don't have to drop to the command-line to get advanced features, it's simpler to set up and manage imho. The main thing that's tricky to do is to get a real shell. I mostly use it as an NFS server for VMs, with dedup - my next big hardware project will be recreating my htpc on top of Xen running on Nexenta w/ flash and 2tb drives. It was the only way I could think of to get full performance to it without some crazy 10gig or fiber install.

[Blue_MiSfit]

foxy,

I'm probably missing something here, enlighten me?

Why use visualization if an HTPC requires a physical machine?

[Blue_MiSfit]

Thanks for the tip though, foxyshadis. I'm checking out Nexenta right now, looks cool - if I can get the right hardware sorted. The spare oddball PCs I had lying around didn't exactly play nice with it :)

Derek