NETWORKING A SMALL CS6 STUDIO OVER GIG E - WILL IT WORK

[Anson Fogel]

All,

We are a small studio, with 4 full-time editors, a dedicated IT/Asset Manager staffer, and 2-3 other staff working on color, sound, graphics and creative. 7 OSX Workstations (towers and iMacs), primarily running CS6 in Mountain Lion, cutting a mix of TV spots, web shorts, and short and long form docs. We migrated to CS6 from FCP a few years ago. We generally edit ProRes online 1080P, and shoot RED, 5D, Alexa and everything else. We store all of our projects and footage on 4 large SAS RAID5/6 boxes, connected to a single 2010 Mac Pro, and we archive on LTO5 tape.

Our current network is sneakernet - we all work on smaller local raid's, and manually shuttle files and projects back and forth from the central "store." We can't afford fiber, or big purpose built servers, and 1GB E is too slow. Complex proprietary systems don't appeal to us. So, our system works. Primitive. Effective. Inefficient.

It looks like 10Gb Ethernet has come down in price, and increased in performance, to the point where we can spend less than 10 grand, and all work on projects on the server(s) over the network rather than on local drives. Tests here, and research, are promising. To be specific, we would all use CS6 (primarily Premiere) with the project and all associated media living on the server(s). No local files. We tested this over 10GB E, and it works great with 1 or 2 clients and the current Mac Pro/SAS host. But we can't test more than 2 clients without buying a lot more equipment.

The question for you all - have any of you done this? Experiences?

 

[Anson Fogel]

Bueller? Bueller?

 

[jake blackstone]

Anson.
In your post you're talking about 10 GigE but the name of the thread is GigE. It could be the reason why you are getting no response.

 

[Anson Fogel]

Anson.
In your post you're talking about 10 GigE but the name of the thread is GigE. It could be the reason why you are getting no response.

Good point Jake, hope you are well man thanks...

 

[David Kuhn]

My labs have 10gig to the data center with gigabit or 2x1gig client side (Cisco switches)... that's sufficient to provide for decent file access, but noticeably slower than ssd or local array (it's faster than local hdd though). I've not seen many client machines or apps that need or could use 10gig... but editing 5k is probably one of the exceptions. The problem you'll have is that if your 7 users are actually USING their 10gig drops your going to be hard pressed to put together a server option that can keep up. I think a more balanced option would be 10gig to the server & dual gige to the clients. Have you thought about a high end storage array? EMC has some nice hardware for a reasonable amount of money - I've got a couple of their VNXe series arrays and they're fast, reliable, and don't crush your bank account.

 

[Jeff Kilgroe]

It's all in the switching hardware. Do your homework! There are some relatively inexpensive options coming around right now, but they may or may not fit your needs. The Cisco/Netgear ProSafe 10GE switches are the most affordable ones out there, but they don't have the internal bandwidth to sustain all their ports under load. That and they're built very cheaply and have fans inside that sound like a small jet at takeoff.

As a good rule of thumb when putting these systems together, plan on 75~80% efficiency in relation to all advertised numbers. You also need to temper that with the hidden numbers that the manufacturers don't like to talk about -- aggregate and internal bandwidth numbers and such. The 8-port NetGear ProSafe 10GbE switch can connect 8 devices at 10G and one of the ports is SFP capable, but it can truly only sustain 3 of those ports, 4 tops, if you have multiple clients hammering on it at the same time.

Consider your storage back end as well. How are your NAS or SAN nodes going to attach to the network? The ports themselves top out around 850~900MB/s in very ideal situations. Some of the best affordable NAS solutions, like the Intel RAID boxes with 10G connectivity, have an onboard RAID host that tops out in the 550~600MB/s range.

 

[Stephen Brock]

Anson,

It sounds like we are working with an almost identical setup as your studio. Cutting long and short form docs. We are also looking to upgrade to a 10GbE shard network this month. I'll post here any research I come up with soon!

 

[Nick Pasquariello]

It's all in the switching hardware. Do your homework! There are some relatively inexpensive options coming around right now, but they may or may not fit your needs. The Cisco/Netgear ProSafe 10GE switches are the most affordable ones out there, but they don't have the internal bandwidth to sustain all their ports under load. That and they're built very cheaply and have fans inside that sound like a small jet at takeoff.

As a good rule of thumb when putting these systems together, plan on 75~80% efficiency in relation to all advertised numbers. You also need to temper that with the hidden numbers that the manufacturers don't like to talk about -- aggregate and internal bandwidth numbers and such. The 8-port NetGear ProSafe 10GbE switch can connect 8 devices at 10G and one of the ports is SFP capable, but it can truly only sustain 3 of those ports, 4 tops, if you have multiple clients hammering on it at the same time.

Consider your storage back end as well. How are your NAS or SAN nodes going to attach to the network? The ports themselves top out around 850~900MB/s in very ideal situations. Some of the best affordable NAS solutions, like the Intel RAID boxes with 10G connectivity, have an onboard RAID host that tops out in the 550~600MB/s range.

A counter-point to this is that very few users tend to max out their available bandwidth.

I used to work in a ~20 user setting, all connected by 1GigE to an Isilon NAS. I would monitor everyone's bandwidth, partly because it was useful and sort of became part of my job, but mostly because I'm a nerd that likes numbers and graphs. We had, at any given busy period, about 10 people in Final Cut Pro 7, and about 10 people in After Effects. Maybe 1/5 of those people would be working with either Red ProRes transcodes, or Red Raw files (the way we transcoded, even at 1/4 the resolution, the files were the same size as the Raw).

The only two people that ever maxed out bandwidth were myself (always moving files either off hard drives or onto our archival LTO5 Cache-A units) and our one Nuke artist (often using EXRs or DPX sequences or rendering out TIFFs). 5-6 streams of ProRes 422, overlaid in a timeline, playing back, will get you close to 1Gig limitations.

Now, of course, more is always better. But I think you'd be surprised how little bandwidth your stations suck down 95% of the time. Usually, that other 5% is when deadlines are breathing down your neck and rendering is pushed to the max, sure. But still, with around 7 users, even using one of the crappy switches Kilgroe is talking about, that might be more than enough.

 

[ERIC PECK]

Well, like others say, do your homework.

Your performance will vary deeply if you have network hardware that is vlan capable.

In fact, on vlan capable hardware you will be able to do lots of things even on gigabit (yes gigabit) ethernet. R3D files are not that bandwidth intensive.

Also, your performance will vary a lot depending on how you are connecting your network. Keep in mind, it is two things to share via iSCSI vs. sharing via say NFS or SMB.

But if you choose to go the iSCSI way, there are issues to deal in regards to Apple compatibility, but is well worth the hassle. iSCSI removes a lot of protocol overhead which improves efficiency dramatically.

 

[L. Langer]

Surplus high-end fibre gear is easy to come by if you know where to look and it's worth the price. Give us a better idea of where you are and we can give you suggestions on where to go dumpster diving locally, if possible.

 

[Bob Zelin]

I build 10GbE shared storage systems for Maxx Digital all the time now, using the Netgear 10GbE switches (8, 12 and 24 port) and in conjunction with the ATTO 10G cards and boxes, it works perfectly fine, and you can have multiple people doing over 300 MB/sec - more than enough for what you are trying to accomplish. If your cable runs are under 55 meters, you can use regular Cat6 cabling, you do not need Cat6A.
I have been doing this previous to the April 2013 release of the Netgear switches with ATTO NS14 10GbE cards, and this too works perfectly fine. Lot's of installs, no complaints.

Bob Zelin