Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

Performance Raid Setup - 187MB/sec sustained

A

Ace

Wedge In
Joined
Dec 28, 2006
Messages
1,493
Reaction score
0
Points
0
Age
42
Location
Sydney, Australia
Website
www.dlshs.net
http://www.lacie.com/products/product.htm?pid=10458

From TFW:

"An extraordinary combination of high capacity - up to 2.5TB, RAID security, and fast speed, this 5-drive tower provides enhanced data protection, massive storage space, and a convenient plug & play user experience. Supporting RAID levels 0**, 1, 0+1, and JBOD, this innovative solution at a fraction of the cost for traditional RAID systems is powered by the newest industry-leading Serial ATA (SATA II) 3Gb/s technology. Simply plug it into a SATA II 3Gb/s host and enjoy a rapid sustained transfer rate of up to 187MB/s. This all-in-one storage solution comes bundled with the LaCie SATA II 3Gb/s PCI-X Card 4E. Designed for business professionals who require high-volume security and speed, this mega-storage solution is ideal for audio/video and creative professionals."

2.5TB $1,679.99
 
OK for Optical port??

OK for Optical port??

Hey Ace - is that fast enough for the optical RAW port from RED? I can't remember the output data rate spec for that?

Cheers, Phil
 
It's only about 1/8th as fast as needed for optical RAW port from RED. Actually, this LaCie solution is a bit on the slow side for what it is. Sonnett and some others have faster RAIDs that are about the same size (both capacity and physcally). I have a 7-drive RAID here that sustains ~360MB/s.

Right now for the 1GB/s or faster (I think 1.2GB/s would be ideal) needed for the RED RAW port, you're probably looking at a system with at least 14 drives at 10K or 15K rpms each. Plus the thing needs to have a 10G-E optical interface. It's not going to be cheap and it's going to be about the same size as a compact, under-counter refrigerator...
 
Thanks Jeff, Best regards, Phil
 
No problem, Phil. :)

FWIW, that LaCie RAID that Ace pointed out is a nice unit for the price. That's very reasonably priced for 2.5TB of self-contained RAID with an eSATA interface. But it comes with a PCI-X card... Not good if you have a newer PC or a Mac Pro or other system with PCI-e slots.
 
Hmm... For most over cranking RAID shots you probably wont be shooting for longer than 30 seconds.. So I wonder if there is some battery powered DDR RAM plug-in solution available for this..
 
Gentlemen, please note the use of the most dreaded words in advertising:

"up to 187 MB/sec"

Note the "up to." That is quite different from "guaranteed minimum sustained performance at all times not dropping frames."

Uncompressed HD RAID systems are usually recommended to maintain 200+ MB/sec at all times for 1080i60 10 bit.
 
ProAvio EditBox 8MP

ProAvio EditBox 8MP

I read that the ProAvio EditBox 8MP box (with a Sonnet E4P [PCI Express for late G5 or MacPro models] or Sonnet X4P [PCI for older G5]) reads & writes at > 330-360MB/sec (can peak at 457MB/sec). The secret is apparently the dual eSata interface (ref : planetvideosystems.com).

My question is ... :help: ... if I want to achieve this rate of transfer using internal drives on a MacPro, what is the config ??? (essentially want to use the internal bays ... but not totally sure what will be the max sustained rate achievable ?).

New MacPro has just arrived ... so need to spec it up

Thanks ...
 
Don't know exact numbers, but I seriously doubt you could get anywhere near that level of performance using internal drives. Although there are four drive bays in a Mac Pro, I would recomend reserving one for a separate system drive, which leaves you only three to stripe together as a media RAID. Even if you striped three drives as RAID 0 (giving no protective redundancy) I doubt you could get that level of data throughput.

Keeping your RAID external has a lot of other advantages too.

http://www.decklink.com/support/detail.asp?techID=62 has some useful information on RAID setups.
 
Internal drives on the Mac Pro isn't such a great setup... To get the performance, you will need a hardware RAID controller. The RAID capabilities built in to the Mac are software-only and that does have a fair amount of overhead or limitation. So given that you'll want a performance RAID controller, you may as well plan on an external system.

The Sonnet ones are one option. But It's entirely possible to roll your own with decent fast drives, a nice external box and a good eSATA RAID controller. There are eSATA hubs and splitters out there to connect more drives and build larger systems like the Sonnet ones. But I don't know how much money you would actually save by doing this yourself. The Sonnet solutions are very reasonably priced for an out of the box solution and you get some level of support along with it. Something to consider.

I would also recommend looking at RAID systems connected via Fiber Channel or Infiniband if you're into something bigger (also more expensive). However, stay away from Apple's XServe RAID solutions. They are so overpriced that I feel insulted when I see what they're selling for. People pay top dollar for them on eBay too and the performance isn't that great compared to other solutions on the market for a lot less money.
 
Our studio tried out an Avid(tm) Raid for a few weeks then tried switching back to a standard 'homebuilt equivalent' for comparison. The Avid Raid smoked the homebuilt. It was a night and day difference.

I would feel much safer if you have the money going with a top dollar 'built' raid solution since they can optimize the drives/controller/etc. If you don't, plan on overengineering the thing to take into account the 'homebuilt lag'.

Or at least that has been my experience.
 
You can build a better RAID setup than the LaCie one pretty easily yourself. For example:
http://www.amug.org/amug-web/html/amug/reviews/articles/enhance/e8/

Jeff Kilgroe said:
However, stay away from Apple's XServe RAID solutions. They are so overpriced that I feel insulted when I see what they're selling for. People pay top dollar for them on eBay too and the performance isn't that great compared to other solutions on the market for a lot less money.
Click to expand...

True! Plus, I have 3 separate horror stories from trusted friends of XServe RAID-5 setups dying at different VFX studios / institutions during peak deadline times ;) I'm not talking about things like a simple drive failure - I'm talking about one day your data is there, the next day your data is not, the Mac can't see the partitions and Apple is unable to help you.

Bruce Allen
www.boacinema.com
 
"homebuilt" is what you make of it. Most nicer RAID controllers have the ability to fine-tune performance and you could take the time to test drives and assemble a top-notch unit. If you want to know a little secret, the Avid units are built out of off-the-shelf components and have no unique or proprietary hardware in them. But building a performance RAID takes a bit of know-how and finesse. You can't just throw a bunch of drives in a box and expect it to sustain 500GB/s.
 
Right which is what I meant. Don't expect that RAID controller off of NewEgg and the HDDs you have lieing around to magically mesh into a great solution without a lot of luck.

You really have to go through the entire setup part by part and make sure you have no bottleknecks, no incompatibilities and everything is playing nice. Which usually comes down to half experience/half blind luck.

Or grab a commercial solution if you don't know what you're doing or don't have the time.

(Actually at one point in time I believe Avid RAIDs did have proprietary hard drives in them... thankfully that's changed.)
 
20 drives seems realllly low for 1.2GBs with 2.5" drives.

I was talking to an IT guy yesterday and he was working on a 50 drive ~1GBs array although that system required a lot of random access performance enhancement (Web hosting).
 
im.thatoneguy said:
20 drives seems realllly low for 1.2GBs with 2.5" drives.
Click to expand...

That's maximum throughput though... I would bet the maximum sustained rate over the entire volume would be in the 500MB/s or so for RAID 5, 6, 7... Probably closer to 650MB/s for RAID-0. Or at least that's what I would think based on the components used.

I'm not really sure what the attraction is with using 2.5" drive units in large RAIDs. Makes little sense from either a cost or performance point of view.
 
This thread sort of terrifies me... thinking about people setting up their own DIY RAIDs for performance at the 1GB/second level. <shudder>

At one of our sites in LA, we are running very high resolution stereo off of SCRATCH at data rates that are 1.3GB/s sustained reads.

It is a non-trivial setup. Getting the disks and controllers is the easy part. Getting everything wired and setup just right takes smart IT and Engineering going through settings very carefully to make sure all is set up correctly.

Here are some of the things we had to tweak in order to get to that magic moment where it "just worked:"

Mobo bus layouts
Core speeds
Multicore configuration
Filesystem config
Filesystem client setup
PCI-e vs. PCI-X
Controller conflict
Bus Conflict
Controller BIOS revs
Controller BIOS settings
Volume config
LUN config
Volume striping
OS "issues"
Monitor refresh
RAID BIOS
OS BIOS

None of this is really rocket science. It is doable and is doable with commercially available units. But it takes a lot of IT and Engineering knowledge to make it work... not just once, but consistently over the course of a typical feature post workflow. And to get that kind of bandwidth reliably is not cheap. We are using the Facilis Terrablock units for this particular config - one XP client running SCRATCH and two 24Ds - list price for everything involved (HW+SW) is around US$100K. And that is *really* cheap considering some of the other SAN options out there.

But when it does work, it's really really cool. : )

Lucas Wilson
------------
ASSIMILATE, Inc.
LA, CA, USA
 
You must log in or register to reply here.
Back
Top