Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

RAID solutions using Red

Jonathan Stevenson

Jonathan Stevenson

Well-known member
Joined
Nov 20, 2008
Messages
1,829
Reaction score
0
Points
36
Location
Denver
Hey folks,

I'm going to be upgrading my Mac Pro with a few new internal hard drives, as well as some external hard drives for backup and overflow. My questions for everyone are:

What are the most reliable internal HD's with the most storage and the best price? I've seen the 1.5TB Seagates, but I thought I remembered hearing those failed at a rate of something like 50%? It'd be great to get at least 4TB inside my Mac.

I'm also quite unfamiliar with the concept of RAID and wondering what the advantages are vs. JBOD. I'm considering getting a hard drive enclosure that can hold 4 disks.

Thoughts?
 
Caldigit provide a HBA that has both internal and the external ports. You can combine the internal drives with up to three 5 bay external raid boxes.
 
On the concept of RAID, it can mean a variety of things. Most commonly are striping data across multiple drives. This takes the capacity of multiple hard drives and makes it look like a single volume to your computer. Besides the space, this gives you a little more speed as well. The downside is that if one disk fails, you lose all of the data, even if you have 3 or 4 drives in this type of RAID.
Another type of RAID is mirrored. In this case, your data is more secure as it mirrors the data across multiple disks, but you don't get the capacity of all the drives you installed. In other words, if you mirror 2 - 1TB drives together, you see only 1TB of useable space. But if one disk fails, the other is there for redundancy.

Striped drives are used a lot for capture scratch or rendering where the data can be rebuilt if lost, but since they are faster, this helps the workflow sometimes.
Mirrored drives are usually used for backups and such where you can't afford to lose any data.

One other form of RAID is known as RAID 5. This is a combination of striping and mirroring. It gives you some redundancy and mirroring so you can lose a drive and stay running with a little faster volume than a single disk. This type of RAID requires a hardware card, or an external enclosure that supports RAID 5.

This is a really basic description of RAID, but hopefully it helps you understand.

For an external enclosure, you might want to look at the Drobo from Data robotics if you are concerned with data integrity. Here's a video explaining how it works.
http://drobo.com/resources/drobodemo.php

I'm still learning the RED workflow but I've seen systems with 2 drives striped for extra speed and capacity and used for rendering and scratch disks. Then use one drive for running the machine and apps then one extra internal for extra online storage. It's usually recommended to not use the boot drive for data storage and scratch because if you fill up the boot drive of a computer, it can make it unstable.

On internal disks, I've heard that the 1.5TB issues are getting better, but haven't followed too closely. There are also a couple of 2TB disks coming out.
Several drive manufacturers have what they call Server class drives or Enterprise class drives. Those are just rated for more hours running and fewer faults while running. They usually aren't that much more (maybe 15-20 percent more than standard drives) and are probably worth the investment.

Not that it matters where you buy a drive, but Other World Computing has some good online resources for explaining the differences in drives. Their tech support is also very good and they have prices that usually compare with most of the discount shops for hard drives and RAM.
http://eshop.macsales.com/

Hope this helps
 
RAID on a Mac Pro...

* DO NOT, I REPEAT, DO NOT BUY THE APPLE RAID CARD! *

CalDigit has an excellent RAID card that supports RAID 0, 1, 5, 6, and combos. It also has an optional bootcamp pass-thru and is the only Mac Pro HBA that works with bootcamp, if you're looking for such a thing. One caveat is that bootcamp still must reside and operate off the primary HDD and that primary HDD can't be a part of any RAID volume.

Unfortunately, there are a couple other issues why someone may not want to buy the CalDigit RAID card. They really only support their own HD-Element units for external RAID. These are 4-drive units. I currently have one hooked up with 4x1.5TB HDDs in it. The units are kinda expensive for what they are. But the performance is great and while they are more expensive than many do-it-yourself approaches, they're still priced rather well for a solid system that just works. I have hooked up SATA devices externally with a mini-SAS to SATA fan-out cable. But this is not officially supported and is not guaranteed to work.

If you want to go to the next level of performance, there's the CalDigit HDPro. I had one of these until recently and sold it while it still had some value for it's capacity, when I swapped the Apple RAID card for the CalDigit w/ HD-Element. The HDPro is amazing, but uses a proprietary interface similar to Infiniband or fiber channel, it's basically some kind of raw-data PCIe implementation capable of 20Gbps, which is pretty cool. Th HDPro systems are amazing if you need that sort of throughput. But it's overkill for RED workflow, IMO. The single HD-Element is more than fast enough to grade 2K DPX in real-time in Apple Color when set up as a RAID-5. I can also increase the speed by about 2.5X by adding two more HD Elements as RAID-5 and then striping all three together in a RAID-0.

Beyond this, we get into systems that take a bit more research and thought to implement.

I highly recommend the CalDigit RAID + HD-Element solution. Otherwise, I would recommend you look at some of the newer SAS RAID systems on the market now. They have really become affordable and provide a more elegant solution than eSATA. The problem with eSATA is you end up with lots of cables and/or port multipliers and most systems hit a performance wall somewhere between 250 and 500 MB/s. So you have to do a lot of homework before putting such a system together and sometimes the money you save may not really make it that great of a deal.

Oh yeah, the Seagate 1.5TB drives are good. Just be sure they have the current firmware. The previous firmware had some timing issues and caused erratic performance leading to drive failure. Do not buy the Maxtor 1.5TB drives -- they are the dictionary definition of junk. If you go with 1TB drives, Hitachi seem to be the most reliable, once again avoid Maxtor. Seagate are good, but they have several production runs that have a high failure rate and they have recalled some of these drives, but many are still out there awaiting unsuspecting consumers.

If you buy Hitachi, make sure to get the Enterprise class units with the longer warranty. They actually have different firmware that tunes them for use in RAID systems as opposed to individual drive operation.

Don't look at Drobo. It's terribly slow and inefficient. Nice concept, but in reality it's just "RAID for dummies". Oh yeah, and you have no idea how slow it is until you actually try one. Copying 1TB of data to a Drobo (even the new FW800 model) tops out at about 48MB/s. It's painful.
 
Jeff Kilgroe
I highly recommend the CalDigit RAID + HD-Element solution.
Click to expand...

OK Jeff help me out.

I'm about to pull the trigger on a new raid. The 6TB CalDigit sounds like a solid choice. I just want to be clear about expansion. I would like to add more storage as needed. Are there any problems when trying to add more storage?

The single HD-Element is more than fast enough to grade 2K DPX in real-time in Apple Color when set up as a RAID-5. I can also increase the speed by about 2.5X by adding two more HD Elements as RAID-5 and then striping all three together in a RAID-0.
Click to expand...

Would you see any advantage with only two raids? Is it even possible?

Thanks for your help
 
Jonathan Stevenson said:
Do you need a Raid card if you're setting up an external Raid?
Click to expand...

Not necessarily. Depends on the external storage.

An external RAID is Storage with a raid level applied. This is "Storage" with a Raid Level. Not just Storage as JBOD (just a bunch of disks)

You do need a way of attaching the storage to the computer though.

So for example..

1-FC Raid over a SAN. No. You need a FC card. The raid hardware is built into the Storage chassis

2-DAS(direct attached storage). Depends.
HD PRO, No. The raid card is in built and the card in the mac/pc is just a bus extension of the pci-e type

3- DAS (JBOD ie no in built raid ability)with esata/infiniband out YES. You'll require a Raid card such as ATTO R380 if you require RAID

4- DAS HW built in raid. You can buy other chassis like the HD-PRO from the lkes of enhancetech that have the HW Raid ability built in. Connections on the back are esata/FW/USB So NO no raid card required if using say FW. TO use the esata link would require a esata card.. not a RAID card.

So to answer your question, no not necessarily. Depends on the Storage unit you purchase.

Si
 
Jeff Kilgroe said:
Oh yeah, the Seagate 1.5TB drives are good. Just be sure they have the current firmware. The previous firmware had some timing issues and caused erratic performance leading to drive failure. Do not buy the Maxtor 1.5TB drives -- they are the dictionary definition of junk. If you go with 1TB drives, Hitachi seem to be the most reliable, once again avoid Maxtor. Seagate are good, but they have several production runs that have a high failure rate and they have recalled some of these drives, but many are still out there awaiting unsuspecting consumers.
Click to expand...

How can you tell if it's the latest firmware or not? Is it printed on the disk somewhere?

Those HDElements look great, especially because you can daisy chain 3 of them. :) 18TB sounds yummy
 
flameop said:
So to answer your question, no not necessarily. Depends on the Storage unit you purchase.

Si
Click to expand...

Which would you say is more reliable? A tower-based card or the chassis based card? Maybe it doesn't make a difference?
 
HighPoint RocketRAID is another HBA option

HighPoint RocketRAID is another HBA option

I'd also like to step in and recommend HighPoint's RocketRAID HBA's for Mac, Windows and Linux. They hold some of the best IO/sec ratings and are very price competitive. Additionally, the company is very pro-Mac, so they won't recommend that you install a Windows system if you have problems on the Mac like many ...

For Mac users, they offer a dedicated portal site:

http://www.hptmac.com/US/index.htm
 
another external RAID enclosure option

another external RAID enclosure option

Other world has several good options for external raid enclosures.
http://eshop.macsales.com/shop/hard-drives/RAID/Rack_Mount/
Their rack mount RAID enclosures are in the $600-$700 range.

Following up on an earlier comment about Drobo, it's definitely not a drive you could work off of directly. But it's a good backup drive since it auto configures it's own raid setup. So if you lose a disk, it will rebuild it's parity down to the remaining disks. Once the rebuild is finished (takes 5 minutes or so) you could lose another disk and not lose data. (as long as you have enough room on the remaining disks)
So it would be a great drive to backup your footage onto.
 
Terabytes

Terabytes

So I was doing some rough math, if I were shooting about 8-10K feet a day in Red 4K Would this make sense? This is for a 5 week job.

350GB per camera a day
x 2 cameras
x 5 days
x 5 weeks
------------------
17.5 terabytes

Does this look right? If so what are you guys doing for this much storage? I see blades here for 4-6 TB are you just ganging them? or does someone know of a better solution?
 
In 4K 16:9, assuming you're shooting 10K per camera, you'd be up for around 400gig per day. If it's 10K total.. and these are rough calculations, you should be able to get away with 200gig per day.

Apologies if I'm wrong - kind of running on the whole
10,000ft =25x 400ft rolls,
equals around 4 mins per roll,
equals 100 mins total.

100mins @ 4K 16:9 RC36 is about 200gig.

Of course, we're talking very ballpark figures here.
Hope that helped, and best of luck.
 
so in that model we are looking at approx 10TB?

any favorites on 10TB of storage?
 
You must log in or register to reply here.
Back
Top