Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

RAID 0,1, or 5?

OR perhaps RAID 10

" The RAID level where information is striped across a two disk array
for system performance. Each of the drives in the array has a mirror
for fault tolerance. RAID 10 provides the performance benefits of
RAID 0 and the redundancy of RAID 1. However, it requires four hard
drives. "
 
IMO, RAID-5 is a false sense of security. It can save your data if one drive fails, however I have had single drive failures that still result in massive data loss due to other factors surrounding the drive crash.

What is your intended use of this RAID system? What sort of backup system or procedure do you have in place? What sort of system will this be installed on or into? Not all RAID-5 controllers and subsystems are created equal. Crap hardware and/or crap management software can often make it difficult or even impossible to recover from a drive crash, when it should theoretically be possible.

RAID 10 (also known as RAID 0+1) can often be a lot more robust than a single RAID-5 volume.

For smaller arrays -- 5 drives or less, I would be most inclined to use RAID-0 and just be sure I have reliable backups in place. 5 drives or more, I would consider RAID-5, but more likely I would consider RAID-6 as it offers the redundancy of a spare drive already available online in the system. That said, I'm running one Mac pro with all 4 internal drives as a RAID-5 and it performs well. Additional external RAID storage for faster performance / workspace.
 
You could also start using ZFS on a mac or opensolaris. It is a new filesystem, that will become standard in snow leopard, which eliminates the write-hole of RAID5.

Once one grasp, that the controller has to see the drives as JBOD, and that all the parity calculations are done by the filesystem, it gets quite ingenious.
256but checksums on every block.

I have 6 discs in a mac pro here. 1 system and 5 pooled to a raidz2 pool, which equals RAID6. All is done by the FS, of course all the parity takes its toll on the CPU cycles, but even when the mac gets destroyed (e.g. RAID card) - I can mount the drives on another mac, or even opensolaris.

Performance-wise, there are statements that ZFS is as fast as HW raid, because it uses the latest CPU for parity, rather than couple of years old from the controller.

I found it sufficient for uncompressed realtime with quite old 500GB drives in 2K, while having equal to raid6, so I can smash 2 drives at once, and still have the data.
 
Just using the drive for normal video editing. Right now that is HD. I just want a bit of security hence me thinking of moving away from RAID 0 towards other options.

I have a backup plan that is conservative at best, I backup every now and then.

Would a RAID 1 be a better option than RAID 5?

P.S thanks for the feedback.
 
My recommendation is side-by-side RAID 5's where you mirror the first RAID 5 on the second RAID 5. I personally think this offers the best solution because you can for example take all the drives from the second RAID five and store them off site in a bank safe deposit box for example. Pick them up every week or two and copy any new files over to the mirror.

-shooter
 
Use two sets of raid-0. One you use and another you make manual or rsync copy to. If you make the bakup manually you also can restore the data when it fails, and not depandant upon something/some one else.

Don't use raid 5, unless you have already spent crazy amount of $ on a raid 5 hardware.
 
JanneJansson said:
Use two sets of raid-0. One you use and another you make manual or rsync copy to. If you make the bakup manually you also can restore the data when it fails, and not depandant upon something/some one else.

Don't use raid 5, unless you have already spent crazy amount of $ on a raid 5 hardware.
Click to expand...

This is EXTREMELY dangerous. If ANYTHING goes wrong with even one single drive in either RAID-0 Set during the rsync copy, you will lose everything in both sets.

-shooter
 
I recently was burned with the Seagate 1TB debacle (I've had 2 external RAID 0 drives sets fail). Right now I've got a RAID 0 set, a RAID 5 set and JBOD set that I sent out of state. Of course I'm referring to my R3D's and sound files. My fcp files get uploaded to online disk storage for redundancy. I've recently noticed that my RAID 5 array is rebuilding redundancy everytime I reboot (takes about 5-10 mins) - has me concerned that the controller may be going...what I really need is some sort of tape backup solution...
 
Christopher Grant Harvey said:
I need 4 drives for a R5 right?
Click to expand...

You need a minimum of 3 drives for RAID-5, it works a bit better (in theory) with 4 or 5 drives. However, having 4 or more for RAID-5 doesn't really increase your security.

Shooter's recommendation for multiple RAID-5 sets is pretty good if you want to maintain backups onto HDD. He is also right in that something can happen while you are creating a backup. I can vouch for that... The last drive crash I had was when I was writing a backup to tape. I got a tape full of garbled data and a RAID-0 volume also with garbled data. Lost about 4 days worth of work, or everything I had done since my previous backup.

Even if backing up to RAID-5 or something that is theoretically more robust than a plain drive or a single tape, etc.. Always back up to something that is not going to potentially destroy any recent backups you may have in the process. That's why it's best to rotate backup tapes and media. Worst-case scenario, if a backup operation blows up and the drive you are backing up dies, takes the backup set with it, you still have another backup you did a few days before.

If may sound like we're going a bit overboard with discussion on backups and data security, but we're really not. And making proper backups is simple to do. It's just a matter of actually doing it. And don't keep all your eggs in one basket, so to speak..

What system and hardware are you working with for this? That can make a difference too, as far as what sort of RAID setup you will want to use. But for 4x500GB, I would say a RAID-5 makes good sense. You will get about 1.5TB of usable storage out of it. Formatted capacity about 1.42TB.
 
Hi Chrsistopher,

RAID0 is the fastest (and least expensive) type of multiple-drive array.
With a RAID0 array, the data is broken up and split equally between all the drives. This why when one drive fails in a RAID0 (also know as a "stripe set") your data is lost for good.

RAID5 is essentially just a stripe set with an extra drive that records only "parity" information. I won't go into what exactly parity data is but let's just understand that it allows you to recover all of your data if a single data drive fails.

This is why RAID5 is ideal for applications that need both speed (striping) and fault tolerance (parity).

RAID5 requires a minimum of 3 drives in order to work -two drives in the stipe set and a single drive for parity data. In the event that a data drive fails and needs to be replaced, the RAID controller can reconstruct all your data within a few hours using the parity data stored on the parity drive.

A drawback of RAID5 from RAID0 is that a dedicated RAID controller card is a must if you are working with high resolution motion media. And if you do loose a drive and have to reconstruct the data - this can take many hours. BUt from a cost-effectiveness perspective, it's the best of both worlds....

The other option (seeing that you want both speed and security) is to do a RAID10 - this is a "mirrored" stripe set. This is essentially two complete stripe sets that are exact "mirrors" of each other. If one drive in one drive array (stripe set) fails, the other "side" picks up without missing a beat...complete redundancy - at the cost of buying twice the number of drives you actually plan to use for data.

For this your would need 8 drives - but, of course, would only be able to use the capacity or speed of 4. You will need a RAID controller for this too.

My advice is to buy another drive (identical to the others) and a good RAID controller - and go RAID5.

In your case, you will probably need 5 drives to support uncompressed HD, 4 drives for the video and 1 drive for parity. You will need an 8-port card (they only come in multiples of 4 channels). This will likely cost you between $500 and $750.
 
Thanks again for all the advice guys!

I've been editing for 5 years now and in the last 2 I had a RAID 0 which never gave problems till recently.

I use CineForm and rarely need anything faster than my current RAID 0 array is outputting. I have purchased extra drives for backup recently too and I think I will stick to the R0 and just do more regular backups.

Like I said my R0 was solid for more than a year and only recently gave trouble, and luckily I did back everything up in time before it failed.

I'm gonna go with R0 again... here's to regular backups.
 
There's a common thread here in that Media users, edit or whatever rely very heavily on RAID as security against a drive failure (about 0.25% probibility I think). This happens on small mid to large media arrays as backing up all the media is time consuming and costly. Creating a mirror virtualy doubles the cost of the system which is usually thought to be prohibitive, that's until a drive ails and all content is lost.

If you create a RAID3 or 5 or whatever which is hardware based, the same applies to software based systems, you should at the very least have a spare drive formatted and ready to swap in to the system in case of failure.

Only recently devices such as LTO4 and software like Bru Server have become a practical solution for back up. Archives can be created in software such as catDV and then Archived (as opposed to back up using DAX as a plug in for catDV.

If you can use a back up system to back up your raid also. Id think its probably cheaper and arguably better to invest in near line back up system than mirror the whole Raid.
 
Again thanks for the feedback guys!

At the moment I have no space for more internal drives which is why I bought some external casings and will do period backups to dedicated drives, that is until I can afford a more robust solution.

Who has ever had a drive fail that was just sitting on a shelf unused for a year?
 
Christopher Grant Harvey said:
Again thanks for the feedback guys!

At the moment I have no space for more internal drives which is why I bought some external casings and will do period backups to dedicated drives, that is until I can afford a more robust solution.

Who has ever had a drive fail that was just sitting on a shelf unused for a year?
Click to expand...

Me.

Drives are known to be bad shelf-sitters. That's the reason why I use LTO. The recent years LTO3, next year I might upgrade to LTO4.

And when we are there: If security is of any concern, regular backup on tape are necessary. And if someone thinks that rebuilding a Raid 5 that had a faulty drive is easy and convenient, it's not. RAID 5 does give some data protection but chep raids like LaCie biggest disc are only having an apparent security. If the controller messes all up your data is gone. I had one of those and dumped it.

I use a high-quality hardware FC-Raid (12 disks) in R6 mode with a hotspare. Meaning, in the hopefully unlikely case of a disk failure the Raid is rebuilding itself in the background. The Raid is running sind 5 years WITHOUT switching it off and never let me down. Now the disks have reached EOL and need to be replaced. Although this is the safest solution we back up a project every other day onto tape.

If you want to safe money, don't do that with storage. Reshooting a project can be much, much more expensive than buying secure storage. And IMO, many of those advertised super-sonic fast raids for 3k are all but safe. Here the conservatism of a seasoned IT-tech is mandatory.


Hans
 
In a way to use raid 5 is to increase the chanse of failure. Data is split across many volumes. Min 4, but ideal 9 drives. If you have 9 drives, one bit from every byte is stored on separate drives, and on the 9'th, a check sum is stored. But sins this cheme is rotating, the actual data value and the checksum is always on different drives. BUT because of this, drive mecanics also work alot to save and retrive all those datapackets, and depending on the actual implementation of the raid 5, it can wear down drives faster then if they where not in the raid set.
 
You must log in or register to reply here.
Back
Top