Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

magic: My RAID5 has jumped back 3 weeks into the past

Miguel "Macgregor" De Olaso

Miguel "Macgregor" De Olaso

Well-known member
Joined
Dec 29, 2006
Messages
796
Reaction score
0
Points
16
Location
Los Angeles
Hi all.
I´m having a huuuge problem here.
For no apparent reason and with not error messages, I just discovered that my RAID 5 (4 drives in HFS+) does not show any of the files I've been working on or created for the past 2 weeks.
Everything else is there, no other data seems lost. The only issue is that the most recent file is from march the 6th. And this means I´ve lost 3 weeks of work (mostly projects in AE).

So I´ve tried any recovery software I can think of and nothing. No luck.
The info should still be there but... somehow is not listed correctly.

Any ideas about how to fix this would be REALLY appreciated at this point (before shooting me in the head).

thanks!
 
No backups?

What are the specifics of the RAID? What controller or RAID hardware? Does the system generate any sort of logs or post errors or transaction info anywhere? Most data recovery type software can't work on a RAID type system, especially something like a RAID-5.
 
If you check your raid controller what does it say the status of the RAID is? If it had a drive failure it might be rebuilding in which case a few hours from now everything should just magically re-appear. I've never heard of a RAID5 doing that... but an oddball controller I could see theoretically doing that.
 
Yeah I checked the RAID status and history and everything is fine and there are no records about anything going wrong.
The Raid uses 4 seagate 750GB drives with a highpoint card in a 09 mac pro.
 
This happened to me once before. Can't remember if I ever got the files back or if I ever fixed the problem. Sorry I don't have a solution for you.

Ever since, I only use raid 0 and make multiple backups.
 
Miguel "Macgregor" De Olaso said:
How do you set up your back up in order not to slow down your raid 0?
Click to expand...

I just manually back up the session every couple of hours using chronosync. Lost all confidence in my raid 5 when directories were failing to show up.

I believe I was able to recover most of the directories with disk warrior. Raid 0 has been trouble free for me.
 
I'm always telling people that RAID-5 (or RAID-6, etc..) are just a false sense of security. Sure, one drive out of every 4 or 5 can theoretically fail and you should be able to retain your data and rebuild the array. But that sums up the intended use of the array right there... Drive fails, should be able to recover. Sometimes drives have issues, controllers have issues. Hard to say what the problem could be without more specific info or the ability to see it all with my own eyes. If you haven't, I would contact HighPoint and see if they have encountered this and if they may have any suggestions. Sorry to say it, but the data is probably gone. Apps like DiskWarrior can't begin to pretend knowing anything about RAID topology. Such apps can recover data from RAID systems, but it depends heavily on the type of data and several other factors. With a RAID system where you're constantly writing redundancy info, it's very odd for files to just go missing unless there is something very wrong, possibly at the host controller level. The additional redundancy data and cached write operations also make it more difficult to retrieve any lost or damaged data as time passes and more info is written to the RAID.

Another thing that many people don't realize with RAID-5 and more intricate RAID systems is that they occasionally need to be "reconditioned". That is essentially like rebuilding the RAID volume and comparing parity/redundancy data as it rebuilds all the data. It is possible that if the files were all there, then suddenly went MIA, that a proper reconditioning of the volume could bring them back. I would recommend a full drive-image backup of each drive in the RAID volume, done at the byte level, before attempting a reconditioning in this situation. You should probably contact HighPoint and see if they have any recommendations or may offer some insight here... If you have not already contacted them.

On all my arrays that are 4 drives and smaller, I use RAID-0. No reason to bother with the sacrifice in performance and capacity for little real-world insurance offered by RAID-5. A good backup strategy with regular backup intervals is the best solution. I typically keep a couple online copies on various systems and/or storage devices for anything I'm working on. Additionally I do regular backups to LTO media -- typically once per week for my current project files and working directories. I do a full system backup with system drive/apps, all my RAID volumes, etc.. once a month. At times it feels like insanity, but every now and then a backup saves my ass and I'm so glad that I had done it.

Anyway, hindsight and backup advice doesn't help the current situation. I would contact HighPoint. I would also see what can be done to test each HDD. It's possible that one or more HDDs are starting to fail, perhaps even reporting SMART errors, and the controller isn't telling you. A rebuild or reconditioning of the array could bring the data back, but be sure to do a proper backup before you start.
 
Last edited:
Miguel "Macgregor" De Olaso said:
How do you set up your back up in order not to slow down your raid 0?
Click to expand...

Personally we use a multi-layered backup system.

1) Windows Shadow Copy at the OS level. This essentially only writes the changes to a file if possible and takes an image of every file that's changed 3 times a day. Once in the morning, once at lunch and once after work.
2) Our working file server gets cloned every night to a second server which also can act as a backup domain controller.
3) Every few weeks we backup all idle or finished jobs to tape.
4) RAID 5.
5) Two portable HDDs that get updated every night. One goes home.

I would just recommend at least doing the last item. Buy two 3TB drives (about $160 each) and then run a backup application every night when the RAID is idle. Maybe once also while you grab lunch. If you aren't working at home that round robin will give you an offsite backup (in case of fire or some other catastrophe) if you do work at home find a friend or get a safety deposit box which can be had for about $30 a year to leave it in and be sure to swap it out every time you really wouldn't want to lose that increment of work under any circumstances.

Lastly I would also use an online service like Live Mesh or Carbonite which offer a few GBs and point it at your active projects folder. Usually composites and edit workspaces are only a few MBs and can be uploaded to the internet during idle disk/network time.
 
Also, the only way I knew something was wrong with my raid 5 was by looking at the istat menu. The computer at idle was using a lot of CPU cycles, even though the raid software was telling me everything was ok. Something to consider installing.
 
Gavin Greenwalt said:
Personally we use a multi-layered backup system.

1) Windows Shadow Copy at the OS level. This essentially only writes the changes to a file if possible and takes an image of every file that's changed 3 times a day. Once in the morning, once at lunch and once after work.
2) Our working file server gets cloned every night to a second server which also can act as a backup domain controller.
3) Every few weeks we backup all idle or finished jobs to tape.
4) RAID 5.
5) Two portable HDDs that get updated every night. One goes home.

I would just recommend at least doing the last item. Buy two 3TB drives (about $160 each) and then run a backup application every night when the RAID is idle. Maybe once also while you grab lunch. If you aren't working at home that round robin will give you an offsite backup (in case of fire or some other catastrophe) if you do work at home find a friend or get a safety deposit box which can be had for about $30 a year to leave it in and be sure to swap it out every time you really wouldn't want to lose that increment of work under any circumstances.

Lastly I would also use an online service like Live Mesh or Carbonite which offer a few GBs and point it at your active projects folder. Usually composites and edit workspaces are only a few MBs and can be uploaded to the internet during idle disk/network time.
Click to expand...




Hi Gavin, I have all the footage backed up in a couple external drives that my friends keep for me. And lately i´m using dropbox to store projects, which sounds like a great idea to keep everything secure and available at any location. BUT it relies on internet speed.
 
anyway, shit happens!

Suerte con eso Miguel.
 
anyway, shit happens!

Suerte Miguel.
 
These issues you encounter are familiar to me. I always have a loss of some files when a drive in our arrays fails. We then replace the faulty disk and the raid starts rebuilding the parity. Then all "lost" files come back. I'm pretty sure that one of your disks is broken and needs to be replaced. Your raid consists of 4 disks which indicates that it is either one of those dreadful LaCie Biggest or similar raids, or a raid that resides inside your computer. Either way it's probably not an enterprise grade array with redundant power supply, hot-spare, Raid6 protection, browser based interface, etc... So you may not find out which of the disk is the dodgy one.

In my newest Raid we caught a bunch of disks from a bad batch. In the last 6 months I lost 4 (!) disks but no single file. Raid protection works well but only if the controller is of high quality and the system is made for serious applications.

In your case I would immediately back-up all available data and then hand over the raid to a specialist.

Hans
 
Hans von Sonntag said:
In my newest Raid we caught a bunch of disks from a bad batch. In the last 6 months I lost 4 (!) disks but no single file. Raid protection works well but only if the controller is of high quality and the system is made for serious applications.
Hans
Click to expand...

Sounds like we got disks from the same batch. I think we're already on our 4th drive in our new RAID for this month. Ridiculous. Thankfully it's under warranty and they have to keep coming out themselves and fixing it's only been annoying for the disk alarm going off every week until someone silences it.
 
@Gavin, @Hans

Stupid question, you're not shipping your drives with UPS are you? They beat the hell out of their packages and I've lost two drives shipped with UPS and none with FEDEX.

Just an idea.

KO
 
You must log in or register to reply here.
Back
Top