Welcome to our community

Be a part of something great, join today!

Log in
  • Hey all, just changed over the backend after 15 years I figured time to give it a bit of an update, its probably gonna be a bit weird for most of you and i am sure there is a few bugs to work out but it should kinda work the same as before... hopefully :)

How to diagnose what drive failed in a 24 bay ARC server?

B

Brian Merlen

Well-known member
Joined
Jul 31, 2010
Messages
3,185
Reaction score
0
Points
0
Age
40
Location
Stamford CT
Website
vestadalux.com
Basically we are in the middle of switching IT companies, all the guys who knew how to fix are not available to help, and I the lowly DIT/editor am trying to diagnose which drive it is. We have a spare drive on hand to swap it, and its a million dollar job worth of data so I want to tread carefully in this process (its a raid 6 with backups, but who knows if multiple drives failed or are about to, we run it all day long with non enterprise drives and swap out fairly often for failures). So I am trying to hit up the old IT guys, no ones answering, and a former IT savvy editor is busy and can't assist. Basically up a creek without a paddle and looking for help! Please email/PM reply to me ASAP! If I can't figure it out I am going to shut it all down and call it a week until our new assistant editor is in on Monday (she is supposedly IT savvy the post super tells me), but we're falling behind if we do this so if I can get it fixed now that would be more ideal... Please help!!!
 
I am not experienced in this type of diagnosis, but here's what i would try first. run a program on the machine which detects what drives are functioning and which ones have errors. after you've identified which drive the computer thinks is bad, gather the serial number of that drive using the programs offered with the basic OS. You should already have these installed on the machine. After you've got the serial number, see if there's corresponding serial numbers on the backs of the physical drives. I still think you'll need someone savvy, because honestly even if you knew which drive was bad... then what? there's probably more technical steps to follow after that.
 
The program that usually tells us such information won't open on that server it gives an error message saying "application has one instance in the system"

The server below that opens the application fine, and that application is what our former IT guys used to figure out which one to swap. The actual swapping is easy like in a normal PC from what I can tell. But I just don't know how to figure out what bay it is. We're going to shut down for the day and hopefully someone can help me on RU, or we get an IT guy in here asap. If not we will shut down until Monday and see if the new assistant can help. Supposedly she is IT savvy, but the problem is these servers don't have support. They are just rack mounts built by our former IT guy, and he hated them too because of the lack of support...

I don't know why IT is so difficult. NAS kind of blows when things go wrong. All the other externals I use are so easy to figure out, but NAS is just a pain man...We have constant permission problems. Right now one of the 4 systems isn't even running because of that, but with a server drive failure that takes priority at the moment
 
Can't say for sure without knowing more about the RAID itself. But typical procedure on larger RAID units like this is first to look at the front of the unit. The failed drive(s) should have status LEDs to indicate they have failed. There is often an LED readout that will tell you which drive(s) are in failure. If it's one of those plain-jane RAID units with no visible status LEDs or readouts, then you will need to access its management console somehow. Sometimes it's web/ java based, sometimes it's a proprietary app.

Most RAIDs of this size are hot-swappable, meaning you can pull the failed drive(s) out and replace the HDD on the sled and put the new one in. Sometimes they will auto-build once a functional drive is inserted. Other times you may need to initiate the rebuild process from the management console.

I've sent you a PM...
 
If its hot-swap most chassis have a disk activity LED at the very least (a lot don't have the fail lights though). If the disk has completely failed and the array is degraded then you should be able to go by disk activity on which disk to replace. That being said how do you know what disk has failed in the first place if the software isn't working?

It would help knowing exactly what raid controller you have. Although even if a disk has failed and its degraded all data should still be accessible and shouldn't delay anything. If the data isn't accessible then I think you have more problems than just a disk failure.
 
Closed down for the day. Tomorrow I'll be there with IT and management. The issue is we need both this drive fixed, and also according to the former IT "the backs keep blowing the drives" and that this server is a PIA, etc... So if that is the case on why we have often failures that part needs to be looked into as well. Plus then we need permission issues addressed. Obviously the first two are whats the most important. I'm hoping they can figure this thing out. Without that software running on that server the post super and I have no idea how to check the status. There is no light indication for failure, all the lights are doing the same thing...but we know its a drive failure cuz it keeps beeping at us...
 
update:

IT didn't come in, but the post super saw that somehow one of the bays had popped open ever so slightly. No one else noticed, for the 24 hours it was an issue...he pushed it back in and the beeping stopped and the array is fine! which is spectacular...!


the only thing that worries me now: ghosts

I was alone with the post super cutting when the beeping started, no one had touched a bay door at all for any reason. I am fairly sure this is a haunted studio now lol....

as to why would ghosts open array bay doors, your guess is as good as mine...
 
After I die, if I can come back to haunt anything, that's what I'm going to do... Just troll around datacenters and render farms and wreak minor havoc like this. Loosen a drive here, power connector there, stop up a fan blade on a power supply so it cooks itself. Yep, now I know exactly what causes all my issues. Frickin' ghosts... And to think I was always blaming those pesky gremlins.

Anyway though, stuff vibrates loose. It happens. Or someone walked through at some point and caught their shirt on it, only for it to fully pop loose a day or three later. :/
 
You must log in or register to reply here.
Back
Top