Relief
Tuesday 25th July, 2006 18:31 Comments: 0
My RAID array died on me as I drove home (although I didn\'t find out until I\'d parked the car), my mobile got the text message to say it had fallen over, and with the spare drive I\'d recently setup, I wasn\'t too worried about losing any data, as it should rebuild onto that disk. And it wasn\'t like the data was bad on the disk that had dropped out. I was more concerned with the beeping that would be coming from my flat.
It turns out, with the cut up case badge sticker, it wasn\'t that loud, and if I closed the door to the room and moved the cupboard doors closer together it was almost quiet enough to ignore. I was watching the progress over Remote Desktop and I decided to try the "unplug" button on the disk that appeared to have failed. Suddenly it stopped responding. My screen faded to various shades of grey. The computer had stopped responding, so I walked in and held down the power button until the computer turned itself off. My guess is the computer either locked up from overheating (as it was trapped) or hitting the unplug button caused Windows to have a big hissy fit. Either way, I\'m not doing either again.
SO, I powered back up, hit the return key a few times, and expected everything to be fine. It wasn\'t. It turns out the drive that had dropped out was on the controller, but not in the spare pool. Not too bad, you might imagine, as I can add it back to the RAID array. Except it wouldn\'t let me. Because the array was broken. Yes, I panicked a little, and it turns out that one of the disks didn\'t get picked up, so I only had 6 out of 7 disks, and one of those disks was the "spare" that hadn\'t finished rebuilding. 5 out of 7 disks means you\'ve lost all your data in a RAID 5 array (you can tolerate losing one disk). So I rebooted, hoping that it\'d pick up the missing disk.
It didn\'t. Not only that, but it couldn\'t find another disk either. That means I had 4 complete disks out of the 7. That's really bad. So I shut down, crossed my fingers, waited for things to cool down for a minute, and poked and prodded the cables to see if any were loose. I powered up, I did the Remote Desktop thing, and OMFG it worked! It picked up the missing disks, saw they contained all the missing data, so now I\'m rebuilding the data, and I have a disk back in the pool again.
Morals from this story:
Don\'t copy large amounts of data on the hottest day we\'ve had this year
I\'d started a 100GB transfer, ironically to backup data so I wouldn\'t lose any important files, and was well over halfway through - the current observation on the BBC website says it's 31 degrees here. I dread to know how hot the hard disks were.
Don\'t keep all the windows and doors closed to make the beeping quieter
And preferably stop running CPU intensive programs like SETI@home. It\'ll just hasten the demise.
Let the computer beep
Restarting the computer while it's trying to rebuild the array is potentially dangerous, as you can lose everything. Wait two hours and then reboot (although I\'ve yet to find out if it stops beeping after a reboot).
Cut up sticky badges don\'t make the beeping much quieter
It looks like I\'ll have to break open the plastic cover and properly kill the speaker. There doesn\'t appear to be a jumper to neatly disable the beeping. At least it was a lot quieter than I remember.
RAID arrays are not particularly safer than keeping data on lots of separate disks
Although I suspect in an air conditioned environment, like work, it\'d be fine. I think you can get portable air conditioning units from about eighty quid, I may have to buy one.
It turns out, with the cut up case badge sticker, it wasn\'t that loud, and if I closed the door to the room and moved the cupboard doors closer together it was almost quiet enough to ignore. I was watching the progress over Remote Desktop and I decided to try the "unplug" button on the disk that appeared to have failed. Suddenly it stopped responding. My screen faded to various shades of grey. The computer had stopped responding, so I walked in and held down the power button until the computer turned itself off. My guess is the computer either locked up from overheating (as it was trapped) or hitting the unplug button caused Windows to have a big hissy fit. Either way, I\'m not doing either again.
SO, I powered back up, hit the return key a few times, and expected everything to be fine. It wasn\'t. It turns out the drive that had dropped out was on the controller, but not in the spare pool. Not too bad, you might imagine, as I can add it back to the RAID array. Except it wouldn\'t let me. Because the array was broken. Yes, I panicked a little, and it turns out that one of the disks didn\'t get picked up, so I only had 6 out of 7 disks, and one of those disks was the "spare" that hadn\'t finished rebuilding. 5 out of 7 disks means you\'ve lost all your data in a RAID 5 array (you can tolerate losing one disk). So I rebooted, hoping that it\'d pick up the missing disk.
It didn\'t. Not only that, but it couldn\'t find another disk either. That means I had 4 complete disks out of the 7. That's really bad. So I shut down, crossed my fingers, waited for things to cool down for a minute, and poked and prodded the cables to see if any were loose. I powered up, I did the Remote Desktop thing, and OMFG it worked! It picked up the missing disks, saw they contained all the missing data, so now I\'m rebuilding the data, and I have a disk back in the pool again.
Morals from this story:
Don\'t copy large amounts of data on the hottest day we\'ve had this year
I\'d started a 100GB transfer, ironically to backup data so I wouldn\'t lose any important files, and was well over halfway through - the current observation on the BBC website says it's 31 degrees here. I dread to know how hot the hard disks were.
Don\'t keep all the windows and doors closed to make the beeping quieter
And preferably stop running CPU intensive programs like SETI@home. It\'ll just hasten the demise.
Let the computer beep
Restarting the computer while it's trying to rebuild the array is potentially dangerous, as you can lose everything. Wait two hours and then reboot (although I\'ve yet to find out if it stops beeping after a reboot).
Cut up sticky badges don\'t make the beeping much quieter
It looks like I\'ll have to break open the plastic cover and properly kill the speaker. There doesn\'t appear to be a jumper to neatly disable the beeping. At least it was a lot quieter than I remember.
RAID arrays are not particularly safer than keeping data on lots of separate disks
Although I suspect in an air conditioned environment, like work, it\'d be fine. I think you can get portable air conditioning units from about eighty quid, I may have to buy one.