Hardware triple mirror RAID 1?

[tomviolin]

I am interested in setting up a triple-mirrored RAID 1 setup in a
Linux environment.

I know that eSATA cards such as the SIL3114r support RAID 1 in
hardware. However, I do not know if their BIOS supports triple-
mirroring.

Failing that, can I safely assume that Linux software RAID supports
triple-mirroring?

Of course another idea would be to take a standard hardware-based
double-mirrored RAID set and mirror it with a single drive in
software.

Thanks in advance for any information.

 

[timeOday]

I am interested in setting up a triple-mirrored RAID 1 setup in a
Linux environment.

I know that eSATA cards such as the SIL3114r support RAID 1 in
hardware. However, I do not know if their BIOS supports triple-
mirroring.

Failing that, can I safely assume that Linux software RAID supports
triple-mirroring?

Of course another idea would be to take a standard hardware-based
double-mirrored RAID set and mirror it with a single drive in
software.

Thanks in advance for any information.

Sorry for not answering your question, but what would be the
disadvantage of software for Raid 1? Doesn't seem like CPU load would
be significant.

 

[Arno Wagner]

I am interested in setting up a triple-mirrored RAID 1 setup in a
Linux environment.

I know that eSATA cards such as the SIL3114r support RAID 1 in
hardware. However, I do not know if their BIOS supports triple-
mirroring.

Failing that, can I safely assume that Linux software RAID supports
triple-mirroring?

Yes, you can.

Any particular reason for triple-mirroring? Double-mirroring
is usually reliable enough.

Of course another idea would be to take a standard hardware-based
double-mirrored RAID set and mirror it with a single drive in
software.

Well, the ''standard'' ''hardware'' RAIDs are actually often software
RAID done on BIOS level. Linux software RAID is equally fast or
faster. Also Linux software RAUD can be done on partitions (and
mixing full drives and partitions), so you can use extra space
for something else if you have different sized disks or you
do not want to RAID the complete disk size.

Arno

 

[tomviolin]

Any particular reason for triple-mirroring? Double-mirroring
is usually reliable enough.

Good question. Basically, I'm a one-man IT department, and I'd like
to be able to sleep at night.

We recently experienced a failure of our 4-disk (plus two spares) RAID
5 array. One of the drives failed, and while the spare was still
rebuilding, another drive in the array failed, thus trashing the
entire array.

Then, a few months later, our Promise RAID box completely crashed, and
simultaneously our tape backup drive failed. I had to wait over a
week for the replacement tape drive to arrive, and then I had to
restore everything to a new RAID box. Users were NOT happy.

I was extremely unlucky, granted, but I resolved after the most recent
failure to implement the following:

1. Mirrored RAID 1. That way if the RAID card or host system or
whatever fails, any one of the disks in the mirror can be taken out
and the data easily retrieved. You just can't do that with any
striped system.

2. Triple mirrors. That way, in order to suffer a loss of data, all
three drives would have to fail before the spare drive was rebuilt.
Barring catastrophic events (flood, fire) that would precipitate
multiple simultaneous drive failures of this magnitude, the data will
be safe and accessible at all times.

I am also considering locating a backup server on the opposite end of
our concrete and steel industrial building that would mirror the data
at least nightly, so that even if there were an accident that took out
the main server, the data would be safely mirrored elsewhere.

Given the fact that hard drive prices have been dropping, this
solution seems workable to me.

 

[Odie Ferrous]

Good question. Basically, I'm a one-man IT department, and I'd like
to be able to sleep at night.

We recently experienced a failure of our 4-disk (plus two spares) RAID
5 array. One of the drives failed, and while the spare was still
rebuilding, another drive in the array failed, thus trashing the
entire array.

Then, a few months later, our Promise RAID box completely crashed, and
simultaneously our tape backup drive failed. I had to wait over a
week for the replacement tape drive to arrive, and then I had to
restore everything to a new RAID box. Users were NOT happy.

I was extremely unlucky, granted, but I resolved after the most recent
failure to implement the following:

1. Mirrored RAID 1. That way if the RAID card or host system or
whatever fails, any one of the disks in the mirror can be taken out
and the data easily retrieved. You just can't do that with any
striped system.

2. Triple mirrors. That way, in order to suffer a loss of data, all
three drives would have to fail before the spare drive was rebuilt.
Barring catastrophic events (flood, fire) that would precipitate
multiple simultaneous drive failures of this magnitude, the data will
be safe and accessible at all times.

I am also considering locating a backup server on the opposite end of
our concrete and steel industrial building that would mirror the data
at least nightly, so that even if there were an accident that took out
the main server, the data would be safely mirrored elsewhere.

Given the fact that hard drive prices have been dropping, this
solution seems workable to me.

Why not consider Raid 6 with real-time backup? (Backs up files as they
change.)

Promise is hardly top-end kit, anyway...


Odie

 

[Arno Wagner]

Good question. Basically, I'm a one-man IT department, and I'd like
to be able to sleep at night.

We recently experienced a failure of our 4-disk (plus two spares) RAID
5 array. One of the drives failed, and while the spare was still
rebuilding, another drive in the array failed, thus trashing the
entire array.

Urgh! Very bad.

Then, a few months later, our Promise RAID box completely crashed, and
simultaneously our tape backup drive failed. I had to wait over a
week for the replacement tape drive to arrive, and then I had to
restore everything to a new RAID box. Users were NOT happy.

Bad again!

I was extremely unlucky, granted, but I resolved after the most recent
failure to implement the following:

1. Mirrored RAID 1. That way if the RAID card or host system or
whatever fails, any one of the disks in the mirror can be taken out
and the data easily retrieved. You just can't do that with any
striped system.
Agreed.

2. Triple mirrors. That way, in order to suffer a loss of data, all
three drives would have to fail before the spare drive was rebuilt.
Barring catastrophic events (flood, fire) that would precipitate
multiple simultaneous drive failures of this magnitude, the data will
be safe and accessible at all times.

Let me suggest some additional things: Use different drives, best from
different manufacturers. At least for Linux software RAID this will
not be a problem. Do this on partitions that span most of the disk and
leave the last 5% or so unused, so differences in capacities will not
matter. This way you should get a different ageing profile and
different specific sensitivities on all drives.

Run full smart selftests on the drives every 14 days or so and
monitor the drives with smartmond. Best do this staggered, e.g. one
drive every 5 days or so. Have it send email on problems.
This may give you advance warning and you can swap-in your cold spare
disk from the cupboard.

Monitor the RAID with mdadm. Have it send email on problems.

It may be possible to send an SMS to your phone in addition to
email. I have not looked into that.

As to reliability, I currently have several TBs in several arrays with
Linux software RAID (1, 5 and 6), and had zero problems in >= 3 years
from the software RAID side (before I only had one small array).

I am also considering locating a backup server on the opposite end of
our concrete and steel industrial building that would mirror the data
at least nightly, so that even if there were an accident that took out
the main server, the data would be safely mirrored elsewhere.

Good idea. You should use at least two or better three rotating copies
for that.

Given the fact that hard drive prices have been dropping, this
solution seems workable to me.

I agree. And tripple redundancy may not be too overengineered in
your case.

Arno

 

[Arno Wagner]

Why not consider Raid 6 with real-time backup? (Backs up files as they
change.)

If his data fits on a single drive, tripple-RAID1 is better, since
it is simpler and he gets three viable, independent copies.
For larger arrays, I agree that RAID 6 is the way to go. I
currently have one 2TB RAID 6 array under Linux software RAID
for 8 months or so, without problems. Speed is fine.

Promise is hardly top-end kit, anyway...

Agreed.

Arno

 

[Michael Hawes]

Urgh! Very bad.


Bad again!



Let me suggest some additional things: Use different drives, best from
different manufacturers. At least for Linux software RAID this will
not be a problem. Do this on partitions that span most of the disk and
leave the last 5% or so unused, so differences in capacities will not
matter. This way you should get a different ageing profile and
different specific sensitivities on all drives.

Run full smart selftests on the drives every 14 days or so and
monitor the drives with smartmond. Best do this staggered, e.g. one
drive every 5 days or so. Have it send email on problems.
This may give you advance warning and you can swap-in your cold spare
disk from the cupboard.

Monitor the RAID with mdadm. Have it send email on problems.

It may be possible to send an SMS to your phone in addition to
email. I have not looked into that.

As to reliability, I currently have several TBs in several arrays with
Linux software RAID (1, 5 and 6), and had zero problems in >= 3 years
from the software RAID side (before I only had one small array).


Good idea. You should use at least two or better three rotating copies
for that.


I agree. And tripple redundancy may not be too overengineered in
your case.

Arno

You could just use 3 drive Raid5 with a Hot Spare, or even 2 spares if
you're really paranoid.

Mike.

 

[tomviolin]

You could just use 3 drive Raid5 with a Hot Spare, or even 2 spares if
you're really paranoid.

Mike.

Keeping a RAID5 array size down to 3 does limit my exposure, agreed.

However, when a drive fails, if either of the remaining drives fails
while a spare is being rebuilt, the array is dead. In addition, I'm
back to a striped setup, in which individual drives are unusable in
case of a system-wide failure. With triple-mirroring, it would take
three failures in the time it takes to build a hot spare to kill the
array, a far less likely event.

 

[tomviolin]

Let me suggest some additional things: Use different drives, best from
different manufacturers. At least for Linux software RAID this will
not be a problem. Do this on partitions that span most of the disk and
leave the last 5% or so unused, so differences in capacities will not
matter. This way you should get a different ageing profile and
different specific sensitivities on all drives.

Excellent ideas. However, I have already purchased a bunch of drives
that I got a bulk deal on. I'll definitely buy different makes for
expansion and replacement.

Also, I'm currently looking at a Silicon Image 3124-based eSATA card
that has built-in RAID1 capability, but I plan to do some comparative
tests of "hardware" vs software RAID1 once I get all the hardware.

My main concern with a purely software solution is configuring the
automatic rebuilding. With hardware solutions the configuration seems
simpler.

Run full smart selftests on the drives every 14 days or so and
monitor the drives with smartmond. Best do this staggered, e.g. one
drive every 5 days or so. Have it send email on problems.
This may give you advance warning and you can swap-in your cold spare
disk from the cupboard.

Solid suggestions.

Monitor the RAID with mdadm. Have it send email on problems.

It may be possible to send an SMS to your phone in addition to
email. I have not looked into that.

There are two options for that: (1) with most cellular providers (at
least in the U.S.) there is an e-mail address available that
corresponds to your phone number (e.g. (e-mail address removed)) that
is sometimes free, which I have with my phone; (2) most U.S. providers
have a web form you can use to send SMS messages to their
subscribers. A script could be written to send a message (if they
haven't implemented protections against it...)

Arno

Thanks again for your good feedback!

-Tom

 

[Arno Wagner]

Excellent ideas. However, I have already purchased a bunch of drives
that I got a bulk deal on. I'll definitely buy different makes for
expansion and replacement.

Also, I'm currently looking at a Silicon Image 3124-based eSATA card
that has built-in RAID1 capability, but I plan to do some comparative
tests of "hardware" vs software RAID1 once I get all the hardware.

You should know that for hardware RAID, you also need a spare controller,
while with Linux software RAID, getting the drives connected somehow
(also to a different computer) is enough to get the array running.

My main concern with a purely software solution is configuring the
automatic rebuilding. With hardware solutions the configuration seems
simpler.

You mean hot-swap and rebuild? You do not get that on cheaper
hardware controllers anyways. Linux cannot do it, unless you write
a script for it, since is does not know whether some new drive is
supposed to be part of an array or not. In the second case data
could be destroyed by automatic rebuild. Whit a bit of scripting, you
can of course do allmost anything. Personally I have stayed with
manual rebuild. One command is really not too much to enter.

Solid suggestions.

There are two options for that: (1) with most cellular providers (at
least in the U.S.) there is an e-mail address available that
corresponds to your phone number (e.g. (e-mail address removed)) that
is sometimes free, which I have with my phone; (2) most U.S. providers
have a web form you can use to send SMS messages to their
subscribers. A script could be written to send a message (if they
haven't implemented protections against it...)

That is what I was thinking. Since my cellphone is mostly off, I
did not look into this more closely.

Thanks again for your good feedback!

You are welcome.

Arno