K
koko
did I wrote usable ? I wrote 'in the labs today' to make it clear that is
not science fiction like this optical x86 cpu
did I wrote usable ? I wrote 'in the labs today' to make it clear that is
not science fiction like this optical x86 cpu
K
keith
Optical switches have been demonstrated, as well. My opinion isn't
changed; yawn.
K
koko
And what exactly is your opinion ? :]
K
Keith R. Williams
I thought I made my opinion rather clear; I'll care even less by
the time either technology is useful.
N
Nate Edel
In comp.sys.intel Bill Davidsen said:
Blank DVDs aren't practical for backing up a whole terabyte, but they're
down to about ~$100/TB in bulk ($40-50 per 100 disks/~450gb).
R
Rob Stow
Nate said:
In the home and SOHO markets, however, DVD is a very common
backup media - when backups are done at all.
A large part of it is due to the fact that in such places there
is no one to do backups except ordinary users - many of whom are
quite comfortable with burning a DVD, yet intimidated all to hell
by tape drives.
Forget about the fact that tape backup has been around since JC
and the boys went out for pizza - to non-techies they are new and
intimidating gadgets.
B
Bill Davidsen
Rob said:
Don't forget that tape backup is expensive, that tapes are not used for
anything else while DVD is used on many systems for data transfer, the
individual media are cheap, and the cost of a 2nd drive is minimal.
Having a 2nd tape drive (or a reasonable size) is very expensive, not
having a 2nd tape drive means you don't have a backup if the 1st one
fails. I've seend that twice in 30 years, and both times the manager's
career took a major hit for not having a backup drive.
There's no help in sight, DL DVDs are too expensive (thanks to DRM for
that), and 25-30GB Blueray or HD-DVD blanks are also unlikely to be
affordable. Any tape format large enough to be useful is expensive, both
drive and media. You can put a TB of disk in for <$1k, but you can't do
a decent backup system for that.
What the world needs is a cheap four bay external usb drive case, put in
four big drives as RAID-5, take a backup and put it in the safe. Don't
know of any.
Anyway, DVD is used instead of tape because you have it anyway, tape has
no other use but backup in most cases.
N
Nate Edel
In comp.sys.ibm.pc.hardware.chips Rob Stow said:
There's also a big factor of labor being cheap compared to hardware; very
much the opposite of professional installations.
N
Nate Edel
In comp.sys.ibm.pc.hardware.chips Bill Davidsen said:
DL DVDs are falling in price, though slowly, largely because there's little
need for them in the consumer sector, and there aren't many drives. I think
you're wrong about BR/HD-DVD prices: it will take a couple of years for the
market to shake out and quantities to build, but I fully expect they'll fall
to a comparable cost-per-byte than current DVDs, even if they cost enough
more to produce than the $40/100 that current DVD-/+R costs.
Software RAID5 is pretty easy, and there are any number of 4-bay external
USB and firewire cases, although those are expensive. Frankly, I don't trust
single-parity enough to use it for backups, though. I'm not sure if a 4+2
RAID6 would be adequate, but given 4 drives for backup, I'd do mirrored
pairs.
T
Terje Mathisen
Nate said:
Certainly much lower cost/byte! There is no way a new and higher
capacity medium in the same form factor can stay at equal or higher
cost/byte, in all previous cases, the new version have eventually passed
the previous generation in this regard, afaik.
With four drives, you'd have the same capacity and better redundancy
with a 2+2 RAID setup instead of 1+1:
You could then lose any single drive or any pair of drives except for
both the data drives. It might also be possible to encode the two
"parity" drives in such a way that they could also be used to recover
the actual data?
I.e. is there a code in which you can encode two blocks of data on four
drives, and recover the data blocks from any pair of surviving drives?
Terje
T
Thomas Womack
Terje Mathisen said:
Unfortunately not -- you need five, encode A A B B A+B, and then you
can recover from any two losses. That pattern extends: two blocks on
six drives survives three failures by A A B B A+B A+B, on eight
survives four by A A A B B B A+B A+B ...
There's probably a beautiful proof, based on some intuitively obvious
property of some geometric object that's clearly the right thing to
imagine, that 4-on-2-surviving-2 doesn't work; but I ran through all
2^16 possible encodings, since getting C to count to 1<<16 is quicker
than acquiring mathematical intuition.
Seven discs can encode 4 discs of content and survive two losses; use
the Hamming code, whose minimum distance is 3 so *removing* (rather
than corrupting) two whole columns still leaves distance between the
codewords. But seven discs are getting rather loud and heavy.
Tom
R
Rob Warnock
+---------------
| I.e. is there a code in which you can encode two blocks of data on four
| drives, and recover the data blocks from any pair of surviving drives?
+---------------
What if the drives stored "A", "B", "A+B", and "A-B" [chunked in some
reasonable sizes], where "+" and "-" are the normal arithmetic operators?
Yeah, that works... *if* you know which drives are bad.
[Note: This doesn't work if the 3rd & 4th drives store "A XOR B" and
"A XOR NOT B", say.]
Though note that with *five* parity drives (2t+1) you can get double
error correction (t=2) [and triple error detection] across a *large*
number of drives, say, a dozen or more [up to 250, actually, using a
truncated Reed-Solomon code and striping at the byte level]. Of course,
doing the correction when needed is likely to be sloooowwwwwww... ;-}
But it might be good for archival purposes.
-Rob
| I.e. is there a code in which you can encode two blocks of data on four
| drives, and recover the data blocks from any pair of surviving drives?
+---------------
What if the drives stored "A", "B", "A+B", and "A-B" [chunked in some
reasonable sizes], where "+" and "-" are the normal arithmetic operators?
Yeah, that works... *if* you know which drives are bad.
[Note: This doesn't work if the 3rd & 4th drives store "A XOR B" and
"A XOR NOT B", say.]
Though note that with *five* parity drives (2t+1) you can get double
error correction (t=2) [and triple error detection] across a *large*
number of drives, say, a dozen or more [up to 250, actually, using a
truncated Reed-Solomon code and striping at the byte level]. Of course,
doing the correction when needed is likely to be sloooowwwwwww... ;-}
But it might be good for archival purposes.
-Rob
R
Rob Warnock
+---------------
| >I.e. is there a code in which you can encode two blocks of data on four
| >drives, and recover the data blocks from any pair of surviving drives?
|
| Unfortunately not -- you need five, encode A A B B A+B, and then you
| can recover from any two losses.
+---------------
You only need *four* -- A, B, A+B, and A-B -- if you know *which*
drives have failed [which you can usually tell from the drives' own
internal error-detection]...
-Rob
| >I.e. is there a code in which you can encode two blocks of data on four
| >drives, and recover the data blocks from any pair of surviving drives?
|
| Unfortunately not -- you need five, encode A A B B A+B, and then you
| can recover from any two losses.
+---------------
You only need *four* -- A, B, A+B, and A-B -- if you know *which*
drives have failed [which you can usually tell from the drives' own
internal error-detection]...
-Rob
C
Casper H.S. Dik
But not always; and if the drives were kind enough to write the data
where you ask it to write the data first.
This is one reason why Solaris ZFS keeps checksums separate from data;
it can tell which part of the mirror returned bad data and repair it
because the checksum fails.
Casper
S
Stephen Fuld
Casper H.S. Dik said:
There are two separate issues here. One is whether you treat the array as
essentially RAID-3, that is, read every drive on each read operation. You
can do this, of course, but your transaction read, i.e. short read,
performance will be terrible. For an archive that wouldn't be to bad, but
for "normal" (that is things like database) use, it is terrible.
The second issue is recovering, or indeed detecting an error where the drive
writes the data to an address different from where it should. If you are
not reading all the drives all the time, this is, in general, not possible.
For example, say you have a typical RAID-5 array. A data read will read one
drive. If you are reading from where the data "should" have been written,
you will instead get the data that was there before, which presumably has
good ECC, etc. so you won't be able to detect the error without some outside
mechanism (as you seem to describe below). This is true no matter how many
"correction drives you have.
Now assume a single parity drive but that you read every drive on every read
operation. You will now detect the error, but, in general, since you have
essentially single parity you can't tell which drive was bad and so cannot
correct the data.
I'm not sure about the situation with two "parity" drives, and reading all
the drives on every read. You might be able to use some sort of burst type
ECC (such as a Reed Soloman) on one of the drives and tread the miswrite as
a burst error somewhere in the stream, but you sure wouldn't want to do that
in software.
I am not familiar with that particular implementation, but in general, if
you write the checksum to a third drive (to protect against systematic write
addressing errors) you have to read both data drives all the time and
compare. Then use the checksum on a miscompare. But in general, write
addressing errors are "silent", that is, they could be occurring for a long
time without you knowing about it. So day this is happening on a drive that
contains the checksum. So you have written the checksum to the wrong place.
Now on a "typical" failure on a data drive, you can't correct. Note that
this is not the case of two simultaneous errors due to the potential for
long standing undetected errors presented by the write addressing failure
mechanism.
In general, write addressing errors are the most difficult to deal with, and
most systems don't deal with them very well, if at all. Fortunately, they
are extremely rare.
S
Stephen Fuld
Rob Warnock said:
There is an existing hardware implementation that uses two "correction"
drives to correct up to any two failing drives in an array of 16 drives (I'm
not sure if the math limits the array size to 16) . But it does assume that
you know which drive failed (almost always a good assumption).
T
Thomas Womack
I don't think that actually works if you constrain the drives to store
data in a power-of-two base: to recover from A+B and A-B you need to
divide by two, and that's not possible mod 2^n. For example, A+B=0
A-B=128 (mod 256) tells you either that A=64, B=192 or A=192 B=64, but
you can't tell which. You could store in base 128 on the data and base
256 on the parity discs, I suppose.
Tom
B
Bill Todd
Casper H.S. Dik wrote:
....
Could you point me to such detailed descriptions of ZFS as that which
included this juicy snippet (a feature that I've been pursuing myself at
my usual glacial pace)? Unless, of course, the information came from
internal-only sources.
- bill
....
Could you point me to such detailed descriptions of ZFS as that which
included this juicy snippet (a feature that I've been pursuing myself at
my usual glacial pace)? Unless, of course, the information came from
internal-only sources.
- bill
C
Casper H.S. Dik
Bill Todd said:
There's no current substantial technical information available; there's a
question and answer session at:
http://www.sun.com/emrkt/campaign_docs/icee_0703/transcript-SEE-091504.pdf
which covers some of this in more detail.
but you must understand we're all paid to sprinkle ZFS teasers in
news groups and blogs
The basic model for checksumming is fairly simple: all data is interconnected
through pointers and with each pointer a checksum of the data at the end
of the pointer is stored.
Casper
C
CJT
Casper said:
Here's the Q/A I find telling:
ShuChih (Q): When will ZFS be included in Solaris 10? We were told first
in late summer 2004, then early 2005,
then May 2005....
Brian Ellefritz (A): ZFS will be in Solaris 10 when we ship the product.
The current projection is the end of
calendar 2004.