Safest long terms archival media?

[Kaptain Apokalypse]

Hello everyone. This is my first post to the NG so apologies if this has
already been covered. I did a quick search here and didn't find anything
similar.

I wonder what this group's views are on the best long-term data storage
media is? In my particular case I would like to archive digital images and
documents, of which I have many thousands of.

I suppose the cheapest and easiest way to archive data is CD/DVD however I
do not know how reliable such media would be over a period of say 20+ years.
Is this dependant on the quality of the CD/DVD media, or is CD/DVD not a
reliable long term storage solution? What about USB memory sticks? The 2
gigabyte sticks are fairly cheap at the moment, but do they offer a safe and
long-lasting storage solution?

Your advice would be appreciated as would any suggestions for more reliable
data storage methods.

Mike.

 

[Fabien LE LEZ]

n Tue, 08 Aug 2006 16:47:44 GMT, "Kaptain Apokalypse"

I wonder what this group's views are on the best long-term data storage
media is? In my particular case I would like to archive digital images and
documents, of which I have many thousands of.

What's the total size?

I suppose the cheapest and easiest way to archive data is CD/DVD however I
do not know how reliable such media would be over a period of say 20+ years.

No media bought today can be called "reliable" over a period of more
than a few years.

You must be prepared to "refresh" the data every few years (i.e. copy
it onto a new media).
The good news is, if you have 10 DVD-Rs now, you'll only have two
blu-ray disks after the next "refresh".

Another reason to "refresh" the data onto new media is the
compatibility. Imagine: today, you find an old 5"1/4 floppy in your
attic; how do you read it?

What about USB memory sticks? The 2
gigabyte sticks are fairly cheap at the moment,

If you have lots of data, an external hard drive would be cheaper.

but do they offer a safe and long-lasting storage solution?

I wouldn't bet on it.
And also, a little mistake and everything is overwritten. That's the
same with hard drive, but not with DVD-Rs.

If you want reliability, you'll need redundancy.

You can have a RAID-1-like redundancy, i.e. burn two copies of the
DVD-R batch.

And/or you can get RAID-5-like redundancy: you burn your batch of
DVD-Rs, and you burn one or two DVD-Rs containing PAR2 files (see
<http://www.quickpar.org.uk/>).

You may want to use different brands of DVD-Rs, just in case a brand
(or batch) has a hidden flaw.

Having a copy on DVD-Rs and a copy on hard drive is also a good
alternative.

Of course, if you have several copies of the same data, don't store
them in the same place!




Note: make sure you use a widely-used format for your files. If
possible, human-readable.

Example: word processor files. The Microsoft Word .doc format might
look like a good idea; but knowing Microsoft, there's a high
probability that it'll be pretty hard to read in several years.
The RTF format, OTOH, is pretty much human-readable, so writing a
filter to decode/render it wouldn't be too hard even if the
specification is lost. Same goes with OpenOffice's format AFAIK.

Also, putting a decoder (with the source code, if possible) along the
data is definitely a good idea.

 

[Arno Wagner]

n Tue, 08 Aug 2006 16:47:44 GMT, "Kaptain Apokalypse"
<[email protected]>:

What's the total size?

Yes, what volume we are talking about here.

No media bought today can be called "reliable" over a period of more
than a few years.

For CD/DVD they will be reliable over a few years only if the
combination of burner and media is well tuned and the
media are stored in the right conditions.

You must be prepared to "refresh" the data every few years (i.e. copy
it onto a new media).
The good news is, if you have 10 DVD-Rs now, you'll only have two
blu-ray disks after the next "refresh".

That is true for mass-market media. MOD and archival take give
you far better archive times. MOD is currently at >50 years
and in the form om 3.5" MOD it is still affordable for private
use. The drawback is that the largest media are 2.3GB in size,
that a drive is about 500EUR/UDS and that a 2.3GB medium is
about 20EUR/UDS. On the other hand these are oprical removable
disks, behave just like a removable HDD (e.g. in an USB enclosure)
and are reliably overwritable.

Another reason to "refresh" the data onto new media is the
compatibility. Imagine: today, you find an old 5"1/4 floppy in your
attic; how do you read it?

Again a reason to use MOD. The manufacturers are commited to
support at least the last 3 media generation. In practice current
drives support all 5 media generations for reading and writing.
The oldest is 128MB MODs froms omething like 15 years back.

Don't depend on them for more than 5 years. Flash is today usually
specified only for up to 10 years and cheap flash, especially
the currently used multi-level cells, is worse.

If you have lots of data, an external hard drive would be cheaper.

I wouldn't bet on it.
And also, a little mistake and everything is overwritten. That's the
same with hard drive, but not with DVD-Rs.

And not MOD. It jas write protection just like a floppy.
Incidentially most memory sticks also have write protection.

If you want reliability, you'll need redundancy.

You can have a RAID-1-like redundancy, i.e. burn two copies of the
DVD-R batch.

And/or you can get RAID-5-like redundancy: you burn your batch of
DVD-Rs, and you burn one or two DVD-Rs containing PAR2 files (see
<http://www.quickpar.org.uk/>).

You may want to use different brands of DVD-Rs, just in case a brand
(or batch) has a hidden flaw.

Having a copy on DVD-Rs and a copy on hard drive is also a good
alternative.

Of course, if you have several copies of the same data, don't store
them in the same place!

Well, IMO this is far to complicated. Use a medium designed

Note: make sure you use a widely-used format for your files. If
possible, human-readable.

Agreed. That is a concern.

Example: word processor files. The Microsoft Word .doc format might
look like a good idea; but knowing Microsoft, there's a high
probability that it'll be pretty hard to read in several years.
The RTF format, OTOH, is pretty much human-readable, so writing a
filter to decode/render it wouldn't be too hard even if the
specification is lost. Same goes with OpenOffice's format AFAIK.

Or PDF and PostScript. Ascii is best, of course. For images
gif should be ok, PNG is definitely good, but the best is
uncompressed TIFF. Unfortunately it is extremely large.
JPEG is a risk, but might work.

Also, putting a decoder (with the source code, if possible) along the
data is definitely a good idea.

If you have it. Agreed. If you don't have it you should consider
going to a different format.

Arno

 

[Fabien LE LEZ]

Well, IMO this is far to complicated. Use a medium designed
for archival instead. DVD-<whatever> is not for archival.

And if the place you store the media in, burns, what happens to the
data?

 

[Rod Speed]

Hello everyone. This is my first post to the NG so apologies if this has already been
covered. I did a quick search here and didn't find anything similar.

It has been discussed here a number of times, but thats no big deal.

I wonder what this group's views are on the best long-term data
storage media is? In my particular case I would like to archive
digital images and documents, of which I have many thousands of.

Generally the best approach is another hard drive if the volume involves
more than a few DVDs. If the data is very valuable/irreplaceable,
you might choose to have more than one hard drive copy.

Its important to have the copy outside the house, basically
so that if the house burns down etc, the copy isnt affected.

I suppose the cheapest and easiest way to archive data is CD/DVD however I do not know
how reliable such media would be over a period of say 20+ years.

Its reliable enough if you make more than one copy and check
the copys occasionally so you can detect corruption early.

It can be quite a viable approach if you say write new stuff to more
than one DVD when you have a new DVD full. Not so practical if you
currently have a 500G drive stuffed with data that hasnt been backed up.

Is this dependant on the quality of the CD/DVD media,

Only in theory. I did originally use the gold CDs for that reason, but it
turns out that I have never lost any of any type, so dont bother anymore.
I do however always write the copys to different brand media so if it
turns out that one brand is flakey over time, you have the other copy.

And its important to not keep all those copys in the one place too.

or is CD/DVD not a reliable long term storage solution?

Its reliable enough with the correct approach.

What about USB memory sticks? The 2 gigabyte sticks are fairly cheap at the moment, but
do they offer a safe and long-lasting storage solution?

Not as good as hard drives and DVDs, and only really viable if
the total space is enough to take all stuff you need to backup.

Your advice would be appreciated as would any suggestions for more reliable data storage
methods.

Ignore tape, its past its useby date for personal backup now.

I dont think MOD is a viable approach myself, its already an
obscure system that is quite expensive and may well vanish.
CD/DVD/hard drives wont any time soon.

 

[Kaptain Apokalypse]

Hello everyone. This is my first post to the NG so apologies if this has
already been covered. I did a quick search here and didn't find anything
similar.

I wonder what this group's views are on the best long-term data storage
media is? In my particular case I would like to archive digital images and
documents, of which I have many thousands of.

I suppose the cheapest and easiest way to archive data is CD/DVD however I
do not know how reliable such media would be over a period of say 20+
years. Is this dependant on the quality of the CD/DVD media, or is CD/DVD
not a reliable long term storage solution? What about USB memory sticks?
The 2 gigabyte sticks are fairly cheap at the moment, but do they offer a
safe and long-lasting storage solution?

Your advice would be appreciated as would any suggestions for more
reliable data storage methods.

Mike.

Thanks to everyone for the advice so far.

To answer a couple of questions, the data I wish to store amounts to only
around 8 gigabytes, and I don't expect this to exceed around 10-15gb so it's
not a huge amount.

 

[Rod Speed]

Thanks to everyone for the advice so far.

To answer a couple of questions, the data I wish to store amounts to
only around 8 gigabytes, and I don't expect this to exceed around
10-15gb so it's not a huge amount.

I'd write it to multiple DVDs, different brand media, more than one
copy of each on different DVDs, check the DVDs for readablility using
Nero's Scandisk every few years, copy the other good copy if any go bad.

And keep the DVDs in a different house to where you have
the main PC they are an archive of, in case it burns down etc.

 

[timeOday]

Hello everyone. This is my first post to the NG so apologies if this has
already been covered. I did a quick search here and didn't find anything
similar.

I wonder what this group's views are on the best long-term data storage
media is? In my particular case I would like to archive digital images and
documents, of which I have many thousands of.

I suppose the cheapest and easiest way to archive data is CD/DVD however I
do not know how reliable such media would be over a period of say 20+ years.
Is this dependant on the quality of the CD/DVD media, or is CD/DVD not a
reliable long term storage solution? What about USB memory sticks? The 2
gigabyte sticks are fairly cheap at the moment, but do they offer a safe and
long-lasting storage solution?

Your advice would be appreciated as would any suggestions for more reliable
data storage methods.

Mike.

You could also look into online data backup services. I haven't really
looked into it, but I think I would consider it if all the following
were true:

1) price below $2/GB/year
2) $1000/GB written guarantee against data loss (other than accidental
deletion on my part)
3) guarantee of data retention for 3 years even if I stop paying,
without too big a fee to regain acess to data.
4) terms of service are fixed for 4-year periods.

It would also help a lot if it were a big, reliable company with a solid
track record, e.g. a bank.



Of course, you could still possibly lose your data. You might forget
what company your account was with, or might forget your encryption key.
And you'd still have to avoid format obsolescence.

 

[Arno Wagner]

On 8 Aug 2006 18:17:01 GMT, Arno Wagner <[email protected]>:

And if the place you store the media in, burns, what happens to the
data?

If it is arcival, then put it into a bank vault. They really do not
burn well. Not that expensive, I once had one about 5cm x 25cm x 50cm
for 40EUR/USD (only for suctomers) per year for this purpose.

Other alternative is two copies, one stored at somehwere else.

Arno

 

[Arno Wagner]

Thanks to everyone for the advice so far.

To answer a couple of questions, the data I wish to store amounts to only
around 8 gigabytes, and I don't expect this to exceed around 10-15gb so it's
not a huge amount.

Then I would advise MOD or high quality memory stick. Some manufacturers
do state a data retention of 30 years. Check that in the datasheet.
And have two copies in different locations, just to be sure.

Arno

 

[Arno Wagner]

You could also look into online data backup services. I haven't really
looked into it, but I think I would consider it if all the following
were true:

1) price below $2/GB/year
2) $1000/GB written guarantee against data loss (other than accidental
deletion on my part)
3) guarantee of data retention for 3 years even if I stop paying,
without too big a fee to regain acess to data.
4) terms of service are fixed for 4-year periods.

Good idea. With the stated 10-15GB, that may be the cheapest of
the reliable options.

It would also help a lot if it were a big, reliable company with a solid
track record, e.g. a bank.

Of course, you could still possibly lose your data. You might forget
what company your account was with, or might forget your encryption key.

Same as with conventional storage: Store the reference in several places.
Sealed envelopes usually work well. If you have a lawyer, they might
store one for you. You can also keep one at work, one with friends,....

And you'd still have to avoid format obsolescence.

That you do. But filesystem obsolescence will not be an issue, since
the storage service will take care of that.

Arno

 

[Neil Maxwell]

I'd write it to multiple DVDs, different brand media, more than one
copy of each on different DVDs, check the DVDs for readablility using
Nero's Scandisk every few years, copy the other good copy if any go bad.

Also, be sure to burn them at half the media's rated speed, and do a
post-burn compare to make sure the initial burn is good.

Don't use the cheapest media you can get. Taiyo Yuden has a good
reputation, and it's had much longer life in CDR than any other brand
I've tried personally. I don't have much failure experience with DVDR
yet, but I'm gathering the data...

 

[Rod Speed]

Also, be sure to burn them at half the media's rated speed,

I've seen no evidence that that is necessary.

and do a post-burn compare to make sure the initial burn is good.

A compare isnt necessary, just the readability scan that you will
be doing every few years done just after the write would be fine.

Don't use the cheapest media you can get.

Sure, particularly with DVDs.

Taiyo Yuden has a good reputation, and it's had much longer
life in CDR than any other brand I've tried personally.

I havent had any failures at all with any CD media except with
one batch of purples that someone else got for me that were fine in
my burner but which plenty of other CD drives had a problem reading.
They were unusual in the sense that you can see right thru them.

I don't have much failure experience with
DVDR yet, but I'm gathering the data...

I have had problems with one batch of super cheaps that
had a problem at the maximum capacity end of the DVD.
Only used them to post avi files on to some pathetic wretch
that is stuck on dialup, but other than that, they've been fine too.

 

[J. Clarke]

Thanks to everyone for the advice so far.

To answer a couple of questions, the data I wish to store amounts to only
around 8 gigabytes, and I don't expect this to exceed around 10-15gb so
it's not a huge amount.

For 8 GB, a DLT drive would be a good bet. DLT has a solid track record
going back 30 years and you can get 10/20GB drives in good condition dirt
cheap on ebay (they're super high quality drives that generally get on ebay
because they're no longer large enough for corporate storage, not because
they're dead). Write to three or four tapes and put them in different
places including sending one to your auntie who lives several thousand
miles away and you should be covered for just about anything short of
global thermonuclear war.

Magneto-optical is another good bet but you're talking more expensive drives
and media.