Archiving Solutions.

  • Thread starter Thread starter Jack Frillman
  • Start date Start date
J

Jack Frillman

What is the recommended way of archiving scans?

The main goal is to get the images archived on DVDs before time takes
more of a toll on the originals. Some are 40-50 years old.

I am using a Minolta 5400 with Vuescan or the Minolta software on a
Linux system.
If I use the Minolta SW it's done on a Mac Powerbook.

The options for archiving I see are:

1 - Just doing raw scans and saving these for later processing.
2 - Saving as processed scans as a finished product.
3 - Use the archive option in Vuescan. Thing is I don't how this option
differs from the print and other options.
 
1 - save raw - you will never get the original scan if you do an
"oops" in processing

2 - every 5 years move scans to the next volume storage media - this
year dvd, 5 years from now magnahelic memory, 10 years from now
holographic crystal - technology changes - how many 8-inch floppy
disks can you read from 15 years ago or 5 1/4 inch from 10 years
ago???
 
I take two copies (on two different DVDs or CDs of course) of each RAW
scan and each edited scan. Right now I am archiving the edited scans as
24 bit/pixel, Adobe RGB, after unsharp mask, as they were just before I
printed them. The RAW scans as 48 bit/pixel, as saved by Vuescan.

Fanta
 
Jack,

As well as taking the sound advice in this thread make sure you have
two copies which are kept at different locations. Fire etct can lead to
the loss of everything on one site.

Michael
 
1 - save raw - you will never get the original scan if you do an
"oops" in processing

2 - every 5 years move scans to the next volume storage media - this
year dvd, 5 years from now magnahelic memory, 10 years from now
holographic crystal - technology changes - how many 8-inch floppy
disks can you read from 15 years ago or 5 1/4 inch from 10 years
ago???
Click to expand...

Can the raw scans made from Vuescan be used by something other than
Vuescan? I guess my concern is for some reason in the future I can't
use Vuescan can I process those raw scans with something else?

That might be stupid question since the raw scans are tiff files.
 
Good points.
Can the raw scans made from Vuescan be used by something other than
Vuescan? I guess my concern is for some reason in the future I can't
use Vuescan can I process those raw scans with something else?
That might be stupid question since the raw scans are tiff files.
Click to expand...

If Vuescan's "raw" format comes out as a TIFF, it'll be readable for a
very very long time. TIFF is an open standard and just about every
program that works with images can hack it. The main thing to worry
about is the compression method and bits-per-sample you use. Some
software packages can't grok PackBits or JPEG-compressed TIFF. Some
software packages can't handle 16 bits/sample (though this is becoming
more common.) If you're scanning color/grayscale, you're almost
certainly using LZW, so don't worry about compression. If you're
scanning at 16 bits/sample, remember that currently, Gimp 2 will
downsample to 8 bits internally even though it will open the file.
(This is going to be improved soon, IIRC.)
 
o Check your backups are readable
o As mentioned 2 copies at different locations
o Ideally those 2 copies on 2 different media types
---- DVD is one
---- DVD-RAM another (altho more expensive, more error correction & phase-change)

Remember media changes every few years - so change with it.

Some people like mini-servers with cheap SATA drives:
o That can keep images online
o It can also keep images regularly examined, CRC'd etc
o Before the decade is out we will be far above 1,000GB on 3.5" HD
o You can also keep the unit somewhere else - relatives & broadband

Verification of any backup is critical.
 
Dorothy Bradbury said:
o Ideally those 2 copies on 2 different media types
---- DVD is one
---- DVD-RAM another
Click to expand...

Is there any DVD media that is considered archival?
(apart from the additional problem of interoperability -
not being able to read media just written on another
brand or model of DVD burner).

When the data set is small enough, or can be made small
enough, I'd lean toward putting it on CD-R, since there
is [claimed] archival media for that, and interop seems
to be less of a problem vs. DVD.

Here's a brand of archival CDR:
<http://store.mam-a-store.com/standard---archive-gold.html>
I haven't used any myself, but plan to try it on my
next media buy.

And yes, if you haven't restored it lately,
it isn't really backed up.
 
Is there any DVD media that is considered archival?
(apart from the additional problem of interoperability -
not being able to read media just written on another
brand or model of DVD burner).
Click to expand...

The most "archival" is DVD RAM which is rat4ed for a far higher
number of uses and comes in a caddy to stop you getting your sticky
fingers all over it :)
When the data set is small enough, or can be made small
enough, I'd lean toward putting it on CD-R, since there
is [claimed] archival media for that, and interop seems
to be less of a problem vs. DVD.
Click to expand...

I wouldn't. CDs are the most likely to degrade compared to DVDs
because of the way they are made and store data.
 
The most "archival" is DVD RAM which is rat4ed for a far higher
number of uses and comes in a caddy to stop you getting your sticky
fingers all over it :)
Click to expand...

Yes, although re caddy/cartridge...
o IIRC only Panasonic do DVD-RAM writers for DVD-RAM cartridge
o LG do DVD-RAM writers (LG-4082B) that need the cartridge opened

Not that big a deal if you take care - if obsessive wear white cotton gloves,
you would if it were very important paper documents so why not media.

DVD-RAM is very slow.

Will be interesting to see how UBO Blue-Laser MO 5.25" does:
o Super high capacity - several 10s of Gigabytes eventually
o Very high speed - far faster than DVD-RAM
o Very reliable, both DVD-RAM & MO are phase-change media
o Still probably very expensive - 20x the price of an LG DVD-RAM drive

On balance, buy 2 brands of DVD-RAM drive & 2 brands of DVD-RAM media:
o Slim DVD-RAM drive from Panasonic can be had quite cheaply (eg, laptop, USB2)
o Full size DVD-RAM drive from LG is cheap too (eg, desktop)

That way you can test readability in both drives - ie, that your drive is managing to
create disks that only it can read after a period of time (had that on old CDR drives).
Also, if a drive fails you can still access or recover your data in a hurry.

DVD-RAM disks aren't that expensive, if you do 2 copies of data I'd say put one
on DVD+/-R, then one on DVD-RAM (which is reusable, but write protectable).
DVD-RAM is not only phase-change but has more error correcting capability,
a long way towards traditional MO solutions without high drive/media cost.
 
Is there any DVD media that is considered archival?
Click to expand...

Why worry about that? Almost any decent brand of CD or DVD will be
"readable" long after there are no more drives to read it, or the file
format is no longer universally recognized. If it's a decent CD or
DVD, kept out of the light (particularly bright sun light), kept
reasonably cool and physically protected, it will last long enough to
migrate to the next latest-greatest technology in either file format
and/or media type. How many 8-track players (that still work) can you
find these days to play any old 8-track tapes still laying around the
house?

C.R.
 
Why worry about that?
Click to expand...

Because I have mid-1990s Kodak PhotoCDs that are already
unreadable, despite ideal handling and storage. And
they went unreadable several years ago.
Almost any decent brand of CD or DVD will be "readable"
long after there are no more drives to read it, ...
Click to expand...

a. I wouldn't count on the media being readable. Some
of the cheaper blanks, esp. Russian, are close to
unreadable right off the spindle stack. I've run into
some that were unwriteable right off the spindle,
and the very first CD+RW I ever used immediately
failed to read.

b. We're going to be able to read (readable) 120mm media
for a very long time to come.
... or the file format is no longer universally recognized.
Click to expand...

That is a consideration (for Kodak .PCD for example, but
probably moreso for various camera raw files), but probably
not for .JPG (or even .BMP and .TIF), as long as we steer
clear of the near-proprietary extensions (like JPEG
compression inside TIFF).
How many 8-track players (that still work) can you find ...
Click to expand...

Counter example: you can still buy a turntable that will
play a 100 year-old 78 rpm record. Why? Because it was
easy to build later disk players to handle the earlier
media - and because the 78 rpm media was more pervasive
than 8-track - and because shellac records were more
archival than tape.

Similarly, the 120mm media size chosen for CD, then CD-ROM,
was also adopted for DVD and will be used for the HD
formats. Each new drive family could read all the prior
encodings on the same size platter (well, perhaps excluding
CDV-5, but only the analog data on that).

Any "archival" effort requires stewardship, including
immediate as well as periodic data validation, plus
a data migration plan. But it helps to start with media
that has at least some promise of a long shelf life.
I'm nervous about all DVD media in that role.
 
Why worry about that? Almost any decent brand of CD or DVD will be
"readable" long after there are no more drives to read it, or the file
format is no longer universally recognized. If it's a decent CD or
DVD, kept out of the light (particularly bright sun light), kept
reasonably cool and physically protected, it will last long enough to
migrate to the next latest-greatest technology in either file format
and/or media type. How many 8-track players (that still work) can you
find these days to play any old 8-track tapes still laying around the
house?

C.R.
Click to expand...
Actually I have a number of discs from only a few years ago that already
have read errors. These have been stored carefully in a dry, cool, dark
CD file box, each in individual sleeves. Personally I am not convinced
that any CD/DVD technology is anything close to archival at this point
in time.

Jim Couch
 
Personally I am not convinced that any CD/DVD technology is
anything close to archival at this point in time.
Click to expand...

TDK CDR blue were once favoured - those made in India were
lousy, those made in Japan were very good. Yet you couldn't tell
them apart and which you got varied from month to month.

So I'd agree there is a risk in any one media type - you need 2.
o I chose DVD-RAM as vastly cheaper than MO
---- unlike DVD it 1) has better error correction & 2) is phase-change
o I also chose HD because it is online (easily verified) & mirrored
---- by verified I mean inspected regularly
---- by mirrored I mean a simple local/remote server holding a copy

CDR media quality went poor as price was forced down, that may
happen with DVD, and potentially with DVD-RAM eventually.

I also use 3.5" MO, you may think that is invulnerable - wrong:
o As MO has had to compete with new media on price, QC dropped
o So as MO volume drops, so economics drop - opposite of new media
o One batch of media had a surface defect - Philips
o One batch of media was miss-formatted at the factory - drive maker brand!

So perhaps there is a degree of "quality sweet-spot" with backup media
where pricing allows all quality steps before it moves into cash-dog area.

So it is a case of migration

There is one other technology to consider:
o I would first caution that I prefer 2 media types v 1 expensive media type
o DLT has dropped in price considerably with the new slimline 40/80GB drives
o You can get a main-brand 40GB DLT drive for ~£450, probably $450-550
o Tapes are expensive, but their quality is real - the world uses DLT

DLT is Digital Linear Tape, derived from the big open reel systems.
It is different to tape systems developed from audio - eg, Digital Audio Tape
which is a Helical Scan technology which has improved but still =/= DLT. As
the cheaper (yet mainbrand, eg, Quantum) drives are now == DAT, it's DLT.

The Sony AIT & other VX standards I'm not too keen on - less reliable than
DLT by many reports and more importantly fewer vendors making the drives.

The more expensive the drive, the more reluctant someone is to change with
technology re sunk-cost - DVD-RAM drives have the benefit of being 50-70$.
With 2.5" HDs rugged/cost pricing, they aren't so far above DLT tapes either.
 
CDR media quality went poor as price was forced down, that may
happen with DVD, and potentially with DVD-RAM eventually.
Click to expand...

DVD-RAM has lost the mass-market. That means fewer vendors offering
DVD-RAM media, which could screw DVD-RAM owners eventually since media
is fairly low-margin.
DLT is Digital Linear Tape, derived from the big open reel systems.
It is different to tape systems developed from audio - eg, Digital
Audio Tape which is a Helical Scan technology which has improved but
still =/= DLT. As the cheaper (yet mainbrand, eg, Quantum) drives are
now == DAT, it's DLT.
Click to expand...

DLT is pretty damn reliable IME. I've never had a serious read or write
error on the 3 DLT systems at work. The DDS systems are much more
finicky and will sometimes throw up errors reading media they wrote
successfully less than a week ago.
The Sony AIT & other VX standards [...] less reliable than DLT by many
reports and more importantly fewer vendors making the drives.
Click to expand...

? AFAIK, all DLT drives are made by Quantum. The "HPCompaq" or
"Dull" logo on them is just a branding stamp and has nothing to do with
the guts of the drive.

Any really long-term archival project will have to do media migration
every so often if only for space/time. I don't relish the thought of
putting the ~200 DDS tapes we have at work onto whatever new tape system
we eventually get, but it'll certainly save physical space and sysadmin
headaches to have 50 backup tapes instead of 200.
 
There is one other technology to consider:
o I would first caution that I prefer 2 media types v 1 expensive media type
o DLT has dropped in price considerably with the new slimline 40/80GB drives
o You can get a main-brand 40GB DLT drive for ~£450, probably $450-550
o Tapes are expensive, but their quality is real - the world uses DLT

DLT is Digital Linear Tape, derived from the big open reel systems.
It is different to tape systems developed from audio - eg, Digital Audio Tape
which is a Helical Scan technology which has improved but still =/= DLT. As
the cheaper (yet mainbrand, eg, Quantum) drives are now == DAT, it's DLT.
Click to expand...

This is the approach I'm taking. Each time I accumulate 40GB of images, I make
two backups onto 40GB DLT. One goes in the fire safe, the other goes in to
work with me. I'll end up with about 80 pairs of tapes, unless I switch to
some other system before I'm done.

I had endless problems with DAT drives, as did the company I work for. They
switched to DLT and never saw another read failure.

John
 
What is the recommended way of archiving scans?

1- Clean your originals using StaticVac; http://www.kinetronics.com/

2- Make shure that your scanner/computer setup get clean power, since
any noise in the line translates into noise in the A/D converter. A
PowerVar ower conditioner at minimum, or a Power Regenerator at best;
www.powervar.com

3- Profile and calibrate your scanner with an IT-8 target;
http://www.coloraid.de/

4- Scan in a ProPhoto RGB color space; (the scanner's raw color space is
proprietary and it may be difficult to have it in the future)

5- Scan at your scanner's maximum optical resolution in 48-bit/RGB

6- Avoid Processing such as sharpening, effects, etc (Do this on your
finals);

7- burn them on Medical Grade DVD-R using AccuBurn-R software, or XVA-2
data tapes. http://www.digitalfaq.com/media/dvdmedia.htm
and
http://www.infinadyne.com/accuburn-r.html

8- Store both the originals and data in a temperature controlled place.

Sounds overkill? Probably, but you asked for "Archive Quality". If you
want "Professional Quality" Then feel free to substitute 2 or 3 of the
above. If you want "Medical or Military Quality", then add: 9 - use an
ICG 360 drum scanner hooked up to a Balanced Power outlet. If you want
"good enough quality", then anything goes here. [you name it].
 
With fire safe's watch how the fire rating is achieved:
o Some use proper fireproofing materials
---- impregnated foams, saffill, treated concrete
o Some use a gel which is saturated with moisture
---- in a fire the gel is superheated to create steam
---- so the media is exposed beyond temp & humdity limits

Media safe's are relatively ok - and may be used in combination
with those gel type (paper usage) fire safes. However don't forget
to shut the media safe properly re ensuring the seal's integrity.

Going to investigate some of the archive grade DVD myself...
 
7- burn them on Medical Grade DVD-R using AccuBurn-R software, or XVA-2

The choice of DVD-R discs is even more limited if you want archival
quality and a quality burn. Here, it's pretty much the Made In
Japan-only Maxell DVD-R discs (I'm saying these are easy to buy vs.
other quality discs you may have to hunt forever to find).

These are top quality discs, have been through multiple revisions, and
have been tested by emedia & dv magazine and may other users and found
to be among #1 in compatibility across a variety of DVD players after
being burned in top-quality burners (not point using a medicore DVD
burner for archival purposes) for many years.

It's basically the same disc Apple uses (rebranded), and in a good
burner (eg. Plextor, Pioneer), they burn with low PI/PO error rates (see
www.cdrinfo.com for burner reviews with PI/PO error rate charts).

Other brands? Who knows.....

---

Besides this, also burn a second copy to any other brand/media
because you never know.....you may just lose the only one you have.

---

Finally, verify after writes (can be turned on in Nero DVD burning
program for example). This includes the physical media itself as well
-- here, use a LiteOn (or BenQ, etc.) burner with PI/PO read capability
(see www.rpc1.org -> forums for help on this).

You need to do this because a marginal disc that passes all verify
after writes is still a marginal disc. Here, you need to make sure the
PI/PO error rates are actually low enough not to cause a read problem in
the near future.

---

8 - add ot that humidity controlled as well. water can damage discs
faster than temperature....

---------

use a good burner. Pioneer A08 or Plextor 712 are solid choices as
they'll make good burns with low PI/PO error rates across a wide variety
of discs.


====

Here, burning to CD-R discs may be an even better choice.

Use Mitsui Gold CD-R discs (100% gold reflective layer) and you'll
ensure an even longer archival storage life than DVDs in general.

(partially due to the use of a non-gold reflective layer (ie.corrosion),
and also due to the glued construction of DVDs from two plastic discs)

CDs also have much higher compatibility, readablility, and so forth vs.
DVDs, which are more fragile and new as a technology. After all, almost
all PCs can read CDs, but quite a few PCs still can't read DVDs.
 
Back
Top