Don is done!!!

  • Thread starter Thread starter Don
  • Start date Start date
D

Don

Well, folks, after 3+ years, 2 flatbed scanners, 2 film scanners and
many false starts - I've finally finished archiving all my images!!!

fx: Champagne bottles popping, fireworks going off, etc... ;o)

Mind you, that's just the first step, the images still have to be
edited as these are only raw scans, but at least I can breathe easier
as the "analog rot" has been halted and all the images are now in the
(lossless) digital domain.

In the end it all fits nicely on 100 DVDs exactly!
64 DVDs of slides
20 DVDs of negatives
16 DVDs of photographs (for which there are no negatives)

Anyway, before I move on I just want to once again publicly thank all
those who have been helpful along the way. Notable mention: Kennedy!

Time permitting, I may pop in from time to time to see how the gang is
doing but the group seems to be winding down as more and more people
finish scanning and move over to digital for good.

Don.
 
Well, congratulations, and I'm sure you'll have fun revisiting some of
those images from decades past.
 
Don,

I'm sure you've informed us before, but for those of us that don't
frequent this group on a regular basis, can you let us know under what
"specs" you scanned your photos? TIF? 48-bit? Also, what brand of DVD
did you choose and do you also have a copy on a hard disk? What editing
do you plan on doing to your photos?

Thanks Don!
 
Thanks Roger! Much appreciated. It was a very nostalgic (and at times
emotional) trip down memory lane as one remembers people and places.

BTW, all the best with your archiving!

You've got many more to do than I ever did but I can happily report
there is light at the end of the tunnel and it's not an oncoming
train! ;o)

Don.
 
Don,

I'm sure you've informed us before, but for those of us that don't
frequent this group on a regular basis, can you let us know under what
"specs" you scanned your photos? TIF? 48-bit? Also, what brand of DVD
did you choose and do you also have a copy on a hard disk? What editing
do you plan on doing to your photos?

Thanks Don!

Hi Tim!

I did everything as TIF because TIF is an open standard and bound to
be supported in future. Besides, if push comes to shove, since the
specs are known I can even write a converter myself, if need be. The
main purpose was to archive and stop any further deterioration of the
original film/photos so I used 48-bit depth.

I had particular trouble with Kodachromes (and to a lesser extent
slides in general) because of dynamic range issues. Even though my
scanner (Nikon LS-50) has 14-bits of dynamic range and, therefore,
over 1.5 bits of headroom, I found that 14-bits was just not enough.
The shadows still had far too much noise. Especially Kodachromes.

So I ended up scanning each slide twice. Once, using normal exposure
(to get the highlights) and the second time at +~3.5 AG/ev boost (to
get the shadows without noise). It was a bit more complicated than
that because I had to use different boost for individual RGB channels
due to the "weird" Kodachrome characteristic film curve. Moreover,
Kodachromes changed over the years so I had to modify that as well.

In post processing, I will then combine the two scans using the
highlights from one and the shadows from the other. However, before I
do that I will have to adjust the color balance of the boosted scans
in order to get a seamless join even if it falls in the middle of a
gradient! I've done all the tests to make sure all this works before I
even started.

Regarding DVDs, in the end, I settled on Verbatims. From the
literature, the dyes Verbatim uses are considered to be "long lasting"
and stable. Verbatim also seem to have good quality control. Of
course, DVD longevity is really guesswork because we don't really
know. Time alone will tell. So, I've done two DVD backups and have the
copies of all images on my (external, USB) hard drive(s) for instant
access and post-processing. It all fits nicely on two 250 GB drives.
The DVD writer I used is the external LG GSA-2166.

Of course, I will have to check the DVDs at regular intervals and
re-do any that start to fail. But I guess before that happens the
HD-DVD will become common and my archive will shrink to 10 HD-DVDs.
The same check goes for hard disks which need to be refreshed to avoid
"bit rust".

Don.
 
Thanks Don! You answered everything just as I hoped you would - I'm
working on scanning all of my photos, though not nearly as many as you
have, and so far I've been following the same path you did. So I guess
I'm on the right track.

On top of what you've done, I will probably export each edited TIF in
high quality JPEG and upload to Flickr for all the family to see. Now
that they've implemented geotagging, it'll be very cool to tag every
photo with a location, in addition to descriptive tags, to provide a
real context to each photo. I'm very excited! Someday when [hopefully]
they increase the maximum photo size and add TIF support, I'll upload
the originals. That should provide me an additional backup source.

Thanks again Don!
 
On top of what you've done, I will probably export each edited TIF in
high quality JPEG

That's the next step for me. After editing (removing dust and
scratches, correcting color balance, etc.) I'll eventually end up with
high quality JPEGs as well. The idea is to fit them all on only a few
DVDs which I can always carry around with me.

This is all actually a part of a long term project for me. I call it
"digitizing my life". I moved around a lot in the past and as a
consequence inadvertently lost a fair amount of personal records along
the way so the plan is to be able to carry as much as possible on me.

Apparently this is also a part of academic research and is known as a
"personal petabyte". The calculation is, all the information a person
accumulates over a lifetime (text, images, sounds, etc) amounts to
about a petabyte of data.
and upload to Flickr for all the family to see. Now
that they've implemented geotagging, it'll be very cool to tag every
photo with a location, in addition to descriptive tags, to provide a
real context to each photo. I'm very excited!

I'm doing the same thing after I've gone digital! I've got a GPS and
note down the location of every photo.

As a point of interest, my first digital camera (Kodak 260) had an
option of having the GPS connected directly and automatically storing
the longitude and latitude with the image. I never made use of this
because the cabling was a bit clumsy (my favorite hobby: recreational
cycling) but I think it's only a matter of time - if it hasn't
happened already - before we see a digital camera with a built-in GPS.
Someday when [hopefully]
they increase the maximum photo size and add TIF support, I'll upload
the originals. That should provide me an additional backup source.

I was thinking about the very same thing! It's a "twofer" (two for the
price of one): back-up and publication. The main problem is image size
but, as you say, that is bound to change especially with various DSLs
becoming more prevalent so the up/down-loading doesn't take forever.
Thanks again Don!

My pleasure. And the best of luck with your scanning, Tim!

Don.
 
Don said:
This is all actually a part of a long term project for me. I call it
"digitizing my life". I moved around a lot in the past and as a
consequence inadvertently lost a fair amount of personal records along
the way so the plan is to be able to carry as much as possible on me.

Funny you say that - I've used the same wording to describe my personal
project - photos being only one part. I even started a blog a while
back to keep track of the general move to digitize everything -
http://www.digitizedlife.net/ - I don't update as often as I should...
too much of my time is taken up with scanning.
Apparently this is also a part of academic research and is known as a
"personal petabyte". The calculation is, all the information a person
accumulates over a lifetime (text, images, sounds, etc) amounts to
about a petabyte of data.

That's only about 2000 times the information you've scanned already! Of
course that's all dependant upon compression, etc. Interesting idea
though.
I'm doing the same thing after I've gone digital! I've got a GPS and
note down the location of every photo.

As a point of interest, my first digital camera (Kodak 260) had an
option of having the GPS connected directly and automatically storing
the longitude and latitude with the image. I never made use of this
because the cabling was a bit clumsy (my favorite hobby: recreational
cycling) but I think it's only a matter of time - if it hasn't
happened already - before we see a digital camera with a built-in GPS.

Well that would save some time wouldn't it! Yes, I've been looking
around a [very] little bit and have found only this -
http://www.geospatialexperts.com/ricoh.html. I would love to go buy a
brand new Canon or Nikon though, with GPS built right in without a huge
adapter. It will certainly happen soon.
I was thinking about the very same thing! It's a "twofer" (two for the
price of one): back-up and publication. The main problem is image size
but, as you say, that is bound to change especially with various DSLs
becoming more prevalent so the up/down-loading doesn't take forever.

I thought it funny at first that we both came to the exact same
conclusions when deciding how to preserve information but it's not
strange at all considering the evolution of this technology. At some
point, if you want to keep things around (and why wouldn't you) you're
going to have to digitally preserve them. End of story.

Thanks again Don. Good luck.
 
Funny you say that - I've used the same wording to describe my personal
project - photos being only one part.

Great minds think alike! ;o)
I even started a blog a while
back to keep track of the general move to digitize everything -
http://www.digitizedlife.net/ - I don't update as often as I should...
too much of my time is taken up with scanning.

I'll sneak a peek when I can (I'm offline as I write this) but as you
say, so much to do - so little time... :-(
That's only about 2000 times the information you've scanned already! Of
course that's all dependant upon compression, etc. Interesting idea
though.

I see that petabyte figure more as an arbitrary approximation, really,
because, like you say, so much depends on all the variables i.e.
compression, resolution, bit depth, etc. But it is interesting that
people are starting to think in those terms i.e. how much data does a
person "weigh" in total.
...I think it's only a matter of time - if it hasn't
happened already - before we see a digital camera with a built-in GPS.

Well that would save some time wouldn't it! Yes, I've been looking
around a [very] little bit and have found only this -
http://www.geospatialexperts.com/ricoh.html. I would love to go buy a
brand new Canon or Nikon though, with GPS built right in without a huge
adapter. It will certainly happen soon.

There are also some drawback to GPS e.g. it doesn't work indoors.
However, in North America there are additional terrestrial (radio?)
signals which some standalone GPS devices use when the satellite
signals are out of range or weak. I remember reading about this in the
manual, years ago, when I got my GPS (Garmin 12CX). I think it was
called "differential..." something. Also, some of the new cell/mobile
phones can use their signals to roughly (!) triangulate the location.

Don.
 
Don said:
Well, folks, after 3+ years, 2 flatbed scanners, 2 film scanners and
many false starts - I've finally finished archiving all my images!!!

fx: Champagne bottles popping, fireworks going off, etc... ;o)

Mind you, that's just the first step, the images still have to be
edited as these are only raw scans, but at least I can breathe easier
as the "analog rot" has been halted and all the images are now in the
(lossless) digital domain.

In the end it all fits nicely on 100 DVDs exactly!
64 DVDs of slides
20 DVDs of negatives
16 DVDs of photographs (for which there are no negatives)

Anyway, before I move on I just want to once again publicly thank all
those who have been helpful along the way. Notable mention: Kennedy!

Time permitting, I may pop in from time to time to see how the gang is
doing but the group seems to be winding down as more and more people
finish scanning and move over to digital for good.

Don.

Don,

I don't know you from Adam but I'm green with envy that you're done and also
very happy for you ;-) A long row to hoe but worth it I believe. I'm
scanning when I get time but my wife is already starting to notice my
absence.

Wondering if you can take a minute to tell me (us) which scanners you used
along the way and what made you move onto the next (worse or better) model?
Any euphoric or horrific moments would be interesting and probably fun to
hear about.

Also, thanks for relaying the information on TIF, slide scanning etc. to
help the rest of us along.

Once again, congratulations!
PK
 
Don,

I don't know you from Adam but I'm green with envy that you're done and also
very happy for you ;-)
:-)

A long row to hoe but worth it I believe.

Oh, definitely! Not only because of the end result, but in the process
I also organized my images. Many weren't dated so I had to create a
timeline first. The date-stamped slides were a great help here, but
for negatives and photos it took a lot of head scratching, going
through old passports to figure out when a particular trip took place
or just looking at the images and comparing them to dated slides, etc.

In the end, I organized them all purely chronologically. Each film has
its own directory in the format: YYYY.MM. That way they all sort
nicely. If I had more than one film in a given month I would either
add the day (.DD) or, failing that, add a suffix (either "-1", "-2" or
"-A", "-B", etc to sequence them). Of course, in some cases I was only
able to narrow it down to the year only. Within those directories, the
images are then numbered according to their number on the film.

Those are all "raw" scans intended purely for archiving. Once I
process them and convert to JPGs "for consumption" I'll use a
different strategy. Probably several depending on intended use.
I'm
scanning when I get time but my wife is already starting to notice my
absence.

:-) Yes, scanning can be a lonely business! Especially if (like me)
one agonizes over all the details.
Wondering if you can take a minute to tell me (us) which scanners you used
along the way and what made you move onto the next (worse or better) model?
Any euphoric or horrific moments would be interesting and probably fun to
hear about.

Oh, boy... There are a lot of horrific moments, that's for sure!

I've been planning to do this for a long time so, in preparation,
about 7-8 years ago, I got the ScanMaker E6 flatbed (8-bit, 600 dpi)
and Nikon LS-30 film scanner (10-bit, 2700 dpi). But, life got in the
way, and the scanners sat unopened in their boxes for several years.

When I finally got around to it, both scanners were hopelessly
outdated and overtaken by technology. I did try the LS-30 briefly but
quickly ran into the "Kodachrome problem". Nikon's so-called "support"
was no help at all, refusing to even admit Nikons have a problem with
Kodachromes (the scans are dark, noisy and have a blue cast). That was
my very first post here!

To cut a long story short (too late! :-)) I then shelled out for the
Nikon LS-50 film scanner (14-bit, 4000 dpi) and Mustek BearPaw 4800TA
Pro flatbed (16-bit, 4800 dpi).

The reason I went with Nikon again, is because of the light source
(LED) which produces pure colors and doesn't drift over time like
conventional light sources. It also results in very sharp scans.
Besides, Nikons are built like a tank!

Still, the Kodachrome blue cast persisted in spite of special LS-50
Kodachrome setting (absent on the LS-30). In the end, I wrote my own
scanner program after getting the developer kit out of Nikon. Nothing
fancy, just for my own use, simply getting as raw a scan as possible.

The trouble is, as I hinted in the other message, the dynamic range of
the LS-50 (14-bit) just doesn't go far enough. I doubt that even
16-bit scanners can really do Kodachrome justice because - judging by
tests - one needs about 17 or 18 bits to really penetrate them.

So, I ended up scanning each slide twice with the idea of combining
the two. This works but I had to write another program to automate it
(without getting into too much detail it extract the film's
characteristic curve on a slide-by-slide basis). This can also be done
with various HDR (high dynamic range) programs out there.

You can also combine them manually (known as "contrast masking") but
if you don't adjust the colors first you can only combine images where
the border between highlights and shadows is well defined. In other
words, the border between the two will be visible if it falls in the
middle of a gradient (even if you use a fuzzy mask when merging them).

Another problem with scanning twice is the slight misalignment between
two consecutive scans. Even though Nikon's registration is very good,
there's still a slight (sub-pixel) misalignment and it varies across
the image as the scanner assembly travels down the length of a slide.
I'm very picky about this, but many people may not even see it as a
problem. Also, many HDR programs will automatically align such images.

The flatbed scanning was relatively painless, by comparison, but I did
run into Newton's rings. Those are the faint rainbow-colored irregular
circles and ellipses usually a problem when scanning slides in glass
mounts. But glossy photographs exhibit the same problem. The thing is
they stick to the glass causing the above rings as well as gray
"splotches". Usually, one needs to view the scan at 100% to see it.

I solved this by using a piece of Plexiglas (but any hard material
will do). Next, I would stick the photo to the Plexiglas to keep it
totally flat (use glue which is easily removable so as not to damage
the photo afterwards when peeling it off). I then cut out strips of
thin cardboard which I placed on the scanner glass around the photo.
The Plexiglas with the photo then goes on top of that, face down.
(Plexiglas is handy because you see the cardboard strips underneath.)

This sounds very complicated, but the idea is to use the cardboard
strips to lift the photo above the glass slightly (so it doesn't stick
and cause Newton's rings and "splotches") but not lift it too far from
the glass so it's out of focus and/or has low contrast. Conventional
scanners (i.e. virtually all scanners except the Canon Lide range
which use a different CIS technology) have some depth of field above
the glass so the image is still sharp.

I scanned the prints at 1200 dpi (48-bit) which is an overkill because
it's commonly accepted that regular prints don't really carry more
than 300 dpi of information, 600 at the most (e.g. contact prints). I
did this to be future-proof as I expect the displays will increase in
size so I don't have to extrapolate images later. (I don't print and
will be only using these images on a display of some sort.)

A couple of other flatbed tips. Compulsive cleaning of the glass is a
must! And this includes the underside (!) as the electronics tend to
evaporate and condensate on the underside showing up as a "fog".
Having tried a variety of "wet" cleaners - in my personal experience -
they all seem to leave some sort of residue. Sometime this shows up
several minutes after the cleaning! So, my advice is to get a good (!)
true (!) microfiber cloth and "dry clean" only!

BTW, not everything calling itself "microfiber" really is microfiber!
When used, a good microfiber cloth has a "rubbery" feel to it as it
"grips" the glass while removing all the junk. When you do find a good
one get two, so while one is the wash you can use the other.

The other tip is checking how clean the flatbed glass really is. To do
this, darken the room (or do this at night), lift the lid, and scan
nothing! This will result in a seemingly black image. But after
importing this into an image editor boost the brightness until the
image becomes gray. At this point every scratch, dust or smudge will
jump out at you! It's quite an eye opener! Do this immediately after
you cleaned the glass and, I bet, you'll shriek in horror! I did!! ;o)

Other than that, the horrific moments for me were when I would go
through, say, 15-20 rolls and (Kodachrome) scans keep getting darker
and bluer and the moment comes when I just throw my hands up in the
air and admit I just can't tolerate this. So I start again. Then I
notice that images are not in focus because old film (especially
cardboard mounted Kodachromes) tend to warp over time. Back to square
one, sampling multiple focus points and averaging. And so on... In the
end, I learned the images in those first few films by heart! :-)

As I often wrote here, I've done many things in my life but nothing
has been so frustrating and full of "gotchas" like scanning. It's like
those Russian dolls. You open one (i.e. solve one problem) only to
reveal another. Argh...! But, all that is water under the bridge now
because - not to rub it in, but - I'm done! ;o)

Seriously, not everyone may be as "picky" as I am, but I would advise
looking at the image at least at 100% magnification (300% is even
better) to catch all those potentially bothersome things early on.

The euphoric part is right now! It's quite true what they say that the
more you work for something the more you appreciate it in the end when
it's accomplished. Sort of the opposite of "easy come, easy go".
Also, thanks for relaying the information on TIF, slide scanning etc. to
help the rest of us along.

The pleasure is all mine! I learned so much here so it's only fair to
give some of it back.
Once again, congratulations!

Thanks! As I said, keep at it because there is light at the end of
tunnel! And - one day - it will not be an oncoming engine! ;o)

Seriously though, the best of luck!

Don.
 
Don said:
Oh, definitely! Not only because of the end result, but in the process
I also organized my images. Many weren't dated so I had to create a
timeline first. The date-stamped slides were a great help here, but
for negatives and photos it took a lot of head scratching, going
through old passports to figure out when a particular trip took place
or just looking at the images and comparing them to dated slides, etc.

In the end, I organized them all purely chronologically. Each film has
its own directory in the format: YYYY.MM. That way they all sort
nicely. If I had more than one film in a given month I would either
add the day (.DD) or, failing that, add a suffix (either "-1", "-2" or
"-A", "-B", etc to sequence them). Of course, in some cases I was only
able to narrow it down to the year only. Within those directories, the
images are then numbered according to their number on the film.
Seriously though, the best of luck!

Don.

Don,

again, thanks. I've started out with a CanoScan 8400F so I've probably got a
lot to learn over the coming months. I'll try and learn from some of your
advice. New 320GB HDD on the way to help with my storage situation (scans,
ongoing digital shots, art, music, movies etc.).

I'd written a lot of dates on at least the first slide/photo/wallet/negative
slip so I'm fortunate to have many of the dates to the YYYYMMDD level ... as
with you some with just get a month. Hey, this is all part of the fun ...
asking friends when they think a certain photo was taken - the variation in
answers is quite amusing and it's no wonder I can't tell myself.

I'm glad you addressed the glass cleaning - I scanned at least 2 rolls
before I noticed that the same negative in a certain location always had the
same piece of dust in the corner. I'm using a dust cloth right now but it's
not working too well - will try microfiber cloth as you suggest (or that in
combo with dust-off compressed air).

My work is mostly on negatives and I don't have anything close to what you
have - only ~800 24/36 exp rolls to scan ... graduating from amateur to pro
has been hard for me.

Again, thanks. Your lengthy and informative post is in my archives.

Kind regards,
PK
 
again, thanks. I've started out with a CanoScan 8400F so I've probably got a
lot to learn over the coming months. ....
I'm glad you addressed the glass cleaning - I scanned at least 2 rolls
before I noticed that the same negative in a certain location always had the
same piece of dust in the corner.

That's the sort of thing that drove me nuts and what caused all those
false starts. It's a steep learning curve but, as you say, that's part
of the fun!
Again, thanks. Your lengthy and informative post is in my archives.

My pleasure, PK. The best of luck!

Don.
 
Well, congratulations, and I'm sure you'll have fun revisiting some of
those images from decades past.

I just changed news servers and this old thread popped up that I had
missed. You are probably long gone Don, but congratulations. I have
to plead guilty to being a bit jealous as I started long ago and still
have a long way to go. I've passed 30,000 slides and negatives and
probably have another 5,000 to go as a wild guess. Unfortunately that
does not include the 100# plus of antique prints and numerous albums
of B&W yet to go.

The first year I stuck with it diligently, but I do have other things
that must be done as well so by the time I finish I'll probably be a
very old man.


Again, good luck and thanks for all the pointers.


Roger Halstead (K8RI & ARRL life member)
(N833R, S# CD-2 Worlds oldest Debonair)
www.rogerhalstead.com
 
Back
Top