Charlie said:
Thanks for the comments and suggestions. I'll certainly try the varius
suggestions and see what looks best.
A bit more explanation.... This is intended to replace a CD that I've
used in past years. The CD is genealogy data, provided for the past 10
years at our family reunion.... and the data has grown too large for a
CD. Also, one of the "features" I intend to put on the flash drive is
essentially a web page with thousands of small html files. I'm sure
that's what is slowing down the process. In the past, I tried that
feature on the CD, but it ran so slow from the CD that I doubt many
people would use it. Another factor is the computer savy level of most
of the users.... it was a challenge to get the CD set up so that all
that was required of the user was to insert it. Zipping the files
might work if I can get it to extract automatically to the flash
drive.... I doubt I could get the users to extract to their computer
and then locate the files to run them. The featured program does run
very well from the flash drive.... just about as fast as from my hard
drive. If anyone is curious about the application, I have a slightly
different one on a web page at
http://my.rootsmagic.com/CRHoffpauir/ .
The one on the flash drive is larger....
Another consideration... These CDs have been sold by our Family
Organization (a registered non-profit organization) for a nominal
price... that's why I haven't put the data on the web and provided a
link to it..... We could decide to simply forget the sale and give the
data away on the web instead..... it might make it easier for everyone
that way.
If you burned a DVD with the information, that would
give you 4.7GB of room. Instead of the 700MB of the CD.
Burning optical media, using ISO9660 as an intermediary
form, should mean no content dependencies - if there
were a lot of small files, you don't pay a "transfer rate"
price for them.
ZIPping the files, in "Store" mode without compression,
is a quick way to test the "real" transfer rate of
the USB stick. That's just to show the transfer
rate could go faster.
Using "dd" to clone a USB flash drive, is an easy
way to avoid paying a "small file, transfer rate"
price for the files. You could prepare one USB stick
(using as many hours as it takes) and make that USB
stick your "master". Others can be re-generated from
the USB stick with a command like this.
dd if=\\?\Device\Harddisk3\Partition0 of=\\?\Device\Harddisk4\Partition0
With the master device Harddisk3 in the computer, you
would first verify with "dd --list" that you have
the correct ID for the thing. The labeling of physical layer
storage in "dd", should be the same order as seen in
Disk Management. Once you've verified that Harddisk3
is the source device, you could plug in empty
Harddisk4, Harddisk5, ... and from separate command
prompt windows, do parallel copies. In the
same form as the example command. In four separate
Command Prompt windows, with four devices, I could do
this (one of these commands per Command Prompt window).
dd if=\\?\Device\Harddisk3\Partition0 of=\\?\Device\Harddisk4\Partition0
dd if=\\?\Device\Harddisk3\Partition0 of=\\?\Device\Harddisk5\Partition0
dd if=\\?\Device\Harddisk3\Partition0 of=\\?\Device\Harddisk6\Partition0
dd if=\\?\Device\Harddisk3\Partition0 of=\\?\Device\Harddisk7\Partition0
Given the transfer rate of USB2 and the slow write rate
of commodity USB2 sticks, that would just about use up
the capacity of one USB2 logic block. Some motherboards
(like mine) have two USB2 logic blocks, so about 60MB/sec
total bandwidth for this kind of work.
Due to a bug in "dd" port, where it doesn't reliably
detect the end of a USB flash stick, you can use
size parameters. For example "dd --list" gives the
actual physical layer storage size in bytes.
The product of the block_size and the count value,
should equal the total device size. That's if
cloning one entire USB flash to another entire USB
flash (same sized USB keys). You glue this onto
the end of the commands above, to help limit
the transfer command to an exact total byte count.
This example assumes the USB flash is 1,048,576,000 bytes.
bs=1048576 count=1000
If the device was older, perhaps you could use
quarter megabyte transfers, using these optional
parameters. The idea is, to attempt to deliver
a large enough block_size so that it matches
the internal preferences of the USB flash chip.
It's a lot better to use a larger block, than
the default of 512 bytes, which would be hard
on the USB flash stick.
bs=262144 count=4000
While there are probably further tweaks possible
with the command, I would just transfer the whole
(master) stick, to multiple slaves, all at the
same time.
I can see that storing a ZIP archive on the USB stick,
might only make guys like me happy. A person who
just wants to see genealogy data, doesn't want to
have to do any post processing. I mean, you could
engineer things, such that the USB stick "unpacked"
itself on the recipient's hard drive, but again,
not everyone will appreciate your largess. Some
people will be security conscious, and want to
"scan" the stick before touching it. Others won't
have a clue, and they'll just want to stick to
do everything for them. No matter what you do,
I'm sure somebody will complain about what they
got with the provided media.
Providing the USB stick, means the recipient is
being given read/write media. Whereas the DVD
or CD is considered read-only (without going
multisession or something). A recipient could
damage their stick, by fooling around with the
content (toss in desktop Trash Can etc). While
there are USB flash sticks with a write protect
switch on them, I think that concept went the
way of the dodo bird, and you likely could
not find one if you needed it. If there was a
switch, you'd flip the switch to read-only,
before placing in the Fedex package.
If you place a USB key in letter-mail, without a box
to protect it, it could get crushed. I used
to work in a Post Office, so I have some
experience crushing stuff for a living
No matter what media you use, it should be
protected.
While I haven't had any recent reports,
some shipping paths in the US, use radiation
based scanners on the items being shipped. There
were a couple reports of motherboards arriving
with no BIOS contents in the flash chip. While
I assume they don't use quite the dosage implied
by that any more, it might still be possible for a
USB key to get erased in transit. If a recipient
complains the computer "wants to format the stick",
that's just one possible outcome for the USB stick.
Most AV scanners will be giving that stick the
once-over, when it is inserted, but that should
not damage anything. And some OSes may just
decide to be finicky eaters, and barf when
given the USB stick. So expect some "tech support"
issues with this operation (charge $5 for stick,
provide $100 worth of over-the-phone tech
support - you know the type of people I'm
thinking of).
Paul