how come lots of small files transfer slower than 1 big one?

[James]

how come my transfers are so slow when copying files from my xp machine to
my win2k machine on my 100mbps switched lan?
large file = 100MB .mdb
small files = total 7MB's of 400 text files

it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer the
large file.

 

[Bob Willard]

how come my transfers are so slow when copying files from my xp machine to
my win2k machine on my 100mbps switched lan?
large file = 100MB .mdb
small files = total 7MB's of 400 text files

it takes 12 mins to transfer the 7 MB's and only 30 seconds to transfer the
large file.

And how come digging a ditch with one backhoe is faster than using
ten men with shovels? Or using a thousand men with teaspoons?

Files are transferred one at a time. For each file, the source PC must
look up the file's metadata (to translate the file name to the file
location, check security params, etc.). Worse still, is what the target
PC must do for each file: it must find some unused space on the HD, then
allocate some of that space, then create the metadata for the new file,
then ask the source PC for the data, then copy the data to the allocated
space, then update the metadata (when all data has been copied), then
notify the source PC that the copy is done.

It takes a lot of work to copy a file across a net, whether the file is
tiny or huge. For a huge file, copying data take most of the time; for a
tiny file, copying data takes far less time than all of the other overhead
work.

 

[James]

but lots of files tranfer fast when i use another workstation to the same
server.

 

[Bob Willard]

but lots of files tranfer fast when i use another workstation to the same
server.

to

the


And how come digging a ditch with one backhoe is faster than using
ten men with shovels? Or using a thousand men with teaspoons?

Files are transferred one at a time. For each file, the source PC must
look up the file's metadata (to translate the file name to the file
location, check security params, etc.). Worse still, is what the target
PC must do for each file: it must find some unused space on the HD, then
allocate some of that space, then create the metadata for the new file,
then ask the source PC for the data, then copy the data to the allocated
space, then update the metadata (when all data has been copied), then
notify the source PC that the copy is done.

It takes a lot of work to copy a file across a net, whether the file is
tiny or huge. For a huge file, copying data take most of the time; for a
tiny file, copying data takes far less time than all of the other overhead
work.

From your question, I assumed that you were interested in why bunches
of files take longer than a single file to transfer; that is the issue
I addressed. If you want to understand why one PC is faster than another,
then you should conduct some measurements in which only one variable at
a time is changed -- and "the workstation" is not one variable, it is
probably many.

 

[James]

what things could be differn't. both are clean installs of xp sp1

 

[Bob Willard]

what things could be differn't. both are clean installs of xp sp1

James wrote:


same

a

overhead

From your question, I assumed that you were interested in why bunches
of files take longer than a single file to transfer; that is the issue
I addressed. If you want to understand why one PC is faster than another,
then you should conduct some measurements in which only one variable at
a time is changed -- and "the workstation" is not one variable, it is
probably many.

NIC speed, RWIN, protocols used, DNS settings, HD speed, CPU frequency,
IDE speed, RAM speed and quantity, pagefile min/max params and HD(s),
XP PRO v. HE, NTFS v. FAT32, user credentials, other traffic on the LAN,
HD RPM, HD fragmentation, CPU type, cache params, MB type/revision, BIOS
type, version, and settings, NIC vendor/model, NIC card v. chip, concurrent
apps running, domain v. workgroup, etc.

And, maybe, lots of other stuff. It takes care and effort to compare two
things that are *exactly* alike except for one variable.

 

[James]

both computers are pretty new i think they should be able to support
transfering files just fine with their hardware. im guessing either a NIC
settting is the case here or a bad NIC. this NIC im using is the onboard one
that came with this msi 865PE neo-2 ls motherboard. any nic hardware
settings u can think of that would cause this?

 

[Bob Willard]

both computers are pretty new i think they should be able to support
transfering files just fine with their hardware. im guessing either a NIC
settting is the case here or a bad NIC. this NIC im using is the onboard one
that came with this msi 865PE neo-2 ls motherboard. any nic hardware
settings u can think of that would cause this?

machine

transfer

must

target

then

file,

allocated

is

for

another,

NIC speed, RWIN, protocols used, DNS settings, HD speed, CPU frequency,
IDE speed, RAM speed and quantity, pagefile min/max params and HD(s),
XP PRO v. HE, NTFS v. FAT32, user credentials, other traffic on the LAN,
HD RPM, HD fragmentation, CPU type, cache params, MB type/revision, BIOS
type, version, and settings, NIC vendor/model, NIC card v. chip,
concurrent

apps running, domain v. workgroup, etc.

And, maybe, lots of other stuff. It takes care and effort to compare two
things that are *exactly* alike except for one variable.

10 Mb/s, either explicitly set or the result of a bad auto-detect.
Or, HDX instead of FDX, explicit or bad auto-detect.

And, I would look at the DNS settings and protocols used for diff's
between those two PCs. Also, make sure you have the same credentials
and the same filesystem (preferably NTFS) with the same file sharing
(simple or ACLs) on both.