USB hard disk speed?

  • Thread starter Thread starter Russell May
  • Start date Start date
R

Russell May

When I recently copied about 4.6GB from my AcomData 7200RPM USB 2.0
60GB external disk drive to a Verbatim 16X DVD+R disk via my Plextor
PX-716A 16X internal drive, there were several pauses to replenish the
cache after it reached about 8X write speed. The copy operation
averaged about 400MB/minute. During verification, it paused again
several times after it reached about 8X read speed. I expected
something faster both directions. Copying the same data between from
an internal hard drive and the DVD drive results in no such pauses or
slowdowns, so I presume the USB 2.0 interface slows the operation.

Is 400MB/minute normal for copying between a USB 2.0 hard drive and
internal 16X DVD drive? Is there something simple to improve it?

Russell May
 
Russell May said:
When I recently copied about 4.6GB from my AcomData 7200RPM USB 2.0
60GB external disk drive to a Verbatim 16X DVD+R disk via my Plextor
PX-716A 16X internal drive, there were several pauses to replenish the
cache after it reached about 8X write speed. The copy operation
averaged about 400MB/minute. During verification, it paused again
several times after it reached about 8X read speed. I expected
something faster both directions. Copying the same data between from
an internal hard drive and the DVD drive results in no such pauses or
slowdowns, so I presume the USB 2.0 interface slows the operation.

Is 400MB/minute normal for copying between a USB 2.0 hard drive and
internal 16X DVD drive? Is there something simple to improve it?

Russell May
Click to expand...

Did you try to copy same data between internal HD and USB 2.0 HD?
 
Did you try to copy same data between internal HD and USB 2.0 HD?
Click to expand...

Reading and writing the USB 2.0 HD subjectively seems a little slow:
Sometimes there are a few pauses while playing video on the USB drive.
Copying between internal HD and USB HD is significantly slower than
copying between internal hard disks, but I have not timed it. I have
just lived with it being a little slow. The USB performance became
more important when I got a DVD drive and larger internal HD.

I did a little research and found this:
http://www.tomshardware.com/storage/20030116/

The review is a couple of years older than my USB drive but I think
the enclosure is from the same manufacturer. The review shows even
lower write performance than what I got while copying to DVD, but
significantly higher read performance.

My external drive also has a Firewire connection which might give
better performance. I have not tried it, but maybe I should do so.
 
Russell May said:
Reading and writing the USB 2.0 HD subjectively seems a little slow:
Sometimes there are a few pauses while playing video on the USB drive.
Copying between internal HD and USB HD is significantly slower than
copying between internal hard disks, but I have not timed it. I have
just lived with it being a little slow. The USB performance became
more important when I got a DVD drive and larger internal HD.
Click to expand...

Russ - I find very little difference in copying from one internal harddrive
to another vs copying from an internal harddrive to a harddrive in a USB 2
enclosure.

Just to confirm, did you measure the copy from the internal harddrive to the
DVD?

Also, how are you 'copying' to a DVD drive? Using packet writing with an
RW?
 
Russ - I find very little difference in copying from one internal harddrive
to another vs copying from an internal harddrive to a harddrive in a USB 2
enclosure.

Just to confirm, did you measure the copy from the internal harddrive to the
DVD?

Also, how are you 'copying' to a DVD drive? Using packet writing with an
RW?
Click to expand...

I timed several operations. I used only Verbatim 16X DVD+R disks.

Data were written from HD to DVD via "Nero 6 Ulta Edition" Burning
ROM. Data were written from HD or DVD to HD using copy and paste in
"My Computer". Data written from internal HD to internal HD were
copied and pasted between directories on the same partition. Data
total 4460MB (4,704,600,604 bytes) from 31616 files and 1604 folders.

Environment: OS=WinXP Home SP2, CPU=P4 2.4GHz, Mbd=Gigabyte 8IEXP,
memory=1.0GB ECC DDR, internal HD=Samsung SP2014N (200GB NTFS 7200RPM
ATA100, indexing enabled), external HD=Samsung SP6003H (60GB NTFS
7200RPM ATA100 in Acom Data USB/FW enclosure, indexing disabled),
DVD drive=Plextor PX-716A (16X ATA66), media=Verbatim 16X DVD+R,
antivirus=CA eTrust, antispam=Microsoft, #tasks=about 48,
applications=none except My Computer or Nero as applicable.

Copy times and rates:
From DVD to internal HD: 26min 10sec (about 174MB/minute)
From internal HD to internal HD: 9min 48sec (about 455MB/minute)
From internal HD to USB 2.0 HD: 10min 5sec (about 442MB/minute)
From USB 2.0 HD to internal HD: 10min 14sec (about 435MB/minute)
From internal HD to DVD: write 7min 55sec (about 565MB/minute)
verify 7min 17sec (about 612MB/minute)
From USB 2.0 HD to DVD: write 10min 5sec (about 442MB/minute)
verify 7min 20sec (about 608MB/minute)

I did not have enough time to measure from DVD to USB 2.0 HD.

Disabling Microsoft AntiSpam had no significant effect.

Snoozing (disabling) CA eTrust Antivirus had a significant effect.
From internal HD to internal HD: 7min 1sec (about 637MB/minute)
about 40% faster than with AV.

Writing to DVD was irregular:
From internal HD to DVD: Write operation paused four times to
replenish the read buffer at about 2'58", 3'10", 3'35", 7'15". Verify
operation started at about 4.9X, paused starting at about 25% progress
and 6.8X, varied irregularly from 960KB/sec to 8X until about 50%
progress, became regular again, finished at 11.8X.
From USB 2.0 HD to DVD: Write operation paused fifteen times to
replenish the buffer starting at about 2'10". It paused about every 30
seconds. Verify operation started at about 4.9X, went to 7.3X at about
25% progress, varied irregularly from there to about 32% progress,
became regular to about 56%, varied again to about 68%, went to 10.7X
there and became regular until about 97% progress, finished at 11.3X.

Plextools Professional Write Transfer rate test results were irregular
DVD disk #1: Write Transfer rate started at 6.76X, went to about 14X
at 3.1GB, dropped to steady 12.23X there, dropped to
steady 4.07X at about 3.6GB, finished at 4.07X.
DVD disk #1: Simulated Write started at 6.76X, dropped to steady
12.23X at about 3.3GB, finished at 12.23X.
DVD disk #2: Write Transfer rate started at 6.76X, went to about 14.5X
at 3.6GB, dropped to steady 12.23X there, finished at
12.23X.
DVD disk #2: Simulated Write started at 6.76X, finished at 16.29X.

Plextools Professional Read Transfer rate test results were regular
for both DVDs: Read transfer rate started at 4.93X, ended at 11.81X.

Writing an image of the internal HD (about 27GB) to the USB 2.0 HD
using Ghost 2003 averages about 410MB/minute, fills up the USB HD.
 
Copy times and rates:
From internal HD to internal HD: 9min 48sec (about 455MB/minute)
From internal HD to USB 2.0 HD: 10min 5sec (about 442MB/minute)
From USB 2.0 HD to internal HD: 10min 14sec (about 435MB/minute)
From internal HD to DVD: write 7min 55sec (about 565MB/minute)
verify 7min 17sec (about 612MB/minute)
From USB 2.0 HD to DVD: write 10min 5sec (about 442MB/minute)
verify 7min 20sec (about 608MB/minute)

I did not have enough time to measure from DVD to USB 2.0 HD.
Click to expand...
Is 400MB/minute normal for copying between a USB 2.0 hard drive and
internal 16X DVD drive? Is there something simple to improve it?
Click to expand...
Data total 4460MB (4,704,600,604 bytes) from 31616 files and 1604 folders.
Click to expand...

There is a pattern. Whenever USB 2.0 HD is involved in data copy, best speed
drops 8 min range to 10 min range. I'm assuming that USB 2.0 HD is slower
than internal HD, more fragmented or USB 2.0 has higher overhead for
transferring that many (30K+) files.

When you say "from internal HD to internal HD" do you mean the same HD or
two different internal HDs?
 
Peter said:
There is a pattern. Whenever USB 2.0 HD is involved in data copy, best speed
drops 8 min range to 10 min range.
Click to expand...
I'm assuming that USB 2.0 HD is slower than internal HD,
Click to expand...

Now isn't that is a big surprise when USB-2 is limited to ~35MB/s.
more fragmented
Click to expand...

Possibly, since the 435/60=~7MB/s is well below the 35MB/s USB limit.
or USB 2.0 has higher overhead for transferring that many (30K+) files.
Click to expand...

That should be a bug in the OS then.
 
There is a pattern. Whenever USB 2.0 HD is involved in data copy, best speed
drops 8 min range to 10 min range. I'm assuming that USB 2.0 HD is slower
than internal HD, more fragmented or USB 2.0 has higher overhead for
transferring that many (30K+) files.

When you say "from internal HD to internal HD" do you mean the same HD or
two different internal HDs?
Click to expand...

Same HD. "Data written from internal HD to internal HD were
copied and pasted between directories on the same partition."
 
There is a pattern. Whenever USB 2.0 HD is involved in data copy, best speed
drops 8 min range to 10 min range. I'm assuming that USB 2.0 HD is slower
than internal HD, more fragmented or USB 2.0 has higher overhead for
transferring that many (30K+) files.
Click to expand...

Copying between internal HD and USB drive was only about 20 seconds
slower (about 3%) than copying from internal HD to internal HD, not
two minutes. The two minute difference between the HDs' times was for
writing to the DVD drive. See above.

Both HDs were defragmented prior to the tests.

The SP6003H is slower than the SP2014N, but both should be capable of
sustained transfer rates well over 657MB/minute (11MB/second).

USB overhead might be significant if the USB enclosure operates like
to the one reviewed by tomshardware, mentioned in my previous-previous
message. The review is about the same age as my USB drive, not two
years older as I said in that message.

OS or seek time overhead seems much more than I expected.

Antivirus had more effect than I expected. Copying without AV
"From internal HD to internal HD: 7min 1sec (about 637MB/minute)
about 40% faster than with AV."

I tried the internal HD to to internal HD test on a different computer
(Dell GX260, OS=Win2K SP4, CPU=P4 2.4GHz, HD=IBM 30GB 75GXP NTFS) and
data (3,278,976,512 bytes; 47552 files; 2119 folders). Disabling
Symantec Antivirus File System Auto-Protect sped it up by about 25%
(16min 58sec versus 21min 8sec).

Copying from the DVD seems slow
"From DVD to internal HD: 26min 10sec (about 174MB/minute)".

Writing to the DVD was irregular and sometimes slow (results deleted
above). I don't have easy access to another computer with a DVD drive.

These tests make me begin to wonder whether there is a problem with
the DVD drive. It's about six weeks old, with the latest firmware.

The media could be bad, but Plextor recommends these Verbatim 16X
DVD+R disks and cdrlabs tested them to be among the best for this
drive.
When you say "from internal HD to internal HD" do you mean the same HD or
two different internal HDs?
Click to expand...

Same HD. "Data written from internal HD to internal HD were
copied and pasted between directories on the same partition."
 
Now isn't that is a big surprise when USB-2 is limited to ~35MB/s.


Possibly, since the 435/60=~7MB/s is well below the 35MB/s USB limit.


That should be a bug in the OS then.
Click to expand...

Both drives were defragmented before the tests.

I knew that USB 2.0 is limited to less than its maximum data rate of
60MB/sec, but I did not how much less. Thanks for the information.
 
Russell May said:
Both drives were defragmented before the tests.

I knew that USB 2.0 is limited to less than its maximum data rate of
Click to expand...
60MB/sec,
Click to expand...

What 60MB/s , it's 480Mbit/sec.

480MBit/s is 480Mbit/s . Not 60MB/s and not 48MB/s either.
It's the clock rate converted to bits. There are 10 bits for each byte
and then there is other overhead for serial protocol and then there is
the usual overhead for commands. Apparently there is about 1/3 overhead
on the converted to bytes number (serial protocol + command overhead).
 
Copy times and rates:
Copying between internal HD and USB drive was only about 20 seconds
slower (about 3%) than copying from internal HD to internal HD, not
two minutes. The two minute difference between the HDs' times was for
writing to the DVD drive. See above.

Both HDs were defragmented prior to the tests.

The SP6003H is slower than the SP2014N, but both should be capable of
sustained transfer rates well over 657MB/minute (11MB/second).
Click to expand...

The problems is that tranferring of 4GB of data in more than 30,000 files
rarely runs at STR speed.
If you read and write on the same HD, it would be much closer to random IO.
USB overhead might be significant if the USB enclosure operates like
to the one reviewed by tomshardware, mentioned in my previous-previous
message. The review is about the same age as my USB drive, not two
years older as I said in that message.

OS or seek time overhead seems much more than I expected.
??


Antivirus had more effect than I expected. Copying without AV
"From internal HD to internal HD: 7min 1sec (about 637MB/minute)
about 40% faster than with AV."
Click to expand...

Yeah. That's how AV works. A lot of overhead on HD writes.
I tried the internal HD to to internal HD test on a different computer
(Dell GX260, OS=Win2K SP4, CPU=P4 2.4GHz, HD=IBM 30GB 75GXP NTFS) and
data (3,278,976,512 bytes; 47552 files; 2119 folders). Disabling
Symantec Antivirus File System Auto-Protect sped it up by about 25%
(16min 58sec versus 21min 8sec).

Copying from the DVD seems slow
"From DVD to internal HD: 26min 10sec (about 174MB/minute)".
Click to expand...

Again, that is rather random IO, not sequential. DVD performance might be
just fine.
Writing to the DVD was irregular and sometimes slow (results deleted
above). I don't have easy access to another computer with a DVD drive.
Click to expand...

Now, writing 30,000 files to DVD might be performed in quasi-sequential
mode, since data has to be burned to disk. That's why writing is faster than
reading.
These tests make me begin to wonder whether there is a problem with
the DVD drive. It's about six weeks old, with the latest firmware.

The media could be bad, but Plextor recommends these Verbatim 16X
DVD+R disks and cdrlabs tested them to be among the best for this
drive.


Same HD. "Data written from internal HD to internal HD were
copied and pasted between directories on the same partition."
Click to expand...

You should perform random IO tests for each device: internal HD, USB 2.0 HD
and DVD drive.
Pick data set size around 4GB with 100KB files.
 
Folkert Rienstra said:
What 60MB/s , it's 480Mbit/sec.

480MBit/s is 480Mbit/s . Not 60MB/s and not 48MB/s either.
It's the clock rate converted to bits. There are 10 bits for each byte
and then there is other overhead for serial protocol and then there is
the usual overhead for commands. Apparently there is about 1/3 overhead
on the converted to bytes number (serial protocol + command overhead).
Click to expand...
USB 1.1 uses NRZI with bit stuffing. To encode a byte takes about 8.1 bits.
Data packets are 1-1024 bytes, with several bytes overhead.

Nobody seems to know why full speed (12Mb/s) has a maxium of 1MB/s,
instead of the expected 1.5MB/s. USB 2 seems to have 60% of 60MB/s.
 
60MBytes/s cretin.



Wrong.
Click to expand...

You guys got me interested, so I looked it up.
See http://www.usb.org/developers/docs/usb_20_02212005.zip
usb_20.pdf sections 5.4.1, 5.6.3, 5.8.3, 7.1.9, 11.3 for examples.

Data is listed as 8-bit bytes. Maximum bandwidth is listed as 60MB/sec
for high-speed isochronous transfers. Bit stuffing (insertion of a
zero after a sequence of six ones) is data dependent and multiplies
packet time by up to 7/6, adding an average of up to 4/3 bit time per
8-bit byte.
 
The problems is that tranferring of 4GB of data in more than 30,000 files
rarely runs at STR speed.
If you read and write on the same HD, it would be much closer to random IO.


Yeah. That's how AV works. A lot of overhead on HD writes.


Again, that is rather random IO, not sequential. DVD performance might be
just fine.


Now, writing 30,000 files to DVD might be performed in quasi-sequential
mode, since data has to be burned to disk. That's why writing is faster than
reading.


You should perform random IO tests for each device: internal HD, USB 2.0 HD
and DVD drive.
Pick data set size around 4GB with 100KB files.
Click to expand...

Could you suggest test program(s) to accomplish this?

I did some more tests. To make a long story short, disabling AV
increased write rate by about 25% to 40% in all "My Computer" copy and
paste operations. Nero write rate from USB 2.0 HD to DVD decreased by
by 9% (!) and verify rate increased by only 5%, but there was only one
brief pause and irregularity to replenish the read buffer instead of
fifteen. It went much smoother than with AV enabled.
 
Russell May said:
You guys got me interested, so I looked it up.
Click to expand...

As I *should* have done. I was so confident that I didn't. Sorry for that.
The datarates of FireWire and USB are often compared as 400Mbit/s
vs 480Mbit/s with an 8MB/s head start on USB so I didn't think they
were different beyond that, as FireWire as well as Serial-ATA both use
8b/10b encoding, so there I went, assuming USB would too. It doesn't.

Given that the raw difference after parallel conversion is even bigger,
40MB/s vs 60MB/s, it's even more amazing that FireWire400 often still
beats USB 2.0.

OK. You may serve up the crow now.
 
USB 1.1 uses NRZI with bit stuffing.
Click to expand...

Judging from an USB 2.0 application document
http://www.intel.com/technology/usb/download/2_0_Xcvr_Macrocell_1_05.pdf
so does USB 2.0
To encode a byte takes about 8.1 bits.
Data packets are 1-1024 bytes, with several bytes overhead.

Nobody seems to know why full speed (12Mb/s) has a maxium of 1MB/s,
Click to expand...

Maybe because that's the real user data datarate?
That is about the 60% that you mention below (and my 2/3).

The bwpaper2.pdf that I found says: 'USB is intended
for devices in the 8Mb/s and below range transfers".
http://www.usb.org/developers/whitepapers/bwpaper2.pdf
instead of the expected 1.5MB/s."
Click to expand...

Well, that is the raw parallel bus rate.
USB 2 seems to have 60% of 60MB/s.
Click to expand...

34MB/s, according to another document.
http://www.usb.org/channel/USB-PerformData-04.pdf
 
As I *should* have done. I was so confident that I didn't. Sorry for that.
The datarates of FireWire and USB are often compared as 400Mbit/s
vs 480Mbit/s with an 8MB/s head start on USB so I didn't think they
were different beyond that, as FireWire as well as Serial-ATA both use
8b/10b encoding, so there I went, assuming USB would too. It doesn't.

Given that the raw difference after parallel conversion is even bigger,
40MB/s vs 60MB/s, it's even more amazing that FireWire400 often still
beats USB 2.0.

OK. You may serve up the crow now.
Click to expand...

There was no harm done. It didn't take long to find the specification,
and I learned a little in the process.

Fortunately the USB 2.0 specification is a free download (which is
unusual) so I could check it easily. Many industry standards are
costly and thereby much less accessible. IEEE 1394-1995 Firewire spec
costs USD$161 from http://webstore.ansi.org/ansidocstore, supplements
add more cost. I did not see titles at http://www.1394TA.org which
obviously would give protocol details, and I don't wish to become a
member. Linda Hall library (website http://www.lhl.lib.mo.us) has
information about it but I am not sure whether even they have the
specification (which is also unusual) and their hours are sometimes
inconvenient to me, like being closed all of this Labor Day weekend.

Most of the sites about Firewire I find with Google are about hardware
or drivers. Could you (or anyone) recommend sites which would give
some details about Firewire protocol?
 
Back
Top