Very slow SATA and eSata peformance

[Smarty]

I have 3 PCs which each have a variety of internal SATA drives, and all of
which have external eSATA ports. The external eSATA ports were either
factory-provided (Dell) or added by me with controller cards from Addonics
and Rosewill with PCI-Express or PCI interfaces.

Regardless of which computer and which internal or external drive I use (and
there are 14 drives in total I have tried), the nominal transfer rates I am
getting are roughly 25 to 30 Mbytes/sec. I will occasionally see a brief
transfer rate of 70-90 Mbytes/second, but this rate only lasts for a very
short time, I assume until some buffer in RAM has either emptied or filled.

The machines are all 32 bit Windows, both Vista and XP, in all cases with
latest patches and service packs.

I assume that SATA speeds closer to the theoretical limit should be
achievable, but have never seen even remotely close to this type of
performance. In fact, this type of performance I experience is much more
similar to USB2 and Firewire 400.

Is there anything I can do to speed things up? Files I read and write are
nominally long sequential transfers of roughly 6 GByte size, in all cases to
and from NTFS volumes / partitions.

Thanks in advance for any advice.

 

[Rod Speed]

Smarty wrote

I have 3 PCs which each have a variety of internal SATA drives, and
all of which have external eSATA ports. The external eSATA ports were
either factory-provided (Dell) or added by me with controller cards
from Addonics and Rosewill with PCI-Express or PCI interfaces.

Regardless of which computer and which internal or external drive I use (and there are 14 drives in total I have
tried), the nominal
transfer rates I am getting are roughly 25 to 30 Mbytes/sec. I will
occasionally see a brief transfer rate of 70-90 Mbytes/second, but
this rate only lasts for a very short time, I assume until some
buffer in RAM has either emptied or filled.

The machines are all 32 bit Windows, both Vista and XP, in all cases with latest patches and service packs.

I assume that SATA speeds closer to the theoretical limit should be achievable,

Not if by that you mean the 3000Mb interface speed.

but have never seen even remotely close to this type of performance.

No one has, it isnt even possible.

In fact, this type of performance I experience is much more similar to USB2 and Firewire 400.

Thats because the real speed achievable is determined by the rate that
the sectors move under the heads on the hard drive, not the interface speed.

Is there anything I can do to speed things up?
Nope.

Files I read and write are nominally long sequential transfers of roughly 6 GByte size, in all cases to and from NTFS
volumes / partitions.

Thanks in advance for any advice.

Even advice to shove you head up a dead bear's arse ?

 

[Smarty]

Smarty wrote


Even advice to shove you head up a dead bear's arse ?

Pathetic...........


Since these drives including the fastest one I own, a WD Velociraptor, can
sustain read/write transfers at several times the rate I am experiencing
when configured with the right RAID controller, I am reasonably certain that
the drive is inherently capable of much faster performance than I am
experiencing with my present SATA controllers.

The "simple" explanation would be that (some) SATA controllers underperform
the SATA specs by a considerable degree, and the 5 different controller
chipsets I happen to be using all suffer from this shortfall. Kinda' hard to
believe......

Again, thanks in advance for any advice...except for advice which is
sarcastic, mean-spirited, or generally lacking in technical integrity
(Rod).....

 

[Rod Speed]

Smarty wrote

Pathetic...........

That's what the bear said........................

Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the rate I am experiencing when configured with the
right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my present SATA controllers.

What are you measuring the thruput with ? Thats
the other obvious possibility for the discrepancy.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree,
Unlikely.

and the 5 different controller chipsets I happen to be using all suffer from this shortfall. Kinda' hard to
believe......

Yep, so maybe the problem is with the measurement.

 

[Paul]

I have 3 PCs which each have a variety of internal SATA drives, and all
of which have external eSATA ports. The external eSATA ports were either
factory-provided (Dell) or added by me with controller cards from
Addonics and Rosewill with PCI-Express or PCI interfaces.

Regardless of which computer and which internal or external drive I use
(and there are 14 drives in total I have tried), the nominal transfer
rates I am getting are roughly 25 to 30 Mbytes/sec. I will occasionally
see a brief transfer rate of 70-90 Mbytes/second, but this rate only
lasts for a very short time, I assume until some buffer in RAM has
either emptied or filled.

The machines are all 32 bit Windows, both Vista and XP, in all cases
with latest patches and service packs.

I assume that SATA speeds closer to the theoretical limit should be
achievable, but have never seen even remotely close to this type of
performance. In fact, this type of performance I experience is much more
similar to USB2 and Firewire 400.

Is there anything I can do to speed things up? Files I read and write
are nominally long sequential transfers of roughly 6 GByte size, in all
cases to and from NTFS volumes / partitions.

Thanks in advance for any advice.

Run HDTune from hdtune.com . Version 2.55 is free for download. A more
full featured version is available for purchase.

A SATA drive should be limited by the sustained performance at the
platter. The fastest SATA is about 120MB/sec or so. The benchmark
results should be a curve, as the disk data transfer rate varies
with position of heads on the platter. If the benchmark results are
a flat line, either you have a flash SSD, or some cabling or
interface is stuck in an incorrect mode (like PIO). That 120MB/sec
sustained drive, may deliver about 60MB/sec near the end of the
benchmark graph. (My current drive might be about 70MB/sec at the
beginning and maybe 35MB/sec near the end.)

HDTune also reports some interface information, on its info page.

The "burst transfer rate", may represent how fast data is transferred to
the cache chip on the drive controller board. Since transfers to cache,
avoid the heads and platters, you get a better measure of other aspects
of the interconnect. The PCI Express x1 may have a theoretical limit
of 250MB/sec, but because packets are used to transfer data, there is
some overhead. The burst transfer test, may help identify whether
something is out of the ordinary, with respect to bus interconnect
or interface operating mode (PIO/DMA etc). Burst transfer can help
identify a bottleneck.

When you see a flat line, at about 4MB/sec, that usually means the
interface is in PIO (polled) transfer mode.

The benchmarks HDTune and HDTach are not perfect. They don't
agree with one another, for one thing. And don't seem to give
correct results with SSD drives. So they should not be quoted
as being absolute and undeniable results. I have a suspicion
the code and test conditions have to be adjusted as a function
of the technology being tested. So maybe some future version
will do a better job with SSD drives.

Paul

 

[Smarty]

There are drives with faster spin speeds, and with bigger caches.
(The 10000 and 15000 rpm spindle motor ones, but then you
are looking at SCSI and a SCSI card which rules out external
- havn't checked if any of the 10000 ones are SATA.)
Add on cards may not deliver .the speeds that you get from
a motherboard SATA port, converted to eSATA if necessary,
(it's just a different plug/jack).
Upping your pagefile, possibly putting it on a differet drive
(if you have more than one), and making the pagefile a
fixed size might help.
Some say RAID striping helps but that may not always
be the case.

Thanks Ato-Zee and Rod, and I will try playing with the pagefile and report
back with my results. I run programs like HD Tune which show the average
transfer rates to the drives being in the range of 40 to 70 MBytes/sec
depending on the drive, controller, fragmentation of the drive, etc. (The
maximum transfer rates can get up to 85 or 90 Mbytes/sec, the minimum
transfer rates as low as 10-15 Mbytes/sec.) My most important 'measurement'
is the actual file transfer times I encounter from drive to drive, typically
5-6 minutes to transfer a 6 GByte file. If the drive I/O is overlapped and
using 2 different controllers, and the bus is far below its theoretical
limit, I would think that I should be able to move a 6 GBytes file in a
little over 90 seconds. In this specific case, I would be reading one drive
at 66 Mbytes per second and writing the other at the same speed. Instead I
am seeing speeds 3 or 4 times slower (approx 15-22 Mbytes/sec).

 

[Rod Speed]

Thanks Ato-Zee and Rod, and I will try playing with the pagefile and
report back with my results. I run programs like HD Tune which show
the average transfer rates to the drives being in the range of 40 to
70 MBytes/sec depending on the drive, controller, fragmentation of
the drive, etc. (The maximum transfer rates can get up to 85 or 90
Mbytes/sec, the minimum transfer rates as low as 10-15 Mbytes/sec.)

My most important 'measurement' is the actual file transfer times I encounter from drive to drive, typically 5-6
minutes to transfer a 6 GByte file.

OK, thats the main reason for the rate being well below what you expect,
because it involves both reading a writing from two different drives.

If the drive I/O is overlapped and using 2 different controllers, and the bus is far below its theoretical limit, I
would think that I should be able to move a 6 GBytes file in a little over 90 seconds.

Not necessarily with the drives that are connected
to cards as opposed to the motherboard controllers.

In this specific case, I would be reading one drive at 66 Mbytes per second and writing the other at the same speed.
Instead I
am seeing speeds 3 or 4 times slower (approx 15-22 Mbytes/sec).

I'd concentrate on your fastest drive, and thats sectors per track multiplied by RPM,
with the drive on a motherboard controller, and see how that 6GB file move times.

Bet you arent actually seeing the OS running each drive flat out.

 

[Mike Ruskai]

transfer rates as low as 10-15 Mbytes/sec.) My most important 'measurement'
is the actual file transfer times I encounter from drive to drive, typically
5-6 minutes to transfer a 6 GByte file. If the drive I/O is overlapped and
using 2 different controllers, and the bus is far below its theoretical
limit, I would think that I should be able to move a 6 GBytes file in a
little over 90 seconds. In this specific case, I would be reading one drive
at 66 Mbytes per second and writing the other at the same speed. Instead I
am seeing speeds 3 or 4 times slower (approx 15-22 Mbytes/sec).

What bus? You mentioned adapter cards, including PCI. PCI slots, especially
on modern motherboards, where they are added almost as an afterthought in this
age of PCIe, frequently share the same bus as several onboard components. You
may think you're below bus saturation, but that doesn't mean it's true.

I've yet to encounter a single SATA controller that didn't operate at the
drive's maximum speed. One possible exception would be on my current
motherboard, where four SATA ports (only three of which work) are controlled
by a single chip which itself only supports two SATA ports. I have one DVD
drive and one Blu-ray drive attached to two of them, and they are obviously
linked in performance (they both read at the rate of the slowest, so if one is
struggling with a disc of questionable quality, the other slows down to the
same speed, despite having no such problem). I'd hate to see what two hard
drives would behave like attached to those ports.

One scenario that could easily show the lousy performance you're describing is
copying from one drive to another where both are connected to the same PCI
controller.

If you stick with the motherboard's SATA ports (those provided by the main
chipset - extras, like the ones described above, are debatable), or those
provided by PCIe adapter cards, you should be OK.

Without knowing which specific drives are connected to which specific
controllers, the above guesses are as good as it gets.

 

[Smarty]

What bus? You mentioned adapter cards, including PCI. PCI slots,
especially
on modern motherboards, where they are added almost as an afterthought in
this
age of PCIe, frequently share the same bus as several onboard components.
You
may think you're below bus saturation, but that doesn't mean it's true.

I've yet to encounter a single SATA controller that didn't operate at the
drive's maximum speed. One possible exception would be on my current
motherboard, where four SATA ports (only three of which work) are
controlled
by a single chip which itself only supports two SATA ports. I have one
DVD
drive and one Blu-ray drive attached to two of them, and they are
obviously
linked in performance (they both read at the rate of the slowest, so if
one is
struggling with a disc of questionable quality, the other slows down to
the
same speed, despite having no such problem). I'd hate to see what two
hard
drives would behave like attached to those ports.

One scenario that could easily show the lousy performance you're
describing is
copying from one drive to another where both are connected to the same PCI
controller.

If you stick with the motherboard's SATA ports (those provided by the main
chipset - extras, like the ones described above, are debatable), or those
provided by PCIe adapter cards, you should be OK.

Without knowing which specific drives are connected to which specific
controllers, the above guesses are as good as it gets.

The replies from you Mike and Rod are suggesting that the bottle-neck is
either a saturated bus, a controller or two with weak management of
concurrent drives, or OS overhead, in some combination. Clearly, the innate
drive performance with benchmarking programs shows a lot more performance in
reading and writing individually than I can achieve when using a pair of
drives to do useful file transfers, by a factor of maybe 4 to 1. Obviously a
lot of latency, contention, postponed activity, or just some other
time-waster is at work here.

While I agree that some or all of the above is "the problem", I want to
bring up the RAID controller as an alternative approach, if only to
illustrate that the same OS, the same drives, and the same bus can achieve
much faster speeds, causing the drives to achieve much closer to their
"flat-out" read-write performance speeds. This would seem to incriminate the
SATA controllers or the way the OS manages them, and such may indeed be the
case.

For whatever it is worth, I have tried quite a few based on chipsets from
Silicon Image, Intel, and Adaptec, and my complaint for all the machines I
have tested and all the drives I have tested, regardless of whether the
motherboard controllers or add-on PCI/PCI Express controllers are used is
essentially the same. For all combinations of drives and controllers, I have
never seen ANY OF THEM do any better than what I am reporting. Thus, I have
3 recent vintage (Penryn class) computers with controllers on all 3
motherboards as well as add-on controller cards on all 3 machines, as well
as over a dozen drives from WD, Seagate, a couple from Hitachi, including a
10,000 RPM Velociraptor, and not a single transfer I have done over the last
year or so with any drive connected to any internal or eSata port has shown
anything better, except very occasionally, and then only for a short portion
of the transfer, maybe the first .5 GBytes (just guessing).

I guess it is entirely possible that the bus is saturated with a single file
transfer which 'measures' 25 MBytes/sec of ultimate file transfer speed. The
two disks are each getting 25, so the bus traffic should be around 50
MBytes/sec total (assuming DMA versus PIO is being used). But even if they
are using 50 MBytes/sec. My single lane PCI E cards are supposedly capable
of 250 MBytes/sec, and I have a real hard time believing that other activity
chews up 80% of the bus (200 out of 250) leaving only 50 remaining for true
disk I/O.

This is still a mystery to me, and I return back to my assertion that RAID
somehow manages to squeeze a lot more performance out of the drive with
everything else seemingly the same. The OS and disks fill at 3 or 4 times
the rate, so I am still mystified.

I know the tempting 'solution' here is to say: "Why not switch to RAID?" and
I have indeed considered it. I do, however, rely heavily on hot-swapping the
drives to move stuff from computer to computer throughout the day, and like
the ease and portability of SATA and eSata in this regard.

Again, thanks for helping.

 

[JR Weiss]

Pathetic...........

Yes, rodless did the only thing he knows: insult via profanity.

Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the
rate I am experiencing when configured with the right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my
present SATA controllers.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree, and the 5
different controller chipsets I happen to be using all suffer from
this shortfall. Kinda' hard to believe......

I just ventured into eSATA. I found out the [relatively] hard way via
an allegedly bad cable and an RMA. However, the fine techies at
IcyDock set me straight!

It turns out that if a SATA controller (e.g., the ICH9R) does not have
hot-swap capability, the eSATA device must be seen and enumerated by
the BIOS at boot-up. Otherwise you run into those speed problems you
noticed, if the device is recognized at all...

Try this. Set up and power up all eSATA devices on all your computers.
Reboot them all. Check the BIOS screens as they falsh by (if possible)
to see if all the eSATA devices are enumerated. Retest the transfer
rates. You might be pleasantly surprised.

 

[Smarty]

Pathetic...........

Yes, rodless did the only thing he knows: insult via profanity.

Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the
rate I am experiencing when configured with the right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my
present SATA controllers.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree, and the 5
different controller chipsets I happen to be using all suffer from
this shortfall. Kinda' hard to believe......

I just ventured into eSATA. I found out the [relatively] hard way via
an allegedly bad cable and an RMA. However, the fine techies at
IcyDock set me straight!

It turns out that if a SATA controller (e.g., the ICH9R) does not have
hot-swap capability, the eSATA device must be seen and enumerated by
the BIOS at boot-up. Otherwise you run into those speed problems you
noticed, if the device is recognized at all...

Try this. Set up and power up all eSATA devices on all your computers.
Reboot them all. Check the BIOS screens as they falsh by (if possible)
to see if all the eSATA devices are enumerated. Retest the transfer
rates. You might be pleasantly surprised.

I did indeed have the very problem arise which you refer to, and one of my
machines (a Dell XPS420) mis-manages the one and only external eSATA port it
has just as you describe. I added my own SATA controller cards rather than
suffer with the (yet unfixed) Dell issue, and have not seen this type of
slowdown since.

The drives which are internal to the computers are present at boot time, are
correctly enumerated by the BIOS when the machine boots, and still have this
same speed problem. My external eSATA ports, provided by add-in cards, are
actually capable of hot-swap, and do not to seem to care whether the drive
was there or not at boot time since they (usually) mount and dismount
external eSATA drives whenever I chose to. The "Safely Remove" feature seems
to work properly.

Most important is the fact that the transfer speeds stay at this 25
MByte/sec rate regardless of whether the drive was there at boot time, added
later on, dismounted and then remounted, etc.

It is most frustrating to me since my USB2 drives perform at about the same
transfer rates..........25 Mbytes/sec give or take. This is NOT what I was
expecting SATA or eSATA to provide, and hence my post to this newsgroup.

 

[Paul]

Even advice to shove you head up a dead bear's arse ?

Pathetic...........

Yes, rodless did the only thing he knows: insult via profanity.

Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the
rate I am experiencing when configured with the right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my
present SATA controllers.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree, and the 5
different controller chipsets I happen to be using all suffer from
this shortfall. Kinda' hard to believe......

I just ventured into eSATA. I found out the [relatively] hard way via
an allegedly bad cable and an RMA. However, the fine techies at
IcyDock set me straight!

It turns out that if a SATA controller (e.g., the ICH9R) does not have
hot-swap capability, the eSATA device must be seen and enumerated by
the BIOS at boot-up. Otherwise you run into those speed problems you
noticed, if the device is recognized at all...

Try this. Set up and power up all eSATA devices on all your computers.
Reboot them all. Check the BIOS screens as they falsh by (if possible)
to see if all the eSATA devices are enumerated. Retest the transfer
rates. You might be pleasantly surprised.

I did indeed have the very problem arise which you refer to, and one of
my machines (a Dell XPS420) mis-manages the one and only external eSATA
port it has just as you describe. I added my own SATA controller cards
rather than suffer with the (yet unfixed) Dell issue, and have not seen
this type of slowdown since.

The drives which are internal to the computers are present at boot time,
are correctly enumerated by the BIOS when the machine boots, and still
have this same speed problem. My external eSATA ports, provided by
add-in cards, are actually capable of hot-swap, and do not to seem to
care whether the drive was there or not at boot time since they
(usually) mount and dismount external eSATA drives whenever I chose to.
The "Safely Remove" feature seems to work properly.

Most important is the fact that the transfer speeds stay at this 25
MByte/sec rate regardless of whether the drive was there at boot time,
added later on, dismounted and then remounted, etc.

It is most frustrating to me since my USB2 drives perform at about the
same transfer rates..........25 Mbytes/sec give or take. This is NOT
what I was expecting SATA or eSATA to provide, and hence my post to this
newsgroup.

Have you tested any drives with HDTune from hdtune.com ?

Paul

 

[Smarty]

Smarty wrote:


Even advice to shove you head up a dead bear's arse ?

Pathetic...........

Yes, rodless did the only thing he knows: insult via profanity.


Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the
rate I am experiencing when configured with the right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my
present SATA controllers.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree, and the 5
different controller chipsets I happen to be using all suffer from
this shortfall. Kinda' hard to believe......

I just ventured into eSATA. I found out the [relatively] hard way via
an allegedly bad cable and an RMA. However, the fine techies at
IcyDock set me straight!

It turns out that if a SATA controller (e.g., the ICH9R) does not have
hot-swap capability, the eSATA device must be seen and enumerated by
the BIOS at boot-up. Otherwise you run into those speed problems you
noticed, if the device is recognized at all...

Try this. Set up and power up all eSATA devices on all your computers.
Reboot them all. Check the BIOS screens as they falsh by (if possible)
to see if all the eSATA devices are enumerated. Retest the transfer
rates. You might be pleasantly surprised.

I did indeed have the very problem arise which you refer to, and one of
my machines (a Dell XPS420) mis-manages the one and only external eSATA
port it has just as you describe. I added my own SATA controller cards
rather than suffer with the (yet unfixed) Dell issue, and have not seen
this type of slowdown since.

The drives which are internal to the computers are present at boot time,
are correctly enumerated by the BIOS when the machine boots, and still
have this same speed problem. My external eSATA ports, provided by add-in
cards, are actually capable of hot-swap, and do not to seem to care
whether the drive was there or not at boot time since they (usually)
mount and dismount external eSATA drives whenever I chose to. The "Safely
Remove" feature seems to work properly.

Most important is the fact that the transfer speeds stay at this 25
MByte/sec rate regardless of whether the drive was there at boot time,
added later on, dismounted and then remounted, etc.

It is most frustrating to me since my USB2 drives perform at about the
same transfer rates..........25 Mbytes/sec give or take. This is NOT what
I was expecting SATA or eSATA to provide, and hence my post to this
newsgroup.

Have you tested any drives with HDTune from hdtune.com ?

Paul

Yes Paul, and HD Tune shows the average transfer rates to the drives being
in the range of 40 to 70 MBytes/sec maximum transfer rates can get up to 85
or 90 Mbytes/sec, the minimum transfer rates as low as 10-15 Mbytes/sec.)

I'm quite convinced that the drives and controllers, if used only one at a
time, are working correctly. The problem is using a pair of drives to do
useful work, moving a large file from one drive to another.

 

[Paul]

Smarty wrote:


Even advice to shove you head up a dead bear's arse ?

Pathetic...........

Yes, rodless did the only thing he knows: insult via profanity.


Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the
rate I am experiencing when configured with the right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my
present SATA controllers.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree, and the 5
different controller chipsets I happen to be using all suffer from
this shortfall. Kinda' hard to believe......

I just ventured into eSATA. I found out the [relatively] hard way via
an allegedly bad cable and an RMA. However, the fine techies at
IcyDock set me straight!

It turns out that if a SATA controller (e.g., the ICH9R) does not have
hot-swap capability, the eSATA device must be seen and enumerated by
the BIOS at boot-up. Otherwise you run into those speed problems you
noticed, if the device is recognized at all...

Try this. Set up and power up all eSATA devices on all your computers.
Reboot them all. Check the BIOS screens as they falsh by (if possible)
to see if all the eSATA devices are enumerated. Retest the transfer
rates. You might be pleasantly surprised.


I did indeed have the very problem arise which you refer to, and one
of my machines (a Dell XPS420) mis-manages the one and only external
eSATA port it has just as you describe. I added my own SATA
controller cards rather than suffer with the (yet unfixed) Dell
issue, and have not seen this type of slowdown since.

The drives which are internal to the computers are present at boot
time, are correctly enumerated by the BIOS when the machine boots,
and still have this same speed problem. My external eSATA ports,
provided by add-in cards, are actually capable of hot-swap, and do
not to seem to care whether the drive was there or not at boot time
since they (usually) mount and dismount external eSATA drives
whenever I chose to. The "Safely Remove" feature seems to work properly.

Most important is the fact that the transfer speeds stay at this 25
MByte/sec rate regardless of whether the drive was there at boot
time, added later on, dismounted and then remounted, etc.

It is most frustrating to me since my USB2 drives perform at about
the same transfer rates..........25 Mbytes/sec give or take. This is
NOT what I was expecting SATA or eSATA to provide, and hence my post
to this newsgroup.

Have you tested any drives with HDTune from hdtune.com ?

Paul

Yes Paul, and HD Tune shows the average transfer rates to the drives
being in the range of 40 to 70 MBytes/sec maximum transfer rates can get
up to 85 or 90 Mbytes/sec, the minimum transfer rates as low as 10-15
Mbytes/sec.)

I'm quite convinced that the drives and controllers, if used only one at
a time, are working correctly. The problem is using a pair of drives to
do useful work, moving a large file from one drive to another.

When you test disk to disk transfer, are large files involved ? Or a
lot of small files ? For benchmarking purposes, you'd want to avoid a
lot of random head movement.

Maybe as a test case, you could try running HDTach and HDTune at the
same time, focused on different hard drives. That might take Windows
out of the picture, as the access there is raw, rather than through a
file system.

In the spirit of general debugging, maybe you could download Process
Explorer from sysinternals.com, and keep an eye on system operation,
while one of your disk to disk transfers is running. When performance
drops to 25MB/sec from one disk to the other, see if any other stats
change at that point in time.

Paul

 

[Smarty]

Smarty wrote:
Smarty wrote:


Even advice to shove you head up a dead bear's arse ?

Pathetic...........

Yes, rodless did the only thing he knows: insult via profanity.


Since these drives including the fastest one I own, a WD
Velociraptor, can sustain read/write transfers at several times the
rate I am experiencing when configured with the right RAID
controller, I am reasonably certain that the drive is inherently
capable of much faster performance than I am experiencing with my
present SATA controllers.

The "simple" explanation would be that (some) SATA controllers
underperform the SATA specs by a considerable degree, and the 5
different controller chipsets I happen to be using all suffer from
this shortfall. Kinda' hard to believe......

I just ventured into eSATA. I found out the [relatively] hard way via
an allegedly bad cable and an RMA. However, the fine techies at
IcyDock set me straight!

It turns out that if a SATA controller (e.g., the ICH9R) does not have
hot-swap capability, the eSATA device must be seen and enumerated by
the BIOS at boot-up. Otherwise you run into those speed problems you
noticed, if the device is recognized at all...

Try this. Set up and power up all eSATA devices on all your
computers.
Reboot them all. Check the BIOS screens as they falsh by (if
possible)
to see if all the eSATA devices are enumerated. Retest the transfer
rates. You might be pleasantly surprised.


I did indeed have the very problem arise which you refer to, and one of
my machines (a Dell XPS420) mis-manages the one and only external eSATA
port it has just as you describe. I added my own SATA controller cards
rather than suffer with the (yet unfixed) Dell issue, and have not seen
this type of slowdown since.

The drives which are internal to the computers are present at boot
time, are correctly enumerated by the BIOS when the machine boots, and
still have this same speed problem. My external eSATA ports, provided
by add-in cards, are actually capable of hot-swap, and do not to seem
to care whether the drive was there or not at boot time since they
(usually) mount and dismount external eSATA drives whenever I chose to.
The "Safely Remove" feature seems to work properly.

Most important is the fact that the transfer speeds stay at this 25
MByte/sec rate regardless of whether the drive was there at boot time,
added later on, dismounted and then remounted, etc.

It is most frustrating to me since my USB2 drives perform at about the
same transfer rates..........25 Mbytes/sec give or take. This is NOT
what I was expecting SATA or eSATA to provide, and hence my post to
this newsgroup.


Have you tested any drives with HDTune from hdtune.com ?

Paul

Yes Paul, and HD Tune shows the average transfer rates to the drives
being in the range of 40 to 70 MBytes/sec maximum transfer rates can get
up to 85 or 90 Mbytes/sec, the minimum transfer rates as low as 10-15
Mbytes/sec.)

I'm quite convinced that the drives and controllers, if used only one at
a time, are working correctly. The problem is using a pair of drives to
do useful work, moving a large file from one drive to another.

When you test disk to disk transfer, are large files involved ? Or a
lot of small files ? For benchmarking purposes, you'd want to avoid a
lot of random head movement.

Maybe as a test case, you could try running HDTach and HDTune at the
same time, focused on different hard drives. That might take Windows
out of the picture, as the access there is raw, rather than through a
file system.

In the spirit of general debugging, maybe you could download Process
Explorer from sysinternals.com, and keep an eye on system operation,
while one of your disk to disk transfers is running. When performance
drops to 25MB/sec from one disk to the other, see if any other stats
change at that point in time.

Paul

My file transfer issue has come up with the only files I frequently
transfer, approximately 6 GB size. The same relative lack of speed shows up
for smaller transfers as well, but it only bothers me when I have to wait an
appreciable amount of time, and thus the smaller transfers which are
typically done in much less than a minute do not concern me if though they
also show about the same transfer rates.

I really like the suggestion of using Process Explorer to see what may be
happening behind the scenes here. I will try that is see what I can find. I
have been pruning the background 'crap' using the Task Manager and other
tools so I have a relatively short list of services and processes running,
but I have not really studied what the actual run-time contention for the
cores may be.

I will report back with any significant findings, and thank you Paul for the
helpful suggestion.

 

[Rod Speed]

The replies from you Mike and Rod are suggesting that the bottle-neck
is either a saturated bus, a controller or two with weak management of
concurrent drives, or OS overhead, in some combination. Clearly, the
innate drive performance with benchmarking programs shows a lot more
performance in reading and writing individually than I can achieve
when using a pair of drives to do useful file transfers, by a factor
of maybe 4 to 1. Obviously a lot of latency, contention, postponed
activity, or just some other time-waster is at work here.

While I agree that some or all of the above is "the problem", I want
to bring up the RAID controller as an alternative approach, if only to
illustrate that the same OS, the same drives, and the same bus can
achieve much faster speeds, causing the drives to achieve much closer
to their "flat-out" read-write performance speeds. This would seem to
incriminate the SATA controllers or the way the OS manages them, and
such may indeed be the case.

For whatever it is worth, I have tried quite a few based on chipsets
from Silicon Image, Intel, and Adaptec, and my complaint for all the
machines I have tested and all the drives I have tested, regardless
of whether the motherboard controllers or add-on PCI/PCI Express
controllers are used is essentially the same. For all combinations of
drives and controllers, I have never seen ANY OF THEM do any better
than what I am reporting. Thus, I have 3 recent vintage (Penryn
class) computers with controllers on all 3 motherboards as well as
add-on controller cards on all 3 machines, as well as over a dozen
drives from WD, Seagate, a couple from Hitachi, including a 10,000
RPM Velociraptor, and not a single transfer I have done over the last
year or so with any drive connected to any internal or eSata port has
shown anything better, except very occasionally, and then only for a
short portion of the transfer, maybe the first .5 GBytes (just
guessing).
I guess it is entirely possible that the bus is saturated with a
single file transfer which 'measures' 25 MBytes/sec of ultimate file
transfer speed. The two disks are each getting 25, so the bus traffic
should be around 50 MBytes/sec total (assuming DMA versus PIO is
being used). But even if they are using 50 MBytes/sec. My single lane
PCI E cards are supposedly capable of 250 MBytes/sec, and I have a
real hard time believing that other activity chews up 80% of the bus
(200 out of 250) leaving only 50 remaining for true disk I/O.

This is still a mystery to me, and I return back to my assertion that
RAID somehow manages to squeeze a lot more performance out of the
drive with everything else seemingly the same. The OS and disks fill
at 3 or 4 times the rate, so I am still mystified.

I know the tempting 'solution' here is to say: "Why not switch to
RAID?" and I have indeed considered it. I do, however, rely heavily
on hot-swapping the drives to move stuff from computer to computer
throughout the day, and like the ease and portability of SATA and
eSata in this regard.

A much more fundamental question is why are you moving files around so much ?

 

[Smarty]

A much more fundamental question is why are you moving files around so
much ?

I'm building a video server with (eventually) ~4 terabytes of content, most
of which already resides on about 14 500GB drives........

 

[Rod Speed]

I'm building a video server with (eventually) ~4 terabytes of
content, most of which already resides on about 14 500GB drives........

So why do you need to move the files around ? I do the same thing,
but just play them from the server, dont move them around. In fact
the PVR and video server are the one thing and its the media PC too.

 

[Smarty]

So why do you need to move the files around ? I do the same thing,
but just play them from the server, dont move them around. In fact
the PVR and video server are the one thing and its the media PC too.

It's mostly an issue of cosmetics and future maintenance Rod. I strongly
prefer to have a single cabinet with 4 drives, each with a terabyte, rather
than spreading the files around on a lot of drives. It allows for much
easier backups, power management, etc.

I just remembered / discovered a very important fact that had not been
considered or mentioned earlier in this post. It occurred only because I
decided to run a backup on my primary system, whose boot drive has a 61 GB
backup file created by Acronis True Image 2009, an excellent disk imaging
program.

I started the program up earlier this evening as I do every week or two, and
had a complete backup / image of my 61 GB hard drive completed in slightly
less than 20 minutes total elapsed time. I had never considered this fact in
my earlier posting here, but this very same system running Vista is somehow
reading the C drive and writing the 61 GB output file at a rate of 3 GB per
minute, roughly 3 times the speed which Vista takes to do a simple file
copy.

I guess this illustrates an important fact to me, namely, that this hardware
and OS can, under some circumstances, move data at the type of speed I would
have expected for SATA. It does NOT explain why Vista cannot do a transfer /
copy at anywhere near this rate, and the only new thought that now comes to
mind is that perhaps a "read after write verify" is always being done by
Vista, effectively cutting the transfer speed by 50%.

I seem to recall long ago that the choice was available in Windows to
disable read after write verify, and now cannot recall seeing this choice in
recent years. I wonder if this may account for a big part of my original
complaint.............

I guess it is also possible that Acronis loads its own drivers and somehow
talks to the disks in a manner different from Windows, but I highly doubt
it. There is a stand-alone bootable "rescue" disk which Acronis also
provides which runs with a Linux kernel, but the app I use is a Vista app
which never leaves Windows as far as I can tell.

More food for thought........

 

[Rod Speed]

It's mostly an issue of cosmetics and future maintenance Rod.

Not really.

I strongly prefer to have a single cabinet with 4 drives, each with a
terabyte, rather than spreading the files around on a lot of drives.
It allows for much easier backups, power management, etc.

Sure, but with appropriate RAID on that system, you still dont
need to furiously copy large files around. You just play them
from there without moving them around before you play them.

I just remembered / discovered a very important fact that had not been considered or mentioned earlier in this post.
It occurred only
because I decided to run a backup on my primary system, whose boot drive has a 61 GB backup file created by Acronis
True Image 2009, an excellent disk imaging program.

Yeah, I prefer it myself.

I started the program up earlier this evening as I do every week or
two, and had a complete backup / image of my 61 GB hard drive
completed in slightly less than 20 minutes total elapsed time. I had
never considered this fact in my earlier posting here, but this very
same system running Vista is somehow reading the C drive and writing the 61 GB output file at a rate of 3 GB per
minute, roughly 3 times the speed which Vista takes to do a simple file copy.

Yeah, like I said, the OS isnt really optimised for fast file copying,
essentially because it makes no sense to furiously copy large files
around. Thats best left to the apps like backup that does much of that.

I guess this illustrates an important fact to me, namely, that this
hardware and OS can, under some circumstances, move data at the type of speed I would have expected for SATA. It does
NOT explain why Vista cannot do a transfer / copy at anywhere near this rate,

Essentially because its not optimised for that sort of thing.

and the only new thought that now comes to mind is that perhaps a "read after write verify" is always being done by
Vista, effectively cutting the transfer speed by 50%.

No, that is not done.

I seem to recall long ago that the choice was available in Windows to disable read after write verify, and now cannot
recall seeing this choice in recent years. I wonder if this may account for a big part of my original
complaint.............
Nope.

I guess it is also possible that Acronis loads its own drivers and
somehow talks to the disks in a manner different from Windows,

Yes it does, particularly with the source disk. It doesnt just copy files around.

but I highly doubt it. There is a stand-alone bootable "rescue" disk which Acronis also provides which runs with a
Linux kernel, but the app I use is a Vista app which never leaves Windows as far as I can tell.

That last is correct. Thats why you can keep using the system while the backup is happening.

More food for thought........

Just watch out that your brain doesnt end up obese.