IBM DS4300 sustained throughput - only 75MBps

  • Thread starter Thread starter Siddhartha Jain
  • Start date Start date
S

Siddhartha Jain

I have one DS4300 with the following specs:
- 10 FC 73GB 15k rpm disks (Eight in a RAID-0 + 2 global hot spares)
- Dual Controller with 512MB cache

The server connected to the DS4300 is an IBM x366 with Quad Xeon 3.06
GHz, 4GB RAM and a single SAS 73GB 10k rpm disk with Windows 2003
32-bit Server.

The HBAs I am using are two QLogic 2340 in redundant/failover mode. I
have used the latest HBA Drivers (Storport as well as SCSIport) from
Qlogic.

The maximum throughput I get when I run HDTune on the box is about
95Mbytes/sec with a sustained rate of 70-75Mbytes/sec.

So what about that advertised 2Gbps (200 Mbytes/sec) througput?

Thanks,

- Siddhartha
 
Siddhartha said:
I have one DS4300 with the following specs:
- 10 FC 73GB 15k rpm disks (Eight in a RAID-0 + 2 global hot spares)
- Dual Controller with 512MB cache

The server connected to the DS4300 is an IBM x366 with Quad Xeon 3.06
GHz, 4GB RAM and a single SAS 73GB 10k rpm disk with Windows 2003
32-bit Server.

The HBAs I am using are two QLogic 2340 in redundant/failover mode. I
have used the latest HBA Drivers (Storport as well as SCSIport) from
Qlogic.

The maximum throughput I get when I run HDTune on the box is about
95Mbytes/sec with a sustained rate of 70-75Mbytes/sec.

So what about that advertised 2Gbps (200 Mbytes/sec) througput?

Thanks,

- Siddhartha
Click to expand...
2Gb/s is the peak rate on the FC, not the STR. 95 MB/s sound kinda poor
for FC, but I've never used that HBA.

Just a thought: if those Qlogic 2340 HBAs are plugged into ordinary
(33MHz 32bit) PCX slots then the PCI will constrain STR to something
like 95 MB/s.
 
Bob said:
2Gb/s is the peak rate on the FC, not the STR. 95 MB/s sound kinda poor
for FC, but I've never used that HBA.

Just a thought: if those Qlogic 2340 HBAs are plugged into ordinary
(33MHz 32bit) PCX slots then the PCI will constrain STR to something
like 95 MB/s.
Click to expand...

95MB/s is the peak I got, 70-75MB/s was the sustained rate. And I have
these HBAs plugged into PCI-X 2.0 266Hz 64-bit slots. The IBM x366 has
no 32-bit 33Mhz slots. The HBAs are Qlogic OEM sold by IBM with their
DS4000 boxen.

- Siddhartha
 
I have one DS4300 with the following specs:
95MB/s is the peak I got, 70-75MB/s was the sustained rate. And I have
these HBAs plugged into PCI-X 2.0 266Hz 64-bit slots. The IBM x366 has
no 32-bit 33Mhz slots. The HBAs are Qlogic OEM sold by IBM with their
DS4000 boxen.
Click to expand...

HDTune might not be the best tool for that test.

Did you try to copy a large file (at least 4GB) and measure time, as a
crude/simple benchmark.
Use performance monitor to see what Bytes/sec you are getting.
 
Peter said:
HDTune might not be the best tool for that test.

Did you try to copy a large file (at least 4GB) and measure time, as a
crude/simple benchmark.
Use performance monitor to see what Bytes/sec you are getting.
Click to expand...

Copying from disk to disk a large file might be a bad performance
measurement tool. The other disk could be slower than the target or
vice versa. I think the best is to use some tool that generates random
data in the memory and writes it to the disk since RAM/CPU will always
be faster than disks. And this is exactly what tools like
HDTune/IOMeter do. The figures I got with IOMeter are even more
abysmal.

- Siddhartha
 
HDTune might not be the best tool for that test.
Copying from disk to disk a large file might be a bad performance
measurement tool. The other disk could be slower than the target or
vice versa. I think the best is to use some tool that generates random
data in the memory and writes it to the disk since RAM/CPU will always
be faster than disks. And this is exactly what tools like
HDTune/IOMeter do. The figures I got with IOMeter are even more
abysmal.
Click to expand...

Did you ever try to:
copy <disk>:\path\largefile nul
?

What parameters did you use for IOMeter? How "abysmal"?
 
Peter said:
Did you ever try to:
copy <disk>:\path\largefile nul
?

What parameters did you use for IOMeter? How "abysmal"?
Click to expand...

Ok, I created 40GB file (filled in 1s) on the SAN *exported* disk and
ran:
copy F:\log2.txt NUL

Results:
- Peak of 93MB/sec
- Avg of 83MB/sec
- Cache Hit 100%
- Read Percentage 100%
- Max I/O per sec - 1444

I am still figuring out how to use IOMeter better.

- Siddhartha
 
Ok, I created 40GB file (filled in 1s) on the SAN *exported* disk and
ran:
copy F:\log2.txt NUL

Results:
- Peak of 93MB/sec
- Avg of 83MB/sec
- Cache Hit 100%
- Read Percentage 100%
- Max I/O per sec - 1444

I am still figuring out how to use IOMeter better.
Click to expand...

Windows 2003 does perform mostly 64KB IOs. Performance would get better with
higher IO block size.
You may also get better results with Linux OSes.

Your results (80-100MB/s) are normal for a single HBA in Windows
environment.

You may design your storage differently if you plan to use it for multimedia
applications.
 
Peter said:
Windows 2003 does perform mostly 64KB IOs. Performance would get better with
higher IO block size.
You may also get better results with Linux OSes.

Your results (80-100MB/s) are normal for a single HBA in Windows
environment.

You may design your storage differently if you plan to use it for multimedia
applications.
Click to expand...

The server has two HBAs but they are only in failover mode since I
don't know how to *team* them. Other than that, the segment size IS
64Kb on the SAN volume.

(Forgive my stupid question), but whats that 2Gbps figure again? Each
port on the DS4300 supposedly supports 2Gbps, each HBA supports 2Gbps
(PCI-X 2.0 133Mhz) and the read cache has a 100% hit which means we are
not reading from disk but from the cache RAM of the controller. So why
the ~1Gbps performance? Whats the *gotcha* in the fine print of these
boxen?

- Siddhartha
 
Ok, I created 40GB file (filled in 1s) on the SAN *exported* disk and
The server has two HBAs but they are only in failover mode since I
don't know how to *team* them. Other than that, the segment size IS
64Kb on the SAN volume.

(Forgive my stupid question), but whats that 2Gbps figure again? Each
port on the DS4300 supposedly supports 2Gbps, each HBA supports 2Gbps
(PCI-X 2.0 133Mhz) and the read cache has a 100% hit which means we are
not reading from disk but from the cache RAM of the controller. So why
the ~1Gbps performance? Whats the *gotcha* in the fine print of these
boxen?
Click to expand...

2Gbps is a maximum throughput of your QLogic HBA card.
With Windows CIFS some other parameters come to play, limiting it to less.

Try convincing youself using SQLIO on a Windows platform.
http://download.microsoft.com/download/f/3/f/f3f92f8b-b24e-4c2e-9e86-d66df1f6f83b/SQLIO.msi
Set threads to 4 (params.txt), outstanding IO to 8 (-o8) and play with
blocksize.
(I have also used 1GB file) and I get around:
120MB/s with -b64
140MB/s with -b128
145MB/s with -b256
but with one thread and one outstanding IO only:
43MB/s with -b64

That is how CIFS on Windows works.
 
Thanks for the tip about using SQLIO. I did and here are the results
with four threads and 2GB file size:

Random Read
===========
b128 97.30 MBs/sec
b256 114.01 MBs/sec
b512 113.90 MBs/sec

Sequential Read
================
b128 158.84 MBs/sec
b256 164.93 MBs/sec
b512 168.94 MBs/sec

Random Write
============
b128 70.18 MBs/sec
b256 76.81 MBs/sec
b512 80.82 MBs/sec

Sequential Write
================
b128 73.98 MBs/sec
b256 78.92 MBs/sec
b512 82.15 MBs/sec

I am surprised to see that there isn't much difference between random
and sequential write.

- Siddhartha
 
Thanks for the tip about using SQLIO. I did and here are the results
with four threads and 2GB file size:

Random Read
===========
b128 97.30 MBs/sec
b256 114.01 MBs/sec
b512 113.90 MBs/sec

Sequential Read
================
b128 158.84 MBs/sec
b256 164.93 MBs/sec
b512 168.94 MBs/sec

Random Write
============
b128 70.18 MBs/sec
b256 76.81 MBs/sec
b512 80.82 MBs/sec

Sequential Write
================
b128 73.98 MBs/sec
b256 78.92 MBs/sec
b512 82.15 MBs/sec

I am surprised to see that there isn't much difference between random
and sequential write.
Click to expand...

That might depend on DS4300 cache configuration.
Some results from IBM benchmark here:
http://www.virtual.com/whitepapers/IBM_xSeries_DS4300_Iometer_Performance_wp.pdf
Some guidelines:
http://www.redbooks.ibm.com/redbooks/pdfs/sg245287.pdf
 
Back
Top