Where's my Bottleneck?

  • Thread starter Thread starter lucas.allen
  • Start date Start date
L

lucas.allen

Hey everyone,
I have a copy station that we use to duplicate data from 3 pressurized
"flight data storage(FDS)" SCSI hard drives to multiple sets of hard
drives...approx 700gb of data to copy. This process takes about 6-8
hours depending on how full the FDS's are to start with. The FDS's are
connected in the aircraft with the FIBRE channel technology so it's
smokin fast but the bottleneck occurs during the duplication....here's
the setup:
3xFDS's (3 SCSI drives) ---> Madison Cable (FIBRE @ 1gb) -----> Qlogic
Fibre channel adapter (Fibre @ 2gb at least) -----> then the computer
is a P4 2.7 GHZ but I dobut processor is even utilized ----> 2
controllers (1 Promise Technology Ultra 133 Tx2) has 2 drives on it and
(1 DC-133 RAID www.dawicontrol.com) has 1 drive on it ----> then we
have multiple sets of 3 hdd (DE75i CA100/B) that are hot swappable into
3 HDD bays.

I am having a lot of trouble finding where the bottleneck is in this
setup but I'm guessing it's with the Controllers and/or the HDD's. I'm
wondering A.) What I can do to our current copy station to make it
faster and B.) Build another copy station we can take on the road with
us using the FIBRE channel technology that's as fast and light weight
as possible to make this process easier and faster.

Any and all information is appreciated.
Thank you all,
Lucas
 
Previously said:
Hey everyone,
I have a copy station that we use to duplicate data from 3 pressurized
"flight data storage(FDS)" SCSI hard drives to multiple sets of hard
drives...approx 700gb of data to copy. This process takes about 6-8
hours depending on how full the FDS's are to start with. The FDS's are
connected in the aircraft with the FIBRE channel technology so it's
smokin fast but the bottleneck occurs during the duplication....here's
the setup:
Click to expand...

Are these standard SCSI drives, just in a special case?
3xFDS's (3 SCSI drives) ---> Madison Cable (FIBRE @ 1gb) -----> Qlogic
Fibre channel adapter (Fibre @ 2gb at least) -----> then the computer
is a P4 2.7 GHZ but I dobut processor is even utilized ----> 2
controllers (1 Promise Technology Ultra 133 Tx2) has 2 drives on it and
(1 DC-133 RAID www.dawicontrol.com) has 1 drive on it ----> then we
have multiple sets of 3 hdd (DE75i CA100/B) that are hot swappable into
3 HDD bays.
Click to expand...
I am having a lot of trouble finding where the bottleneck is in this
setup but I'm guessing it's with the Controllers and/or the HDD's. I'm
wondering A.) What I can do to our current copy station to make it
faster and B.) Build another copy station we can take on the road with
us using the FIBRE channel technology that's as fast and light weight
as possible to make this process easier and faster.
Click to expand...

Hmm, you see about 100GB/hur, that is about 27MB/sec. That is not slow!
Depending on the SCSI disk drives, that may just be their maximum
sustained read rate, expecially if this is a filesystem-read (with seeks)
and not a linear read.

Also the disks you are writing to may be the bottleneck, if these
are either older or 2.5" disks or thiese are (again) writes
to a filesystem.

Ok, first: What OS? What filesustem? What disks on the PC side?

Arno
 
Arno said:
Are these standard SCSI drives, just in a special case?



Hmm, you see about 100GB/hur, that is about 27MB/sec. That is not slow!
Depending on the SCSI disk drives, that may just be their maximum
sustained read rate, expecially if this is a filesystem-read (with seeks)
and not a linear read.

Also the disks you are writing to may be the bottleneck, if these
are either older or 2.5" disks or thiese are (again) writes
to a filesystem.

Ok, first: What OS? What filesustem? What disks on the PC side?

Arno
Click to expand...

I would think the bottleneck is on the writes. One thing you could do
is if the station OS is Windows, is to change the filesystem you are
writing to, to be compressed. If it's linux, you can "cat" the source
file, pipe it to gzip with the output to standard out, and then
redirect that output to a file on the local system. Doing so will
reduce the amount of data you are writing out to the ATA disks...

Good Luck!
 
Back
Top