What's the fastest hard drive for a HP Pavilion (Ultra IDE, non-SATA)? Scsi?

[dean]

Taking a look around, I cannot quickly find a PCI card, with 32 bit bus
connection, offering a U320 bus. There are plenty of cards offering
U160, which is closer to the 120MB/sec limitation that comes with a
PCI 32 bit, 33MHz bus. Chances are, the U320 drive you are buying, could
work at a number of different SCSI bus transfer rates, all the way down to
20MB/sec or maybe even lower. So the declared drive speed, does not mean
you need to buy a U320 card. The interface on the drive will probably be
LVDS (low voltage differential swing), and that level and type of interface
has existed for years. A U160 would still support the sustained media
transfer rate of the fastest disks. The main benefit of the U320
transfer rate, when using a single disk, would be the ability to
"burst to cache" on the SCSI disk's controller.

There are cards which use one of the faster PCI standards. For example,
you can have 32 bit PCI cards operating at 66MHz. But you are unlikely
to find support for that on any old desktop motherboard. Another option,
would be cards that are 64 bits wide. That means a longer connector on
the PCI card.

This is an example of a longer card.

http://discountechnology.com/Adaptec-2120S-U320-LVD-SCSI-RAID-Control...

Once you get into this league, there are some other things to consider.
The PCI-X higher speed cards, for example, operate at 3.3V only. Some
people get a rotten surprise, when they think they found a bargain
somewhere, and then they notice that the voltage slots cut in the card,
prevent it from fitting in their motherboard.

In this closeup picture, you can see three slots cut. The right most
one, separates the two 32 bit parts of the interface. If you plug a 64
bit card, into a 32 bit slot, it can still work, and 32 of the contacts
are left "hanging in the air". The rightmost slot is there to leave room
for the end of the connector. This is a U320 card, but if plugged into a
32 bit, 322MHz PCI desktop slot, the max transfer rate achieved will be
120MB/sec or a little less.

http://files.discountechnology.com/products/ADP-2120S-BN-FW/ADP-2120S...

Next to the 32 bit separator slot, is the 5V key. A lot of desktop
motherboards, would have that key in place. It means the VIO selected
for the motherboard is 5 volts. A motherboard designer has to make some choice
of either 5V or 3.3V when the motherboard is designed, and 5V is compatible
with a lot of older stuff.

The left-most slot cut in the sample card, is for 3.3V. Since that
card has both 3.3V and 5V VIO capability (i.e. universal), it can
work with motherboards that choose to supply 3.3V or 5V. So that
card should be able to plug into a desktop, and also into the faster
slots (like PCI-X or PCI 64) on a server motherboard.

You have to be careful with this stuff. At least Adaptec, provides information
for some of their products, so you know whether the card will work for sure
in a PCI 32 bit slot or not.

So if you buy a U160 card, with an ordinary PCI slot connector on it,
then there should be no problem plugging it into a desktop motherboard.
For some of the other types, you can either rely on the slots cut in
the board, as an indicator, or check the manufacturer's web site, for
confirmation of interoperability. Notice, our example ADP-2120S card
above, is listed as "64-bit or 32-bit *" and "66 or 33 MHz" in the following
web page. The 33MHz part is reassuring, since that is the speed of a
desktop PCI slot. The card has no voltage limitation shown, and we knew
that since it had both the 5V and 3.3V slots cut in it, for VIO choices.
The asterisk seems to suggest they are looking for a more recent PCI slot
standard (2.1 or 2.2) and PCI 2.1 has been around for some time.

http://adaptec-tic.adaptec.com/cgi-bin/adaptec_tic.cfg/php/enduser/st...

So you can grab a card that has a 64 bit connector on it, but it would
pay to do some research first, to confirm it would work. There is the
odd case, where the plastic frame on the 32 bit connector, is too wide
to fit into the separator slot in the 64 bit card, but I don't seem to
see that mentioned any more.

One test I've used before, to eliminate disk speed from an experiment,
is to use RAM disk software. That makes storage space, from a section of
RAM. That is only feasible, if you have lots of RAM to play with. For
example, I wanted to see how fast a network connection I could make
between two computers. I had a Gigabit connection, and both computers
had Win2K. Each computer had 1GB of RAM, so I made a 100MB RAM disk on
each computer. I was only able to achieve 40MB/sec, only 1/3rd of the
available gigabit link speed (which is what that particular OS is known
to be able to achieve) with the setup. By using RAM disks, I was attempting
to eliminate the storage devices from messing with my experiment. The
RAM disks I've used, were not the cleanest things to install and remove,
so they tend to be "creaky" pieces of software. Microsoft has a sample
RAM disk implementation, and some of the available RAM disks are based
on the sample implementation. I'm not sure if anyone makes a "smooth"
and "flexible" RAM disk implementation, that is free from flaws.

If you are processor limited, then the solution to that should be
more straight forward.

Paul

Paul - thanks for that update, its very useful. Took me quite a while
to determine even what kind of PCI slots I have.

For the ram-disk you mentioned, can you not do the same thing by
loading files up twice - first time comes from the actual disk, second
time its in the system cache memory? If, as I mentioned above, I
'reverse' the jpegs and replay the ones that were just loaded viewed,
I see a nearly doubleing of frame rate, since the OS knows the jpegs
are in cache and have not changed on the hard drive.

 

[dean]

If you're considering a (relatively) expensive SCSI PCI
card, you might as well consider an SATA card if you felt
the WD (Or a RAID0 of two of them) might be an option.- Hide quoted text -

- Show quoted text -

The specs are are 2.54GHz P4 and the pics are on a WD 7500AAK external
USB 750GB drive. I've compared it to the 76GB internal drive (sorry
not sure what brand) and there's no difference in speed.

If I reverse the images and load them all from memory cache instead,
its just less than twice as fast (not quite 2x faster), so it seems I
am indeed (as mentioned) slowed by my system - I bet that is all I can
possibly get given my processor. Maybe a new PC is in order.

Am I wrong or are there no Ultra320 SCSI cards out there for PCI (not
PCI-X or PCI-E) slots?

 

[dean]

There are bare disk drives which are 1TB in capacity. So you can
buy a single drive that holds a terabyte.

Paul

Paul - thanks for that update, its very useful. Took me quite a while
to determine even what kind of PCI slots I have.

For the ram-disk you mentioned, can you not do the same thing by
loading files up twice - first time comes from the actual disk, second
time its in the system cache memory? If, as I mentioned above, I
'reverse' the jpegs and replay the ones that were just loaded viewed,
I see a nearly doubleing of frame rate, since the OS knows the jpegs
are in cache and have not changed on the hard drive.

 

[dean]

These numbers don't seem right to me - or at least they seem misleading. I
don't believe that any hard disk available would be 50% faster than a
Raptor. SCSI might be 10-20% faster at a push, but you would have to look at
a fast SCSI RAID setup to get a noticable improvement in speed. That means
you are probably talking a new power supply, new case to fit it all in, more
cooling and more noise etc.

Is it really worth it? Either I'm missing something or this is a strange
exercise - if you need to examine all the images in detail, then image load
times won't be the slow part of the process. Can't you just open paint shop
pro and let the browse function index the folder, or just use WinXP
thumbnail view to preview things?

Here's another thing to consider: If you are opening all these images at
once, or sequentially without closing them, then perhaps you are running out
of RAM and the computer is resorting to virtual memory paging on the same
drive as it is trying to load the images from?- Hide quoted text -

- Show quoted text -

10K WD Raptor: 84MB/s max.
15K Cheetah 125MB/s max.
That is 48% faster sustained transfer rate.

Here's what is happening. I wrote a progam that loads up each frame
(taken once every second) and blits it to the screen. Each file is
loaded from memory, and then painted to the windows canvas, and then
freed from memory. 1 day's woth (86000 pictures) is way too much to
put into memory. One camera is 30K per picture, the other is 300K,
totaling 2GB and 20GB per day.

I pull them up off an FTP server onto a local USB drive, and once that
is done I can scan the pics quite fast.

I can speed up the viewing by skipping files (only viewing ivery 3rd
picture, for example). I am not examining each picture, I am watching
to see if anyone at all has been in the area.

So instead of slipping images, I would rather blit them faster to the
screen, and the only way I can do this is to improve some combination
of processor speed and/or disk speed or some OS tricks. I'll look at
the setting you mentioned above tonight.

Thanks for all the help too!

Dean

 

[~misfit~]

Somewhere on the interweb "dean" typed:

Can anyone recommend fastest drive for my HP pavilion? It only has IDE
right now, and I was thinking about buying a Ultra320 SCSI controller
card (PCI) and a 15K rpm SCSI drive to match. Cost as around $400 for
the pair.

Is this an ok plan?

(Use case is: I want to be able to scroll through a series of JPGs on
hard drive from a security camera as fast as possible. )

I skimmed the long list of replies and didn't see temperature problems
addressed.

It is my experience that HP/Compaq/Dell etc. systems are built with little
tolerance for change.

Superfast drive = lots of heat output. Unless that heat is removed it will
also = dead HDD. Regardless of interface.

 

[kony]

Paul - thanks for that update, its very useful. Took me quite a while
to determine even what kind of PCI slots I have.

For the ram-disk you mentioned, can you not do the same thing by
loading files up twice - first time comes from the actual disk, second
time its in the system cache memory? If, as I mentioned above, I
'reverse' the jpegs and replay the ones that were just loaded viewed,
I see a nearly doubleing of frame rate, since the OS knows the jpegs
are in cache and have not changed on the hard drive.

Yes, but the cache size has to be increased by the registry
entry to be sufficiently large enough for several hundred MB
worth of JPEGs, you have to have enough system memory, and
portions of that cache may be flushed out if multitasking,
so they'd have to be reread from disk at that point. The
advantage of the ramdrive is nothing flushes it out, it is
reserved for the purpose. The problem is the size it will
have to be to hold more than a few hours worth of JPEGs at
any reasonable resolution and (not too great, so as to
preserve quality, though I've no idea if you can control
this) compression rate.

There's another ramdrive option, an add-on card onto which
you can install that memory instead of usurping system
memory for the job. For example Gigabyte's i-Ram product
(though there may be newer versions of this today).
http://www.ewiz.com/detail.php?name=MB-RAMDISK&src=Deal

 

[kony]

But either will still be limited by the PCI bus!

I suspect this is not going to be as much of a bottleneck as
drive internal I/O rate, filesystem overhead, or CPU
decompression performance.

 

[kony]

10K WD Raptor: 84MB/s max.
15K Cheetah 125MB/s max.
That is 48% faster sustained transfer rate.

Here's what is happening. I wrote a progam that loads up each frame
(taken once every second) and blits it to the screen. Each file is
loaded from memory, and then painted to the windows canvas, and then
freed from memory. 1 day's woth (86000 pictures) is way too much to
put into memory. One camera is 30K per picture, the other is 300K,
totaling 2GB and 20GB per day.

I pull them up off an FTP server onto a local USB drive, and once that
is done I can scan the pics quite fast.

I can speed up the viewing by skipping files (only viewing ivery 3rd
picture, for example). I am not examining each picture, I am watching
to see if anyone at all has been in the area.

So instead of slipping images, I would rather blit them faster to the
screen, and the only way I can do this is to improve some combination
of processor speed and/or disk speed or some OS tricks. I'll look at
the setting you mentioned above tonight.

While you have mentioned the processor used, it would be
difficult for us to determine what % utilization is seen
during the task. I suggest you monitor the Task Manager to
see if you have plenty of CPU reserve processing power or if
this is a primary bottleneck. Since you reported a doubling
of display rate with the cached images, certainly it has
some reserve ability, but a main memory cache is quite a bit
faster than even the highest-end hard drives.

The problem is then you can't reasonably have enough memory
for about 22GB of files. Yes it would seem the 15K Cheetah
is the fastest drive solution - even faster if RAID0'ed two
of them. You'll have to hunt down the card you want to use.
Ultra320 can't work on 32bit 33MHz PCI - just not enough
bandwidth, you'd have to go with an Ultra160 card.

Next the question is available space and whether that is the
better option (only one 15K Cheetah) or an SATA card and two
WD Raptors RAID0'ed. I suspect the Cheetah would still be
faster but not having these drives in that system for that
use... retrospect is the best way to be sure.

 

[Paul]

There's another ramdrive option, an add-on card onto which
you can install that memory instead of usurping system
memory for the job. For example Gigabyte's i-Ram product
(though there may be newer versions of this today).
http://www.ewiz.com/detail.php?name=MB-RAMDISK&src=Deal

I thought they stopped making those

That card draws power via the PCI slot. So the PCI card connector
is being used as a stand, to hold a PCB in the PC. The actual
data connection is via SATA 1.5Gb/sec cable, to a SATA port on
the motherboard.

Some people on 2CPU.com, tried to use those things in a RAID array.
The idea being, they'd use 12 or 16 of them, use an ARECA raid
controller, and get 2GB/sec of bandwidth and more than 32GB
total storage space. The problem was, in testing small quantities
of them first, that the RAID controller wasn't getting rhe right
kind of info from the emulation of a disk, to work properly.
Apparently, if you want to RAID those cards, some Southbridge
RAID interfaces would work OK, but more expensive and complicated
hardware RAIDs didn't work. Might have had something to do with
each disk not having a custom "serial number" to make the RAID
happy, when parsing reserved sector metadata. I'm not going to
look up the details now, except to say that if you plan on using
a lot of those things, there can be some issues if trying to achieve
the very highest speeds. Maybe if you were just spanning them,
it wouldn't be a problem.

The idea was, the people in question compared more traditional
suppliers of RAMDisk hardware, to the Gigabyte solution, and by
using the Gigabyte, a SATA RAID card, and a whole bunch of them,
they could save a ton of money. Real RAMDisk hardware is priced for
business/server use and you could buy a new car for the prices of
even a small one of those.

The interesting part of all this, would be the patents behind it.
I've seen articles, where small (almost hobbyist) sized outfits
promised to turn out more of these things, and I never see them
for sale. I thought Gigabyte may have stopped making them, because
of patent issues. Good to see you can still get them, for small
experiments.

Paul

 

[dean]

While you have mentioned the processor used, it would be
difficult for us to determine what % utilization is seen
during the task. I suggest you monitor the Task Manager to
see if you have plenty of CPU reserve processing power or if
this is a primary bottleneck. Since you reported a doubling
of display rate with the cached images, certainly it has
some reserve ability, but a main memory cache is quite a bit
faster than even the highest-end hard drives.

The problem is then you can't reasonably have enough memory
for about 22GB of files. Yes it would seem the 15K Cheetah
is the fastest drive solution - even faster if RAID0'ed two
of them. You'll have to hunt down the card you want to use.
Ultra320 can't work on 32bit 33MHz PCI - just not enough
bandwidth, you'd have to go with an Ultra160 card.

Next the question is available space and whether that is the
better option (only one 15K Cheetah) or an SATA card and two
WD Raptors RAID0'ed. I suspect the Cheetah would still be
faster but not having these drives in that system for that
use... retrospect is the best way to be sure.- Hide quoted text -

- Show quoted text -

Yeah you are right - there are only U160 cards for the PCI slots I
have. I have ordered one 15K cheetah and a card.

(Some of this is purely for fun I quite like the idea of the raptor
though - with its clear side, might be interesting to see it working
away.) I don't have sata connectors though, and I am out of slots on
the boards, once I put in the scsi.

 

[Noozer]

Yeah you are right - there are only U160 cards for the PCI slots I

have. I have ordered one 15K cheetah and a card.

(Some of this is purely for fun I quite like the idea of the raptor
though - with its clear side, might be interesting to see it working
away.) I don't have sata connectors though, and I am out of slots on
the boards, once I put in the scsi.

Not sure what software would do this... but why not convert the group of
images together into a movie? Should be something that you can automate at
the end of the day.

Playing through the images would then be a snap.

 

[dean]

Not sure what software would do this... but why not convert the group of
images together into a movie? Should be something that you can automate at
the end of the day.

Playing through the images would then be a snap.

I have tried some programs that knit them together into an AVI file or
mpeg, its painful and slow and produces much larger files than the
original group. With individual files its easy to scroll forward and
back in either direction using the keyboard (at least that's what I
managed to write easily enough) - do you know of any other avi-player
that will allow different play-back speeds, reverse, frame-by-frame,
etc?