Previously J. Clarke said:Arno Wagner wrote:[...]Previously J. Clarke <[email protected]> wrote:
Not if he's using the disk exclusively for his data and if necessary writing
to several drives. Not unless the OS is generating some unnecessary
overhead. All he has to do is burst his 533k (or whatever the exact
number he gave was) into the buffer and then move on and let the drive
digest that. If the drive hasn't digested that by the next cycle then use
two drives and alternate. If that doesn't allow long enough for the drive
to digest the burst, then three, or four.
Nothing elaborate like RAID, just several drives and alternating writes.
Seagate says that the minimum sustained sequential transfer rate on a 36 gig
X15 is 80 MB/sec, which should be adequate.
PCI-X, which is both 64 bit and 66 MHz or faster. PCI-E is limiting--there
are very few host adapters out for it as yet, one SATA that I know of and
no SCSI.
Previously Peter said:Probably not. These are the maximum rates:
32bit PCI, 33MHz - 133MB/s
CWatters said:30fps = 33mS...
You need to set up some buffers. Could you set up a double buffer?
Compute on one frame while writing the results of the previous frame to disc.
That gives 33mS for writing the data bringing the bandwidth required down
to about 15MBytes/Sec.
J. Clarke said:You may be hitting the real world limit of the PCI bus here. Typically
in network servers the PCI bus becomes a bottleneck somewhere between
250 and 400 Mb/sec--400 works out to 50 MB/sec with no overhead.
That means that no matter how fast your disks are if you're connecting
the host adapter via the PCI bus you're not likely to get above 50 MB/sec.
In your application in principle you might be able to but it's not the
way to bet.
So to start out, I'd try staying with my existing disks and going to a
faster bus and host adapter. While you're about it it you might want
to consider going to an AMD-64 system--some of them can handle 32
gig or more of RAM--with a PCI-X bus and a PCI-X host adapter.
Beyond that, you should be able to burst 513K to any disk on the market in
8ms if the bus will handle it--they all have 2 meg buffers or larger, some
up to 16, and interfaces that can handle 100+ MB/sec into that buffer.
CWatters said:No not "every", he said "in" 8ms. He implies he only has 8ms left in a 30
fps (eg 33ms) cycle. He's capturing data every 33ms, then appears to be
computing it for 25ms. That produces 513K of data that (he says) needs to
be written to disk before the next lot of computation has to start 8ms later.
The long term average is around 15MBytes/s but the way he's doing it the
peak is >65MBytes/sec
which as JC points out is probably hitting the PCI bus limit.
Arno Wagner said:As I (and others) said, RAM and solid-state disks are the only solution
with a filesystem, since all other will have larger access times, at least
occasionally.
Maybe a single 15000RPM SCSI disk used in sequential mode (i.e. no
filesystem, sequential sector writes only) with no other devices on the
SCSI bus will work also.
In both cases 66MHz and or 64 bit PCI, alternatively PCI-E,
would likely need to be used as bus.
J. Clarke said:Not if he's using the disk exclusively for his data and if necessary writing
to several drives. Not unless the OS is generating some unnecessary
overhead.
All he has to do is burst his 533k (or whatever the exact number he gave
was) into the buffer and then move on and let the drive digest that.
If the drive hasn't digested that by the next cycle then use two drives
and alternate. If that doesn't allow long enough for the drive
to digest the burst, then three, or four.
Nothing elaborate like RAID, just several drives and alternating writes.
Seagate says that the minimum sustained sequential transfer rate on a 36 gig
X15 is 80 MB/sec, which should be adequate.
Peter said:Probably not. These are the maximum rates:
32bit PCI, 33MHz - 133MB/s
64bit PCI, 33MHz - 266MB/s
64bit PCI, 66MHz - 532MB/s
64bit PCI-X, 100MHz - 800MB/s
64bit PCI-X, 133MHz - 1GB/s
Yes, those are the _maximum_ rates at which bits move across the PCI bus.
Generally speaking, a bit moving across the PCI bus is going _from_
somewhere, _to_ somewhere, with a stop in RAM on the way. So to move a bit
from one PCI device to another takes two transfers. Then there is
overhead. Then there is everything _else_ that the system is doing that
uses up bandwidth.
So in the _real_ world, not your idealized theoretical one, you get about
30-50 MB/sec of sustained disk I/O on PCI. This became a problem when
gigabit Ethernet became popular because a PCI machine just plain can't fill
a gigabit pipe.
Now, it might be that if he does everything right the structure of his data
and the design of his program may allow a greater percentage of the
theoretical bandwidth of PCI to be utilized in his particular application,
but that's not the way to bet.
Peter said:I have assumed that he dumps data from memory (application) to a disk. Not
from disk to disk, network to disk, etc.
And why does he need to complete disk transfer in 8ms, if there are
1000/7.5 = 133ms between cycles?
He said that his app is
multithreaded...What about OS cache for storage devices? Is he planning to
turn it off?!
Folkert Rienstra said:You'll need dual ported memory.
CWatters said:I didn't mean a hardware buffer.
Oh, what *did* you mean then?
And what has a hardware buffer got to do
with dual ported (ie: dual-channel) memory?
CWatters said:Yeah sorry. I'm old school.... To me "Dual Ported" is different to "Dual
Channel". Dual channel means two memory arrays for twice the bandwidth -
as found on modern motherboards. To me "dual ported" means ONE memory
array with two independant data ports/busses so that two processors can
access it at the same time. eg
http://www.maxim-ic.com/appnotes.cfm/appnote_number/539
Would he need the increased bandwith of dual channel memory system for
15MBytes/S? I'm thinking that if he breakes up the process of writing the
data into smaller blocks spread over 33mS rather than 8mS perhaps he would
be OK?
Run it from a server machine with a RAM drive of 8GB?
I don't know offhand what the largest RAM drive you can have will be - I
used to use something called Ramdisk Pro and I had the option of
formatting the ram disk as NTFS or FAT32 - so I assume you will be able
to use 8GB formatted as NTFSs.
Jukka said:From memory: 4 GB is max with 32-bit and ramdisks which works with
windows I have seen max limit to be 2 GB or even less.
Solid state HD's might be the way if money isn't limited.
CWatters said:Yeah sorry. I'm old school....
To me "Dual Ported" is different to "Dual Channel".
Dual channel means two memory arrays for twice the bandwidth -
as found on modern motherboards.
To me "dual ported" means ONE memory array with two independant
data ports/busses so that two processors can access it at the same time.
eg http://www.maxim-ic.com/appnotes.cfm/appnote_number/539
Would he need the increased bandwith of dual channel memory system
for 15MBytes/S?
I'm thinking that if he breaks up the process of writing the data into
smaller blocks spread over 33mS rather than 8mS perhaps he would be OK?
Here's an off-the-wall notion if the machine doing the calculations has a
PCI-X slot. There are several Compaq Proliant DL590/64 Itanic servers on
ebay at the moment that seem to be going for 400 bucks or so--they'll hold
32-64 gig of directly-addressable RAM and have 8 64/66 slots. So put 16
gig or so of RAM in the thing and a gigabit NIC and maybe just for hohos
spend 60 bucks or so to fill the other two processor slots, put Linux on
it, configure it with most of the memory a RAM disk, and use it for a solid
state disk. If the application will port easily to Linux I'd try compiling
it 64 bit and throw the Itanics at it just for jollies--might be enough
horsepower to run the whole thing on the Proliant, which would simplify
life I suspect.
Al said:You need an OS that can address the RAM that's above 4GB. In the
Windows world that means a version of server.
I don't know what the
state of Linux is for this but I bet someone's got it to work. ISTR
that Intel called it PAE (program address extention)
Windows XP 64-bit edition for Itanium directly addresses 16 GB of RAM. If
you can find a copy. It won't be cheap. The 4 GB limitation is for 32-bit
Windows. 32 bit Windows will not run on an Itanium.
No, that was something in the 32-bit world, a crutch to deal with the 32-bit
address space of the 386 and derivatives. Itanic has 48 address lines for
280 terabytes directly addressable. It needs no crutches to deal with the
64 gig that a DL590 can hold. Linux for ia64 has been available since
2003. It can as far as I know fully utilize the processor.