advice on getting good speed

[Neil Jones]

I am considering embarking on a project that requires considerable computing
speed.

I am looking to build a good spec machine ( or machines) to handle the task.
I have taken appart and assembled machines before but not for some time.

The critical application makes very very little use of the disk drives and
appears to hold its data in ram while it works on it. ( This makes sense as
it is needs to be as fast as possible to work.)
It does not appear that the application is multithreading so dual core
procesors. I believe are of little help.


There are a plethora of different processors with numbers unreleated to
their speed.

I need to work out a combination of motherboard ram and processor that gives
me reasonable speed without excessive cost.

State of the art may not be the answer since the task can be divided up and
two cheaper machines may give more power than one more expensive model.

Can anyone provide me with information or direct me to a resource that can
help me?

Neil Jones
(e-mail address removed)

 

[Clint]

Can you define "reasonable speed" and "excessive cost"? Can you be more
specific about the type of task? Some processors are better different types
of tasks then others.

Just based on what you're posting, I'd say something like an Intel 5xx or
6xx processor, or an AMD Athlon 64 3700+ processor (1MB cache). Locally,
those will run around $150 to $250 (Canadian). Find an Asus, Gigabyte, or
other relatively large name-brand motherboard, and DDR or DDR2 memory to go
along with it, and you'd have your base.

I've used www.anandtech.com for information, and www.tomshardware.com. Both
of them offer motherboard, memory, and processor reviews.

Clint

 

[Dave]

I am considering embarking on a project that requires considerable
computing
speed.

I am looking to build a good spec machine ( or machines) to handle the
task.
I have taken appart and assembled machines before but not for some time.

The critical application makes very very little use of the disk drives and
appears to hold its data in ram while it works on it. ( This makes sense
as
it is needs to be as fast as possible to work.)
It does not appear that the application is multithreading so dual core
procesors. I believe are of little help.


There are a plethora of different processors with numbers unreleated to
their speed.

I need to work out a combination of motherboard ram and processor that
gives
me reasonable speed without excessive cost.

State of the art may not be the answer since the task can be divided up
and
two cheaper machines may give more power than one more expensive model.

Can anyone provide me with information or direct me to a resource that can
help me?

OK, first, you need to be aware that dual core and hyperthreading are
different technologies. Essentially, dual core is two processors in the
same physical package. If a task is not multithreading, dual core can still
help it run better. The reason is simple. A dual core processor means that
there is more than one processor to share the load of handling OTHER
applications that are running at the same time.

Not knowing more about what you are wanting to do, it's hard to guess at
what hardware you might need. But unless you intend to run Windows Vista
while doing it, I think you'd get buy OK on any low-end dual-core processor
that is not crippled (like sempron or celeron, if they even make dual-core
versions of that), plus 1GB of really high-quality name-brand RAM. Find a
good name-brand mainboard with a decent chipset to go with it. Don't skimp
on the power supply, try seasonic or enermax.

Then build 2 or 3 of them. That's the way I'd approach it. If you want to
run Vista though, that is a whole other can of worms. -Dave

 

[Neil Jones]

OK, first, you need to be aware that dual core and hyperthreading are
different technologies. Essentially, dual core is two processors in the
same physical package. If a task is not multithreading, dual core can
still
help it run better. The reason is simple. A dual core processor means
that there is more than one processor to share the load of handling OTHER
applications that are running at the same time.

Not knowing more about what you are wanting to do, it's hard to guess at
what hardware you might need. But unless you intend to run Windows Vista
while doing it, I think you'd get buy OK on any low-end dual-core
processor that is not crippled (like sempron or celeron, if they even make
dual-core
versions of that), plus 1GB of really high-quality name-brand RAM. Find a
good name-brand mainboard with a decent chipset to go with it. Don't
skimp on the power supply, try seasonic or enermax.

Then build 2 or 3 of them. That's the way I'd approach it. If you want
to
run Vista though, that is a whole other can of worms. -Dave

Thanks to both of you who have replied so far.
I think a few details might help. Essentially this is a number crunching
operation that performs statistical work on text files.
I don't think that having a spare processor around on the chip will help
since the machines will be totally dedicated to the task. I don't intend to
be running anything else that will be getting in the way.

There are the following restrictions placed on me.
I cannot process all the data at once. It is divided into sections.Currently
I have experimented with 2 Mb chunks and smaller.
My experimentation so far shows that it will take several hundred hours to
complete the processing on a 800 MHZ machine. The obvious conclusion is to
get a faster machine and more than one if possible. The machines will be
dedicated to that task but will need supervision to load and save data
after each chunk is run.
Currently I have experimented with a 800MZ machine and a 1.27 Ghz machine
the later gives a 1.6-1.8 times speed increase.

My question really is how I can build a set up that does this as fast as
possible.

THe question of cost versus speed can be expressed as follows.

There are two types of machine

Type A - state of the art machine as fast as possible but expensive.

Type B - A slower machine but much cheaper.


Since I am interested in getting the job done in the minimum time. Building
more Type B's may be more cost effective at getting the job done.
Particularly considering the premiums on state of the art technology.

The problem for me is knowing the figures that I need to know in order to
work this out.

I will not be using Vista. One of my test machines was actually running
Windows 98.

Neil Jones
(e-mail address removed)

 

[Clint]

I suspect you don't need to go "state of the art", although the price on the
new Conroe processors looks pretty tempting for the performance. Even the
low-end ones (like the E6300 with an estimated price < $250) look like they
can beat more expensive current generation processors
(http://www.xbitlabs.com/articles/cpu/display/core2duo-e6300.html).

What you're doing sounds similar to the SETI@home project
(http://setiathome.berkeley.edu/index.php) or the "World Community Grid"
(http://en.wikipedia.org/wiki/World_Community_Grid). These applications
seem to work fine on a dual core (or multi-processor) machines and utilize
the extra processing available. Take an application like Prime95, as a
simplest example. It's got a command line switch to set the affinity of the
instance running, meaning on my Intel D 830 processor, I can have two
threads running, both utilizing 100% of a core. While this may not be
running at 200% of an equivalent speed Intel 630 processor, it's a heck of a
lot better, I suspect. You can check out the "Interactive CPU chart" at the
Tom's Hardware site
(http://www.tomshardware.com/2006/04/05/spring_2006_interactive_cpu_charts_update/)
for some data.

The advantage to going to a dual core system in your case would be that the
cost of the processor is only incrementally more expensive (Intel D 930
processor is $220 locally, compared to a Intel 630 processor at $210), but
you don't have to buy two boxes (motherboards, memory, hard drives, cases,
etc), and it only takes up half as much physical space. Alternatively, you
could set up two units, put in dual core processors in both, and have 4
cores churning away.

Of course, if this is a custom application, you may have fun getting the
multi-threaded code to work. But there should be enough examples out there,
if you look hard. You can even download the Prime95 source code, which may
help. And if it's not a custom application, and the original developer
hasn't or isn't building multi-threading capabilities into it, you may not
have an option.

Clint

 

[John Weiss]

I think a few details might help. Essentially this is a number crunching
operation that performs statistical work on text files.
I don't think that having a spare processor around on the chip will help
since the machines will be totally dedicated to the task. I don't intend to
be running anything else that will be getting in the way.

A second core may help take over OS overhead. Also, if you can run 2 instances
of your application on a single machine, a dual-core at a slightly lower clock
speed will run more data than a single-core. So, you need to price out the
options of an older but fast single-core vs a newer dual-core. Note that 2
machines ==> 2 HDs, 2 MoBos, 2 power supplies, 2 cases...

A relatively new Socket 939 MoBo and an AMD Athlon X2 3800+ may be the best
overall investment. When done with your project, you still have a viable
computer for anything else.

There are the following restrictions placed on me.
I cannot process all the data at once. It is divided into sections.Currently
I have experimented with 2 Mb chunks and smaller.

Unless there is significant generated data that also remains in RAM, you should
have no problem whatsoever running 2 x 2MB data sets on 1 or 2 GB of RAM. Note
also that OS overhead on 1 dual-core machine is half that of 2 x single-core
machines...

THe question of cost versus speed can be expressed as follows.
Type A - state of the art machine as fast as possible but expensive.
Type B - A slower machine but much cheaper.

Since I am interested in getting the job done in the minimum time. Building
more Type B's may be more cost effective at getting the job done.
Particularly considering the premiums on state of the art technology.

"Several hunderd hours" of data crunching could mean a week or 2. Agonizing
over buy decisions can cost as much or more!

The problem for me is knowing the figures that I need to know in order to
work this out.

Figure out your costs for representative machines. Express productivity as a
function of clock speed (x number of cores, if applicable). Cost : productivity
ratio should be minimized. Simplest is $ / [clockspeed x numberofcores].

I will not be using Vista. One of my test machines was actually running
Windows 98.

I don't know if Win98 will take advantage of dual cores; I tend to doubt it.
Find a copy of Win2000 if you don't want to pay for XP...

 

[Neil Jones]

A second core may help take over OS overhead. Also, if you can run 2
instances of your application on a single machine, a dual-core at a
slightly lower clock
speed will run more data than a single-core.

Now that had not occured to me. I will run some tests to confirm that it is
possible. I think so, but will need to confirm I can run multiple instances
of the program.


So, you need to price out

the
options of an older but fast single-core vs a newer dual-core. Note that
2 machines ==> 2 HDs, 2 MoBos, 2 power supplies, 2 cases...

A relatively new Socket 939 MoBo and an AMD Athlon X2 3800+ may be the
best
overall investment. When done with your project, you still have a viable
computer for anything else.

There are the following restrictions placed on me.
I cannot process all the data at once. It is divided into
sections.Currently I have experimented with 2 Mb chunks and smaller.

Unless there is significant generated data that also remains in RAM, you
should
have no problem whatsoever running 2 x 2MB data sets on 1 or 2 GB of RAM.
Note also that OS overhead on 1 dual-core machine is half that of 2 x
single-core machines...

THe question of cost versus speed can be expressed as follows.
Type A - state of the art machine as fast as possible but expensive.
Type B - A slower machine but much cheaper.

Since I am interested in getting the job done in the minimum time.
Building more Type B's may be more cost effective at getting the job
done. Particularly considering the premiums on state of the art
technology.

"Several hunderd hours" of data crunching could mean a week or 2.
Agonizing over buy decisions can cost as much or more!

The problem for me is knowing the figures that I need to know in order to
work this out.

Figure out your costs for representative machines. Express productivity
as a
function of clock speed (x number of cores, if applicable). Cost :
productivity
ratio should be minimized. Simplest is $ / [clockspeed x numberofcores].

Yes I was working with these kind of figures, I just didn't have all the
info. Your suggestion about the dual cores makes more sense. Thanks.

I don't know if Win98 will take advantage of dual cores; I tend to doubt
it. Find a copy of Win2000 if you don't want to pay for XP...

Win98 wont. The others will. I have done some searches to confirm this.

Neil Jones
(e-mail address removed)