Dataset performance

  • Thread starter Thread starter Alberto Ortega
  • Start date Start date
A

Alberto Ortega

hi, i´ve got this situation:
* One dataset with 5 tables
* One of those tables has almost 140.000 records
* No more than 5000 records on the other 4 tables

Then I´ve got to do calculations over those 140000 records and it´s really
slow. For example the first thing I do is to filter that table with so many
records:

foreach( DataRow dr in ds.TableWithManyData.Select( FILTER )){
...
}

I need to know if there is any way to do this for better performance

Thanks

Maxi Menasches
 
When you work with mass amounts of data, you are better to calculate, if
possible, on the database server rather than pull the data to your
application. You will get the greatest performance if you do that.


---

Gregory A. Beamer
MVP; MCP: +I, SE, SD, DBA

***************************
Think Outside the Box!
***************************
 
Alberto,

I was curious so I tried it with a simple test.

I think that this will speed up very much

DataRow[] dtC = ds.TableWithManyData.Select(Filter);
foreach (DataRow dr in dtC){
.....
}

I hope this helps?

Cor
 
Cor, Is there a difference between this:

foreach( DataRow dr in ds.TableWithManyData.Select( FILTER )){
...
}

and this

DataRow[] dtC = ds.TableWithManyData.Select(Filter);
foreach (DataRow dr in dtC){
.....
}

?

In reply to Gregory`s post it is not possible to do the calculation on the
Database Server, it is a requirement for us to consume the least Database
Server resources as possible, that`s why we only retreive "raw" data to fill
the DataSet.
That`s why we are looking for a performance improvement for the DataSet
calculation.

Thanks

Alberto
 
Alberto,

That was what came from my test. Maybe you can test it yourself as well.

Cor
 
Sahil,

Good answer.

(Although XML seems for me on that a nice way, I have a lot seen in this
newsgroups who did it that way).

:-)

Cor
 
thanks a lot for your answers but dont forget the scenario, in this case is
extremely important not to process the data on the SQL...

Alberto Ortega
 
Back
Top