Problems filling datatable with large data

  • Thread starter Thread starter Emil Stoev
  • Start date Start date
E

Emil Stoev

Hi,
I have some problems when filling a datatable with lots of data.

Here's the code I use:

OleDbConnection conDBF = null;
conDBF = new OleDbConnection("provider=VFPOLEDB.1 ;data source='" +
DBFDir + "';password='';user id=''");
OleDbDataAdapter daDBFF = new OleDbDataAdapter("Select * FROM " +
DBFFile, conDBF);
DataTable dtDBF = new DataTable();
daDBFF.Fill( dtDBF );

It works perfectly with small data but where the .DBF file has more than
400K records it just hangs.

Any ideas?
 
Yes. Retrieving 400,000 records is not only time consuming, but memory
consuming - you are talking about huge amounts of data here.
Restructure your code so that you don't need all 400K records all at once.
 
Hi Mariana,

Yes, I do know that is resource consuming task. Yet, the application is very
specific and I do need all the records at once.
Please note that we are not talking about slow process or something. The
application totally hangs.

I am testing this on 3.6Ghz machine with 2GB of RAM.
250K records go fine (for less than 10min) and more than 250K just hangs (no
CPU usage, no swapping, nothing).

Is that some .NET framework limitation?
 
Back
Top