Basic data processing question

[Guest]

I am in the process of writing the middle tier for an application. This application will grab records, runn them through a process and save them in a table. In going through this process I'll have a staging table from where I grab the records before they are analized and saved into the history table. There is no telling how many records there will be in the staging table for processing, but potentially it could be in the thousands (or at least I would like to be prepared for that many records from a performance stand point)

My question is performance wise, do I use arrays to load all the records for processing or do I just parse through a recordset? My guess would be that arrays are much faster than recordsets because you have the database connectivity issue, but I still have the issue of loading the working array from the staging table.

 

[William Ryan eMVP]

Danny:

Why not just use a Dataset with multiple datatables?

I am in the process of writing the middle tier for an application. This

application will grab records, runn them through a process and save them in
a table. In going through this process I'll have a staging table from where
I grab the records before they are analized and saved into the history
table. There is no telling how many records there will be in the staging
table for processing, but potentially it could be in the thousands (or at
least I would like to be prepared for that many records from a performance
stand point).

My question is performance wise, do I use arrays to load all the records

for processing or do I just parse through a recordset? My guess would be
that arrays are much faster than recordsets because you have the database
connectivity issue, but I still have the issue of loading the working array
from the staging table.

 

[Guest]

Would that be faster than arrays? I guess I have the idea (maybe erroneous) that recordsets are more resource intensive than arrays that are already disconnected from the database.