serializing dataset with huge data

[MeDhanush]

Hi,

I needed to load a dataset with x number of million records, to avoid
database trips.
To load the dataset with this amount of data, it is taking almost 10
minutes.
We just wanted to see if, we can read it from file in less amount of
time.
So, this requires serialiaztion of existing data into a file. So,
first I needed to load into dataset and serialize data set.
I'm using binary formatter.

It was able to load into dataset, but failed to serialize, as it
throwing OutOfMemory exception.
Virtual memory size has been set to its maximum.

Any ideas please...?

TIA
MeDhanush

 

[Yuancai \(Charlie\) Ye]

Hi,
See the article at http://www.udaparts.com/articles/fastsocketpro.htm

--
Yuancai (Charlie) Ye

Fast and securely accessing all of remote data sources anywhere with
SocketPro using batch/queue, asynchrony and parallel computation with online
compressing

See 30 well-tested and real OLEDB examples

www.udaparts.com

 

[Fritz]

Hi,

I needed to load a dataset with x number of million records, to avoid
database trips.
To load the dataset with this amount of data, it is taking almost 10
minutes.
We just wanted to see if, we can read it from file in less amount of
time.
So, this requires serialiaztion of existing data into a file. So,
first I needed to load into dataset and serialize data set.
I'm using binary formatter.

It was able to load into dataset, but failed to serialize, as it
throwing OutOfMemory exception.
Virtual memory size has been set to its maximum.

Any ideas please...?

TIA
MeDhanush

Sounds more like some sort of circular thing you have in the dataset.