Out of Memory Exception

  • Thread starter Thread starter Guest
  • Start date Start date
G

Guest

----Begin : Background description----
My Application reads flat file and based on the record number in the flat
file, it populates data to the on-fly Datatables of 23 different types. The
finally output expected is XML.
The Application runs on 2GB RAM server. The input flat file size can be upto
800MB.
----End : Background description----

----Begin : Technical description----
Visual Studio 2003, .Net 1.1
Console application in C#
----End : Technical description----

----Begin : Problem description----
When the input file is of size 500MB or more, the application started
throwing Out of Memory exception.

Any idea to resolve this problem guys?
----End : Problem description----

Thanks in advance

With regards
Venkat
 
Hi,
Even I too agree. But in this case each table records are related to each
other.

For eg. Table 7, 8, 9 10, 11, 12 and 13 had parent child relation so as
other tabless too have such relation.... In this case to do?

Thanks

With regards
Venkat

Miha Markic said:
Hi,

You should read file line by line (or a piece by piece) and write xml
directly using XmlWriter perhaps.
Sou don't keep all those MBs in memory.

--
Miha Markic [MVP C#]
RightHand .NET consulting & development www.rthand.com
Blog: http://cs.rthand.com/blogs/blog_with_righthand/

So y
Venkatachalam said:
----Begin : Background description----
My Application reads flat file and based on the record number in the flat
file, it populates data to the on-fly Datatables of 23 different types.
The
finally output expected is XML.
The Application runs on 2GB RAM server. The input flat file size can be
upto
800MB.
----End : Background description----

----Begin : Technical description----
Visual Studio 2003, .Net 1.1
Console application in C#
----End : Technical description----

----Begin : Problem description----
When the input file is of size 500MB or more, the application started
throwing Out of Memory exception.

Any idea to resolve this problem guys?
----End : Problem description----

Thanks in advance

With regards
Venkat
Click to expand...
Click to expand...
 
Hi,

Venkatachalam said:
Hi,
Even I too agree. But in this case each table records are related to
each
other.

For eg. Table 7, 8, 9 10, 11, 12 and 13 had parent child relation so as
other tabless too have such relation.... In this case to do?
Click to expand...

I would simulate a database - go forth and back through file to load just
necessary records.
Perhaps you might create an index of records in text file (you would index
id and their position in file).
You might also create more complex solutions - depends on the structure of
text file and the way you want to do it.
 
Back
Top