OutOfMemory Exception but heaps of free memory

  • Thread starter Thread starter harry
  • Start date Start date
H

harry

Hi,

Can one tune the Framework memory management?

I have an app that fills a large table (approx 10 million records - 2 int
columns), however prior to filling the table an exception is thrown claiming
I am out of memory. I've checked the memory usage at this point (using
TaksManager Performance tab) and it varies but it's approx 1.3GB or so.

I have 4GB of memory (4 x 1GB SIM).

Also I've loaded some huge Word files up to 2.1GB to check if there is some
problem with memory however no problem. Besides the pc is new and the memory
is Intel Certified and was tested after the pc was built.

So I guess it's something to do with the way memory is managed by .NET. I
specifically purchased 4GB or memory so that I could load large tables (ok I
know perhaps this is not the most elegant solution - I admit I am being a
little lazy with my code) but now I find .NET can't manage it.

Any suggestions?

Thanks
Harry
 
Hi Harry,

Would you not first have a check in your program, this is almost for ever a
endless loop or maybe filling a table more thems than you was thinking you
did. As far as I know is help in lazyness is not suported by the framework.

Cor
 
A few remarks:

Even though you have 4 GB of RAM, Win NT (or XP) divides the space in 2 GB
of system space and 2 GB of user space by default. So, your process cannot
use more than 2 GB of memory. You have the option to boot in 3 GB mode in
which case the split is 3 GB user / 1 GB system.

Furthermore, the .NET runtime does not let you go up to 2 GB. The garbage
collector works by copying live objects, so it needs a fair amount of space
to perform its copies. For an ASP application, the amount of memory is
controlled by the processModel/memoryLimit setting in your machine.config
file. Microsoft recommends that you set it no higher than 60% and also that
the corresponding memory size does not exceed 800 MB (so, you should set it
at 32% if you have 4 GB of physical mem). You will find more details in
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/scalenetchapt17.asp

So, even though you have 4 GB of physical memory, you are likely to run into
OutOfMemoryException when your process starts to use more than 800 MB of
memory (my experience is that you can usually go up to 1 GB but that you
take a high risk above it). This is rather frustrating but this is life. You
can get a little more air by booting in 3 GB mode but still, you won't be
able to take full advantage of your large memory. The only relief will come
with 64 bit OS.

Bruno.
 
So, even though you have 4 GB of physical memory, you are likely to run
into
OutOfMemoryException when your process starts to use more than 800 MB of
memory (my experience is that you can usually go up to 1 GB but that you
take a high risk above it). This is rather frustrating but this is life.
Click to expand...

Won't the process reset, per the machine.config file, once you hit your 60%
limit or whatever your limit setting is? If so, why does an
OutOfMemoryException occur if this would be the case?

Raymond Lewallen
 
the process should recycle, you are correct. in practice though it doesn't.
my theory is that the asp.net process is getting clobbered and cannot honor
the request fast enough to avoid the exception. that's a theory, it won't
stand up in court.
 
I am loading a s--t load of data into memory. In order to do this I had to
do the following.

1. Buy 4G RAM (somewhat arbitrary amount)
2. Use the /3GB option in Windows
3. Link my program with the LARGEADDRESSAWARE option

Step #3 is key! I wrote the application in VB.NET, which doesn't seem to
allow you to set linker options directly <soapbox>another VB thing which
drives me nuts!</soapbox>. So I use the EditBin utulity to set the option
after my EXE has been compiled. The syntax is editbin /LARGEADDRESSAWARE
myApp.exe.

A little late, but hopefully this will help the next person who comes here
looking for an answer.

Later,
curt


Alvin Bruney said:
the process should recycle, you are correct. in practice though it doesn't.
my theory is that the asp.net process is getting clobbered and cannot honor
the request fast enough to avoid the exception. that's a theory, it won't
stand up in court.

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok
Raymond Lewallen said:
Won't the process reset, per the machine.config file, once you hit your
60%
limit or whatever your limit setting is? If so, why does an
OutOfMemoryException occur if this would be the case?

Raymond Lewallen
Click to expand...
Click to expand...
 
thanks for reporting this fix

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok
Curt Koppang said:
I am loading a s--t load of data into memory. In order to do this I had to
do the following.

1. Buy 4G RAM (somewhat arbitrary amount)
2. Use the /3GB option in Windows
3. Link my program with the LARGEADDRESSAWARE option

Step #3 is key! I wrote the application in VB.NET, which doesn't seem to
allow you to set linker options directly <soapbox>another VB thing which
drives me nuts!</soapbox>. So I use the EditBin utulity to set the option
after my EXE has been compiled. The syntax is editbin /LARGEADDRESSAWARE
myApp.exe.

A little late, but hopefully this will help the next person who comes here
looking for an answer.

Later,
curt


Alvin Bruney said:
the process should recycle, you are correct. in practice though it
doesn't.
my theory is that the asp.net process is getting clobbered and cannot
honor
the request fast enough to avoid the exception. that's a theory, it won't
stand up in court.

--
Regards,
Alvin Bruney
[ASP.NET MVP http://mvp.support.microsoft.com/default.aspx]
Got tidbits? Get it here... http://tinyurl.com/27cok
Raymond Lewallen said:
So, even though you have 4 GB of physical memory, you are likely to
run
into
OutOfMemoryException when your process starts to use more than 800 MB
of
memory (my experience is that you can usually go up to 1 GB but that
you
take a high risk above it). This is rather frustrating but this is
life.

Won't the process reset, per the machine.config file, once you hit your
60%
limit or whatever your limit setting is? If so, why does an
OutOfMemoryException occur if this would be the case?

Raymond Lewallen
Click to expand...
Click to expand...
Click to expand...
 
Back
Top