Out-of-memory error: sample app

  • Thread starter Thread starter George1776
  • Start date Start date
G

George1776

I'm seeing some unexpected behavior in Framework 2.0 / IIS 5.1 in high
memory usage situations. The first problem I had is that the default
machine.config for Framework 2.0 does not include a memoryLimit entry -
Framework 1.0 was set to 60% by default. But even after adding a
memoryLimit = 60 to my processModel section all available memory still
gets used up and the process dies - the same app under Framework 1.1
returns the requested rows - no matter how many I request.

The other problem I've seen is that after the 2.0 thread dies or is
killed if I return to that page - without clicking the button to
request datarows - memory begins to skyrocket again immediately. It
seems like ASP.NET is somehow remembering what it was doing when it
died and wants to pick up and do it again - which of course leads to
another crash.

Note: I know it's poor practice to load so much data into a grid. I'm
dealing with a huge production legacy application here and we are in
the process of refactoring. In order to take advantage of some third
party tools we upgraded to framework 2.0. At this point we started
having memory problems with some admittedly poorly designed pages that
allow users to pull ridiculous amounts of data. So if your only answer
is "don't pull so much data" well . . . thanks, no kidding.

Here's my sample app - basically it creates a huge datatable and tries
to load it into a grid:

BigTable.aspx:

<%@ Page language="c#" Codebehind="BigTable.aspx.cs"
AutoEventWireup="false" Inherits="BigTable.BigTable" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<title>BigTable</title>
<meta name="GENERATOR" Content="Microsoft Visual Studio .NET 7.1">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name="vs_defaultClientScript" content="JavaScript">
<meta name="vs_targetSchema"
content="http://schemas.microsoft.com/intellisense/ie5">
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="Form1" method="post" runat="server">
<asp:DataGrid id="grdRows" style="Z-INDEX: 101; LEFT: 1px; POSITION:
absolute; TOP: 1px" runat="server"
Width="792px"></asp:DataGrid>
&nbsp;
<asp:Label id="lblRowsToLoad" style="Z-INDEX: 104; LEFT: 8px;
POSITION: absolute; TOP: 152px"
runat="server" Width="144px">Rows To Load</asp:Label>
<asp:TextBox id="txtRowsToLoad" style="Z-INDEX: 103; LEFT: 8px;
POSITION: absolute; TOP: 176px"
runat="server" Width="80px">5</asp:TextBox>
<asp:Button id="btnLoadRows" style="Z-INDEX: 102; LEFT: 8px;
POSITION: absolute; TOP: 200px"
runat="server" Text="Load Rows"
OnClick="btnLoadRows_Click"></asp:Button>
&nbsp;
</form>
</body>
</HTML>




BigTable.aspx.cs:


using System;
using System.Data;

namespace BigTable
{
public partial class BigTable : System.Web.UI.Page
{
protected System.Web.UI.WebControls.DataGrid grdRows;
protected System.Web.UI.WebControls.Button btnLoadRows;
protected System.Web.UI.WebControls.TextBox txtRowsToLoad;
protected System.Web.UI.WebControls.Label lblRowsToLoad;

protected void btnLoadRows_Click(object sender,
System.EventArgs e)
{
System.Data.DataTable dt = new DataTable();
for (int x = 0; x < 10; x++) //10 columns
{
dt.Columns.Add(x.ToString(), typeof(String));
}
System.Data.DataRow dr;
int yTotal = int.Parse(txtRowsToLoad.Text);
for (int y = 0; y < yTotal; y++) //Add the requested number
of rows.
{
dr = dt.NewRow();
for (int x = 0; x < 10; x++) //insert a string value
into each column of each row.
{
dr[x] = x.ToString();
}
dt.Rows.Add(dr);
}
grdRows.DataSource = dt;
grdRows.DataBind();
}
}
}





BigTable.aspx.designer.cs:

//------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by a tool.
// Runtime Version:2.0.50727.42
//
// Changes to this file may cause incorrect behavior and will be
lost if
// the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

namespace BigTable {

public partial class BigTable {
protected System.Web.UI.HtmlControls.HtmlForm Form1;
}
}
 
See responses inline:
The other problem I've seen is that after the 2.0 thread dies or is
killed if I return to that page - without clicking the button to
request datarows - memory begins to skyrocket again immediately. It
seems like ASP.NET is somehow remembering what it was doing when it
died and wants to pick up and do it again - which of course leads to
another crash.
Click to expand...

Sounds like it's taking the ViewState of the textbox and re-submitting
which is of course, going to ask for the same data again, and thus,
crash your app again.

Does this happen when you CLOSE the web browser and re-open?
Note: I know it's poor practice to load so much data into a grid.
Click to expand...

Yes, Indeed it is. There is no way you can limit the amount of data to
be populated?

How much data are we actually talking about here? 50 rows? 500 rows?
5000 rows? 500,000 rows?

I'm
dealing with a huge production legacy application here and we are in
the process of refactoring. In order to take advantage of some third
party tools we upgraded to framework 2.0. At this point we started
having memory problems with some admittedly poorly designed pages that
allow users to pull ridiculous amounts of data. So if your only answer
is "don't pull so much data" well . . . thanks, no kidding.
Click to expand...

Don't pull so much data. =)
Here's my sample app - basically it creates a huge datatable and tries
to load it into a grid:

BigTable.aspx:

<%@ Page language="c#" Codebehind="BigTable.aspx.cs"
AutoEventWireup="false" Inherits="BigTable.BigTable" %>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
<HTML>
<HEAD>
<title>BigTable</title>
<meta name="GENERATOR" Content="Microsoft Visual Studio .NET 7.1">
<meta name="CODE_LANGUAGE" Content="C#">
<meta name="vs_defaultClientScript" content="JavaScript">
<meta name="vs_targetSchema"
content="http://schemas.microsoft.com/intellisense/ie5">
</HEAD>
<body MS_POSITIONING="GridLayout">
<form id="Form1" method="post" runat="server">
<asp:DataGrid id="grdRows" style="Z-INDEX: 101; LEFT: 1px; POSITION:
absolute; TOP: 1px" runat="server"
Width="792px"></asp:DataGrid>
&nbsp;
<asp:Label id="lblRowsToLoad" style="Z-INDEX: 104; LEFT: 8px;
POSITION: absolute; TOP: 152px"
runat="server" Width="144px">Rows To Load</asp:Label>
<asp:TextBox id="txtRowsToLoad" style="Z-INDEX: 103; LEFT: 8px;
POSITION: absolute; TOP: 176px"
runat="server" Width="80px">5</asp:TextBox>
<asp:Button id="btnLoadRows" style="Z-INDEX: 102; LEFT: 8px;
POSITION: absolute; TOP: 200px"
runat="server" Text="Load Rows"
OnClick="btnLoadRows_Click"></asp:Button>
&nbsp;
</form>
</body>
</HTML>




BigTable.aspx.cs:


using System;
using System.Data;

namespace BigTable
{
public partial class BigTable : System.Web.UI.Page
{
protected System.Web.UI.WebControls.DataGrid grdRows;
protected System.Web.UI.WebControls.Button btnLoadRows;
protected System.Web.UI.WebControls.TextBox txtRowsToLoad;
protected System.Web.UI.WebControls.Label lblRowsToLoad;

protected void btnLoadRows_Click(object sender,
System.EventArgs e)
{
System.Data.DataTable dt = new DataTable();
for (int x = 0; x < 10; x++) //10 columns
{
dt.Columns.Add(x.ToString(), typeof(String));
}
System.Data.DataRow dr;
int yTotal = int.Parse(txtRowsToLoad.Text);
for (int y = 0; y < yTotal; y++) //Add the requested number
of rows.
{
dr = dt.NewRow();
for (int x = 0; x < 10; x++) //insert a string value
into each column of each row.
{
dr[x] = x.ToString();
}
dt.Rows.Add(dr);
}
grdRows.DataSource = dt;
grdRows.DataBind();
}
}
}
Click to expand...

Can we see any "actual" code? This seems to just be an example of
building a DataTable.

BTW, your Web frontend really "shouldn't" be creating your data model
on the fly. Simply from a maintainability stand-point. I always defer
these tasks to another tier/assembly. Too much messiness in the UI =)

Hope this might help!

Sean
 
Can we see any "actual" code? This seems to just be an example of
building a DataTable.
Click to expand...

The idea is to compile and run this same little app under 1.1 and 2.0
and observe the differences when requesting something outrageous like
500,000 rows. Under 1.1 if you stick around long enough you'll
eventually get the rows - and even better it tends to keep the CPU
running at about 50% so that other sessions aren't completely choked.
Under 2.0 memory seems to zoom up and down, the CPU remains red-lined
for long periods of time, and eventually the process crashes.
 
well,

IMO, there is a couple of things here that is not complete.

1.) you mentioned your running this in IIS 5.1, how does this perform
when you test it under IIS 6.0? same results?

2.) Have you ran a profiler or tested this against an ACT project?
perhaps then you can pinpoint exactly where the problem lay, if it is
in memory allocation, CPU reads, etc...

3.) What type of machine are you running this on? How much memory in
the box? RAID drives? What server version? 2000 or 2003? or is it just
XP?

There has been a number of times that when I run something on my dev
machine, it is slow as molasses, but as soon as I plop it on a 2003
server running IIS 6 with 2 gigs of ram, dual cpu's and a beefy RAID
array, that slowness is negligable for what I am doing in code.

I guess my final point here is, when all you have is a hammer,
everything looks like a nail =)

Can you provide some of the details above?

Thanks

Sean
 
Back
Top