.NET eats memory like a pig

  • Thread starter Thread starter Mike P
  • Start date Start date
M

Mike P

I know everything about reference counting and making sure you don't have
large objects lying around. I have also profiled my app with multiple tools.
I know about the fact GC collects memory but not necessary give it back to
the OS.



It seems that .NET win app will only return memory to the OS when the OS is
asking for it. But!!! When the OS is asking for it is usually too late, tons
of swapping and slow performance.



I just don't get it, how can .NET a superior technology is eating so much
RAM? I see that some people force their app to minimize or trim their
workspace just to free the memory - that is a bad idea. Other people unload
a application domain to save memory... a little better but still lots of
work just to get some memory back to the OS.



Some people tell me it is the price to pay for a runtime. Sorry, I have used
other runtimes for windows applications and they don't eat so much memory.
For the love of god, AcuCobol runtime does not take so much RAM (not that I
want to code in COBOL now)



At the end, it seems it is like running an OS within an OS. I am frustrated
that a stupid windows application can take 20M for doing nothing, and if you
have an app that does a little more stuff, it can take 40M and you have to
worry about each Dispose and reference so it will not jump to 100M.



and I am not alone with this problem....



http://www.dotnet247.com/247referen...asp.net/pwilson/archive/2004/02/14/73033.aspx
 
Well, from your comments it sounds like you might not fully understand how
memory is managed in .NET. .NET does NOT use reference counters like COM
applications do.

Ultimately, it's not the Framework that is tying up memory, it is your
application. In any programming environment, if you write sloppy code, you
get poor performance. I'd start by examine your code and make sure you are
calling Dispose on any object that uses unmanaged resources, call Close on
any object that is opened, manage any multi-threading you may be doing with
a watchful eye, use the Marshall.ReleaseComObject class to allow the
Framework to release its references to any and ALL COM objects you may be
using, set the ASPCompat property of ASP.NET pages to True when using COM
objects in ASP.NET pages, etc. etc.

In short, before you beat up on the platform, examine what you are doing in
that platform. You haven't told us anything about the code you are
attempting to run. If you do, we can get a feel for how we may be able to
help or point you in the right learning direction.
 
Mike,

Net is not designed for a Commodore 64. Even the current PDA's have more
memory than those.

Don't try to put methods used for computers as the Commodore 64 in this
century.

I have programmed for computers where I had to win a single byte to get what
I wanted, I am lucky that is not necessary anymore and in this century
everything to release unused memory will be done using the managed code by
instance with Net.

I find the method that you only use processing time if it is needed the best
one.

To give a methaphor, I know people who are defragmentating there disk every
hour. If you do that as well, fine however probably not the most productive
one.

Just my opinion.

Cor
 
Thre is no code!

Just write a win form app that connects to Access or SQL server, open the
connection and closes it, you app will take between 15 to 20M

Belive me I have coded my application very carefully and when I talk about
refrence couting I don't mean COM, I mean I make sure there are zero
refrences for an object, so the GC can collect it.
 
Thre is no code!
Just write a win form app that connects to Access or SQL server, open the
connection and closes it, you app will take between 15 to 20M
Click to expand...

Well, I'd call what you've stated above some code. Let's see exactly what
you wrote because if I write code that does what you suggest, it doesn't
come anywhere near the memory usage you are experiencing.
Belive me I have coded my application very carefully and when I talk about
refrence couting I don't mean COM, I mean I make sure there are zero
refrences for an object, so the GC can collect it.
Click to expand...

Although you say that I should "believe you" that you've coded carefully,
you continue to give us no examples of what you are doing. And, since I've
used .NET since early betas (nearly 4 years now) and have not experienced
what you are experiencing but have seen what you describe over and over
again with code that is not written properly, I'm going to have a hard time
just "beliving" that you are doing everything correctly.

It almost sounds like you don't want to solve your problem and would just
prefer to complain about .NET. Let's see your code!
 
What's also interesting is that in the article you provide, your experience
seems to fall into the one described in the first paragraph, which starts
off: "Newbies to .NET often criticize its memory management and its garbage
collector,
but the criticisms are typically based on a lack of understanding and
nothing more."


The rest of the article doesn't seem to apply to what you have experieced.
 
You are right, I am going to provide you the code, most of it is generated
by Visual Stuido, all this program does is open an Oledbconnection and
closes that connection. This program takes 20M of RAM on my machine.

I know, most of this RAM is used by .NET memory managemnt and pooling
objects, I still believe that it is too high. If you minimize the program
and maximize it again, the working size is trimmed to about 4M.

using System;

using System.Drawing;

using System.Collections;

using System.ComponentModel;

using System.Windows.Forms;

using System.Data;

namespace WindowsApplication4

{

/// <summary>

/// Summary description for Form1.

/// </summary>

public class Form1 : System.Windows.Forms.Form

{

private System.Data.OleDb.OleDbDataAdapter oleDbDataAdapter1;

private System.Data.OleDb.OleDbCommand oleDbSelectCommand1;

private System.Data.OleDb.OleDbCommand oleDbInsertCommand1;

private System.Data.OleDb.OleDbCommand oleDbUpdateCommand1;

private System.Data.OleDb.OleDbCommand oleDbDeleteCommand1;

private System.Data.OleDb.OleDbConnection oleDbConnection1;

private System.Windows.Forms.Button button1;

/// <summary>

/// Required designer variable.

/// </summary>

private System.ComponentModel.Container components = null;

public Form1()

{

//

// Required for Windows Form Designer support

//

InitializeComponent();

//

// TODO: Add any constructor code after InitializeComponent call

//

}

/// <summary>

/// Clean up any resources being used.

/// </summary>

protected override void Dispose( bool disposing )

{

if( disposing )

{

if (components != null)

{

components.Dispose();

}

}

base.Dispose( disposing );

}

#region Windows Form Designer generated code

/// <summary>

/// Required method for Designer support - do not modify

/// the contents of this method with the code editor.

/// </summary>

private void InitializeComponent()

{

this.oleDbDataAdapter1 = new System.Data.OleDb.OleDbDataAdapter();

this.oleDbSelectCommand1 = new System.Data.OleDb.OleDbCommand();

this.oleDbInsertCommand1 = new System.Data.OleDb.OleDbCommand();

this.oleDbUpdateCommand1 = new System.Data.OleDb.OleDbCommand();

this.oleDbDeleteCommand1 = new System.Data.OleDb.OleDbCommand();

this.oleDbConnection1 = new System.Data.OleDb.OleDbConnection();

this.button1 = new System.Windows.Forms.Button();

this.SuspendLayout();

//

// oleDbDataAdapter1

//

this.oleDbDataAdapter1.DeleteCommand = this.oleDbDeleteCommand1;

this.oleDbDataAdapter1.InsertCommand = this.oleDbInsertCommand1;

this.oleDbDataAdapter1.SelectCommand = this.oleDbSelectCommand1;

this.oleDbDataAdapter1.TableMappings.AddRange(new
System.Data.Common.DataTableMapping[] {

new System.Data.Common.DataTableMapping("Table", "Generic", new
System.Data.Common.DataColumnMapping[] {

new System.Data.Common.DataColumnMapping("Generic", "Generic"),

new System.Data.Common.DataColumnMapping("ID", "ID"),

new System.Data.Common.DataColumnMapping("NTE", "NTE"),

new System.Data.Common.DataColumnMapping("PinNum", "PinNum"),

new System.Data.Common.DataColumnMapping("Stock", "Stock")})});

this.oleDbDataAdapter1.UpdateCommand = this.oleDbUpdateCommand1;

//

// oleDbSelectCommand1

//

this.oleDbSelectCommand1.CommandText = "SELECT Generic, ID, NTE, PinNum,
Stock FROM Generic";

this.oleDbSelectCommand1.Connection = this.oleDbConnection1;

//

// oleDbInsertCommand1

//

this.oleDbInsertCommand1.CommandText = "INSERT INTO Generic(Generic, NTE,
PinNum, Stock) VALUES (?, ?, ?, ?)";

this.oleDbInsertCommand1.Connection = this.oleDbConnection1;

this.oleDbInsertCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Generic",
System.Data.OleDb.OleDbType.VarWChar, 50, "Generic"));

this.oleDbInsertCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("NTE",
System.Data.OleDb.OleDbType.VarWChar, 50, "NTE"));

this.oleDbInsertCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("PinNum",
System.Data.OleDb.OleDbType.VarWChar, 50, "PinNum"));

this.oleDbInsertCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Stock",
System.Data.OleDb.OleDbType.VarWChar, 50, "Stock"));

//

// oleDbUpdateCommand1

//

this.oleDbUpdateCommand1.CommandText = @"UPDATE Generic SET Generic = ?, NTE
= ?, PinNum = ?, Stock = ? WHERE (ID = ?) AND (Generic = ? OR ? IS NULL AND
Generic IS NULL) AND (NTE = ? OR ? IS NULL AND NTE IS NULL) AND (PinNum = ?
OR ? IS NULL AND PinNum IS NULL) AND (Stock = ? OR ? IS NULL AND Stock IS
NULL)";

this.oleDbUpdateCommand1.Connection = this.oleDbConnection1;

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Generic",
System.Data.OleDb.OleDbType.VarWChar, 50, "Generic"));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("NTE",
System.Data.OleDb.OleDbType.VarWChar, 50, "NTE"));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("PinNum",
System.Data.OleDb.OleDbType.VarWChar, 50, "PinNum"));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Stock",
System.Data.OleDb.OleDbType.VarWChar, 50, "Stock"));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_ID",
System.Data.OleDb.OleDbType.Integer, 0,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "ID", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Generic",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Generic", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Generic1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Generic", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_NTE",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "NTE", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_NTE1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "NTE", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_PinNum",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "PinNum", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_PinNum1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "PinNum", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Stock",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Stock", System.Data.DataRowVersion.Original, null));

this.oleDbUpdateCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Stock1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Stock", System.Data.DataRowVersion.Original, null));

//

// oleDbDeleteCommand1

//

this.oleDbDeleteCommand1.CommandText = "DELETE FROM Generic WHERE (ID = ?)
AND (Generic = ? OR ? IS NULL AND Generic IS N" +

"ULL) AND (NTE = ? OR ? IS NULL AND NTE IS NULL) AND (PinNum = ? OR ? IS
NULL AND" +

" PinNum IS NULL) AND (Stock = ? OR ? IS NULL AND Stock IS NULL)";

this.oleDbDeleteCommand1.Connection = this.oleDbConnection1;

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_ID",
System.Data.OleDb.OleDbType.Integer, 0,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "ID", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Generic",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Generic", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Generic1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Generic", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_NTE",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "NTE", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_NTE1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "NTE", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_PinNum",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "PinNum", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_PinNum1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "PinNum", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Stock",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Stock", System.Data.DataRowVersion.Original, null));

this.oleDbDeleteCommand1.Parameters.Add(new
System.Data.OleDb.OleDbParameter("Original_Stock1",
System.Data.OleDb.OleDbType.VarWChar, 50,
System.Data.ParameterDirection.Input, false, ((System.Byte)(0)),
((System.Byte)(0)), "Stock", System.Data.DataRowVersion.Original, null));

//

// oleDbConnection1

//

this.oleDbConnection1.ConnectionString = @"Jet OLEDB:Global Partial Bulk
Ops=2;Jet OLEDB:Registry Path=;Jet OLEDB:Database Locking Mode=1;Jet
OLEDB:Database Password=;Data
Source=""C:\Database\ApplicationCore.mdb"";Password=;Jet OLEDB:Engine
Type=5;Jet OLEDB:Global Bulk
Transactions=1;Provider=""Microsoft.Jet.OLEDB.4.0"";Jet OLEDB:System
database=;Jet OLEDB:SFP=False;Extended Properties=;Mode=Share Deny None;Jet
OLEDB:New Database Password=;Jet OLEDB:Create System Database=False;Jet
OLEDB:Don't Copy Locale on Compact=False;Jet OLEDB:Compact Without Replica
Repair=False;User ID=Admin;Jet OLEDB:Encrypt Database=False";

//

// button1

//

this.button1.Location = new System.Drawing.Point(96, 48);

this.button1.Name = "button1";

this.button1.TabIndex = 0;

this.button1.Text = "button1";

this.button1.Click += new System.EventHandler(this.button1_Click);

//

// Form1

//

this.AutoScaleBaseSize = new System.Drawing.Size(5, 13);

this.ClientSize = new System.Drawing.Size(292, 266);

this.Controls.Add(this.button1);

this.Name = "Form1";

this.Text = "Form1";

this.ResumeLayout(false);

}

#endregion

/// <summary>

/// The main entry point for the application.

/// </summary>

[STAThread]

static void Main()

{

Application.Run(new Form1());

}

private void oleDbDataAdapter1_RowUpdated(object sender,
System.Data.OleDb.OleDbRowUpdatedEventArgs e)

{


}

private void button1_Click(object sender, System.EventArgs e)

{

oleDbConnection1.Open();

oleDbConnection1.Close();

}

}

}
 
You don't understand me at all. I love C# and .NET, in fact I will probably
work in this environment in many years to come.



My frustration is that memory management in .NET is not well documented. I
find that the GC is well documented, but it is clear that what the GC is
freeing goes back to the "manage heap" and not the OS.



I am not criticising the GC at all, I think it does a good job, I am
criticising the lack of documentation or functionality when it comes to
Process memory in regards the OS (not the runtime or the managed heap).



I want to know when memory is returned to the OS, I find that when the
system needs the memory it is simply "too late".



Maybe, this is not even a .NET issue and should be regarded as an OS issue.
 
Inline

Willy.

Mike P said:
You don't understand me at all. I love C# and .NET, in fact I will
probably
work in this environment in many years to come.



My frustration is that memory management in .NET is not well documented.
Click to expand...

There is no such thing as Memory management in .NET, Memory management is a
task of the OS, the OS knows nothing about .NET and there is no such thing
like a .NET process, so it's not documented as part of .NET. (see later).
find that the GC is well documented, but it is clear that what the GC is
freeing goes back to the "manage heap" and not the OS.
Click to expand...
Basically, that's the task of the GC, manage the object allocation and
de-allocation from the GC heap, no more no less. A .NET process is just
another windows process, it has private heaps allocated from the process
heap, and one of them is called the GC heap, but basically it's in no way
different from say the (ref. counting GC collected heap of a VB6 process
(yes VB6 has also a GC). Oh, I hear you say 'but it's size is much larger',
sure it is, so is the size taken by Windows XP the OS and Explorer and do
you know why? because memory size higly relates to features like growing
disk size, ever wondered why explorer takes > 40 MB when you open and start
navigating a 200GB drive?
The same is true for .NET, it comes at a price, the price is that the memory
footprint is somewhat (5-6MB) higher than a VB6 or C++ or any other "native"
application, but heck this is only the footprint anything else doen't take
much more memory than their native counterparts, the difference is that
no-one ever cared to look at Taskman to watch it's "memory consumption",
but now they know that there is a GC that is playing with 'their' toys, they
spend more time looking at 'Mem used' than trying to understand the features
of the languages, the framework library and the richness of the API's, what
can be done with it and what can't or shouldn't be done with it.
I am not criticising the GC at all, I think it does a good job, I am
criticising the lack of documentation or functionality when it comes to
Process memory in regards the OS (not the runtime or the managed heap).
Click to expand...
Process memory allocation and memory management for Windows is well
documented in the Platform SDK documentation. There are a number of good
books arround like Mark Russinovich and Dave Solomon's "Windows Internals"
who tell you more than you ever need to know about it, there are a number of
real good tools arround to help you understand what memory consumption is
all about, but again, only if you realy care, it's not that important. Only,
if you start arguing about it and draw conclusions, be prepared to know what
you are talking about, know how you measure things yourself, don't base your
findings alone on what you read on the internet.

I want to know when memory is returned to the OS, I find that when the
system needs the memory it is simply "too late".
Click to expand...
What you find is not important, it's the OS memory manager who knows better
than anyone else when memory should be returned and more importantly what
and who's memory should be returned.
Those who are trying to be smart and call GC.Collect and call the Win32 API
to trim the Working Set don't have an idea how they are disturbing the GC
and the OS Memory manager by doing this, all they see is the "Used memory"
going down in Taskman and they are happy and they can (thank God freedom of
speech) blog about their findings and tell the world how smart they are and
how dumb the MSFT developpers are.

Maybe, this is not even a .NET issue and should be regarded as an OS
issue.
Click to expand...

Well it's none of both, it's simply not an issue, really.
I don't mean .NET is perfect, it's NOT perfect at all, there are other, much
more important issues to be solved, the upcoming version 2.0, takes care of
a lot of them, but as far as memory goes it reduces the base working set by
a fair amounth , but at the same time new features (requested by the
community) are added, which take more memory so the net result is about the
same.

Willy.
 
Well said Willy. And, by the way, much of what you've said here is plainly
stated in the article the OP himself included.
 
Back
Top