Memory problem

  • Thread starter Thread starter S.Berwanger
  • Start date Start date
S

S.Berwanger

Hallo all,

I have a problem with the memory management of the compact framework.
Without a obvious reason the VB.NET program uses more and more memory which
ends with a system stop because there is no more free memory.

My program has a global dataset. If these dataset has changed data it will
be created a new temporary dataset with a changed data for updating a
database. After the updating the temporary dataset is not further necessary
and should be distroyed automatically.

I have made a simple test program showing the same behaviour.
These program has a global dataset with some changed rows in a datatable.
Then it calls a function with checks the dataset for modifications and creat
a new dataset, copies the changed rows in these dataset and gives the new
dataset back. That's all. The temporary dataset will not be used further.
Because I do not call "accept changes" each check finds changed rows in the
global dataset.
For checking the dataset contents I use a timer event.

Has everyone a Idea what I can do agains the increasing of the used memory?
 
Unless you keep references to all these temporary data sets, that's not the
problem.



What else do you do in this application? For example, are you using anything
which allocates native resources?

Perhaps Bitmaps? Sql(CE)Command/Sql(CE)Connection? If so, are these disposed
of?



By the way, what do you do with these "changes only" DataSets?

If you're using DataAdapter, it's completely unnecessary as DataAdapter
would only update changed records anyway.

That makes sense if you're sending changes to Web Service, though.


Also, are you getting OutOfMemory exception?


Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

*** Want to find answers instantly? Here's how... ***

1. Go to
http://groups-beta.google.com/group/microsoft.public.dotnet.framework.compactframework?hl=en
2. Type your question in the text box near "Search this group" button.
3. Hit "Search this group" button.
4. Read answer(s).
 
Hallo Ilya,

thank's for your reply.
My real application works with a Webservice for data exchanging with a SQL
Server database. I only transfer changed rows back to the database, because
there are some other WinCE devices which can change the data too.
A other function of the WS reads the hole data from the database and merges
then the datatables from device and from the server.

When I let run the program until there is no more free memory then a
messagebox is displayed for aborting a running program.


Meanwhile I have found the statement which produces the memory leak: I call
the Select function from the datatable for receiving the changed rows. Then
I creat a new datatable and copy the rows returned from the select call.
When I do not use this select function then the free memory remains
constant.
I have found that using the select function produces memory laeks generally.
It isn't a big problem for me to replace the call at this position, but I
use the select function on many places and in more complicated cases in the
program. I need a real solution.
 
I am having this exact same problem. I don't know for sure whether it is my
selects, though it could be. How do you know how much free memory you have
left, and how can you tell what calls are causing the leaks?

I am getting Out Of Memory exceptions after an hour or so of running. If a
select from SQL CE causes memory leaks I can't fix, what am I supposed to
do?
 
Hallo Geoff,

finding the problem was a big thing.
But at the end I have made a little test program with a timer which calls a
little part of the real program. At the beginning of the timer event
function I call a function which gives me the actual free memory back. As I
checked the function for getting me changed rows back I found that the free
memory went down after some calls but periodically. This function is easy
and as I replaced the select statement with similar code then I saw no
memory decreasing.

Hope that this help you
 
What function do you call to get the free memory? I'm using Visual Basic, so
that might limit me, but I could write a test function in something else if
necessary.
 
That probably GC.Collect() and GC.WaitForPendingFinalizers().

That might help if you're not disposing of objects which allocate native
resources properly.

It is a bad practice, instead you should find objects you're not disposing
of and fix that.



The rule is: any object which implements IDisposable should be disposed of
manually.

The only exception to that rule is DataSet which is 100% managed and there's
no need to dispose of it.

In fact, dispose does nothing on DataSet.



In C# there's "using" operator which would call IDisposable as soon as you
leave the block:



using (SqlCeConnection connection = new
SqlCeConnection("DataSource=whatever")) {

// Use your connection here

} // Connection is disposed of here even if you have exception inside.


Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

*** Want to find answers instantly? Here's how... ***

1. Go to
http://groups-beta.google.com/group/microsoft.public.dotnet.framework.compactframework?hl=en
2. Type your question in the text box near "Search this group" button.
3. Hit "Search this group" button.
4. Read answer(s).
 
Hallo Ilya,

the call GC.Collect isn't the problem. The Select function of the datatable
is the problem.
Please see the following code:

Private Sub cmdTest_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles cmdTest.Click
Dim i As Integer
Dim DS As DataSet

DS = genTestDS() 'Generates a DS with 100 rows and all kinds of
rowstates

For i = 1 To 500
GC.Collect() 'Minimize the memory
GC.WaitForPendingFinalizers()

ShowMemoryInfo() 'Show aktuell free memory (PInvoke)

'TestSelect1(DS) 'Test with datatable.select
TestSelect2(DS) 'Test without datatable.select
Next

End Sub

Private Function TestSelect1(ByVal DS As DataSet) As DataRow()
Dim rf() As DataRow = Nothing

rf = DS.Tables(0).Select("ID IN (10,20,30,40,50,60,70,80,90)")
Return rf
End Function

Private Function TestSelect2(ByVal DS As DataSet) As DataRow()
Dim rf() As DataRow = Nothing, r As DataRow, c As Integer = 0

For Each r In DS.Tables(0).Rows
If r.RowState <> DataRowState.Deleted Then
Select Case r.Item("ID")
Case 10, 20, 30, 40, 50, 60, 70, 80, 90
If IsNothing(rf) Then
ReDim rf(c)
Else
c += 1
ReDim Preserve rf(c)
End If
rf(c) = r
End Select
End If
Next
Return rf
End Function

Friend Function genTestDS() As DataSet
Dim DS As DataSet = Nothing, dt As DataTable, r As DataRow, i As
Integer
Dim rf() As DataRow


DS = New DataSet
dt = New DataTable
dt.Columns.Add("ID", GetType(Integer))
dt.Columns.Add("Datum", GetType(Date))


'Unchanged
For i = 1 To 50
r = dt.NewRow()
r.Item("ID") = i
r.Item("Datum") = Now
dt.Rows.Add(r)
Next
dt.AcceptChanges()

'Deleted
rf = dt.Select("ID IN (2,3,4,5,6,7,8,9, 40,41,42)")
For Each r In rf
r.Delete()
Next

'Inserted
For i = 51 To 100
r = dt.NewRow()
r.Item("ID") = i
r.Item("Datum") = Now
dt.Rows.Add(r)
Next

'Updated
rf = dt.Select("ID IN (12,13,14,15,16,17,18,19, 20,21,22)")
For Each r In rf
r.Item("Datum") = Now
Next

DS.Tables.Add(dt)

Return DS
End Function


When you start TestSelect2 the used memory jumps between some values (in the
Emulator), but do not growing up.
If you start TestSelect1 the used memory jumps between different values too
but these values are growing up and goes never back.

After I have changed my 'GetChanges' function in a similar way I have made
over 30.000 calls without any memory increasing. Then I have finished the
test.
With the original version (with the select function) I haven't had enough
memory to receive this number of calls.
 
I do not see the problem. I've created a small project with couple text
boxes, added your code and changed test as follows:

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Button1.Click
Me.TextBox2.Text = "Wait, running..."
Dim memStart As Long = GC.GetTotalMemory(True)
Dim memEnd As Long
RunTest()
memEnd = GC.GetTotalMemory(True)
Me.TextBox2.Text = String.Format("Start:{0} End: {1} Difference: {2}",
memStart, memEnd, memEnd - memStart)
End Sub

Public Sub RunTest()
Dim i As Integer
Dim DS As DataSet
Dim memMax As Long = 0
Dim memCurrent As Long
DS = genTestDS() 'Generates a DS with 100 rows and all kinds of rowstates()
For i = 1 To 500
memCurrent = GC.GetTotalMemory(False) ' Get allocated memory
If memCurrent > memMax Then
memMax = memCurrent
End If
Me.TextBox1.Text = String.Format("Loop:{0}, Max: {1}: Now: {2}", i, memMax,
memCurrent)
' No need for these
'GC.Collect() 'Minimize the memory
'GC.WaitForPendingFinalizers()
TestSelect1(DS) 'Test with datatable.select
TestSelect2(DS) 'Test without datatable.select
Next
End Sub

It runs just fine with both methods at once; memory difference at completion
is about 250K which is probably the size of JITed code.
Memory allocated at the end of test remains the same on test reruns, so
there's no leak.

Do you have SP3 installed? One of the fixes in it was a stop to rare random
memory leak.
Can you be leaking memory in your call to ShowMemoryInfo()?

Best regards,

Ilya

Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

*** Want to find answers instantly? Here's how... ***

1. Go to
http://groups-beta.google.com/group/microsoft.public.dotnet.framework.compactframework?hl=en
2. Type your question in the text box near "Search this group" button.
3. Hit "Search this group" button.
4. Read answer(s).
 
Hallo Ilya,



Of course I have installed CF 1.0 SP3 and have used NCFSetup for updating
the real devices.

The cgacutil program on the devices shows version 1.04292.00.

For calculating of the used memory I have used the function
GlobalMemoryStatus from the example PInvoke.

This code is from MS and should not contain a error. The rest in the
function

ShowMemoryInfo is simple: saving the start value at the first run,
calculating the current used memory and showing this information with 3
textboxes. There is nothing for wasting memory.

By the side: In the test I have only one loop and because I was lazy I only
disabled one version and enabled the other version. If it should exist a
problem in the function ShowMemoryInfo, the problem would be appearing in
both cases.



In the listing of the test program I have a loop with 500 passes, because
this was enough for me to see the problem.

In the following table you see the results of the used memory for different
devices and different counts of passes. (All values in KB).



Passes
Emulator

Compaq PPC (SH3)
Teklogix WinCE (ARMV4I)


TestSelect1
TestSelect2
TestSelect1
TestSelect2
TestSelect1
TestSelect2

500
352
160
300
56
248
84

1000
604
172
512
56
492
84

1500
824
204
816
56
740
84

2000
1100
204
1064
56
1004
84







It is obviously: TestSelect1 (calling select of the datatable object) have a
memory problem. If you want you can estimate after how many passes no more
free memory is there.

On the devices is only CF 1.0 SP3 installed -no further hotfix.



If you should have a CF 1.0 version with showing not this behaviour please
send me your CF version.

Because my application must run 24 hours a day and 356 days a year I need
urgent a solution for this problem.





Best regards,



Siegfried.
 
I've added native memory print out using GlobalMemoryStatus() and I do not
see any leaks.

Difference reaches zero on 3rd run and stays at zero no matter how many
times you rerun the code.

I'm using the same SP3 you have: 1.0.4292.0.



Repro I'm using is attached, please review the code and try it on your
device.



If you're getting OutOfMemory exception, please post a simple compliable
repro, I'll run it overnight or longer if needed.



Also, here's a good reading on the subject:



http://www.danielmoth.com/Blog/2005/01/memory-problems-faq.html


Best regards,

Ilya

This posting is provided "AS IS" with no warranties, and confers no rights.

*** Want to find answers instantly? Here's how... ***

1. Go to
http://groups-beta.google.com/group/microsoft.public.dotnet.framework.compactframework?hl=en
2. Type your question in the text box near "Search this group" button.
3. Hit "Search this group" button.
4. Read answer(s).
 
Hallo Ilya,





I have send you a email direct because the news server has refused my
answer. The answer with the appendix was too long.

I hope the email will achieve you.



Best regards,



Siegfried.
 
Back
Top