Memory...issues

  • Thread starter Thread starter James
  • Start date Start date
J

James

I have a record set of roughly 1200 records. The logic flow is like this:

Dim ds As New DataSet
Dim da As New SqlDataAdapter("SELECT * FROM Fax_Queue", objConnection)
da.Fill(ds, "Queue")

Dim dr As DataRow
Dim rpt as FaxReport = New FaxReport


For Each dr In ds.Tables("Queue").Rows
rpt.listid = dr("List_ID")
rpt.Run(False)
Dim xTIFF As DataDynamics.ActiveReports.Export.TIFF.TiffExport = New
DataDynamics.ActiveReports.Export.Tiff.TiffExport

tiffPath = path & ".tiff"
xTIFF.Export(rpt.Document, tiffPath)
xTIFF.Dispose()
GC.Collect() ' ???

Dim del As New SqlCommand("DELETE FROM Tbl_Collection_Fax_Queue
WHERE Queue_ID = " & dr("Queue_ID"), objConnection)
objConnection.Open()
del.ExecuteNonQuery()
objConnection.Close()
Next


As this application progresses and the TIFF files are generated, the memory
consumed by this process increases to a point where it basically locks up.
How can I prevent this memory leak? Is there a logic problem or can I
forcibly release memory somehow? In my head this process should be
consuming roughly the same amount of memory consistently.

Thanks for any help you can offer,
James
 
As this application progresses and the TIFF files are generated, the
memory consumed by this process increases to a point where it
basically locks up. How can I prevent this memory leak? Is there a
logic problem or can I forcibly release memory somehow? In my head
this process should be consuming roughly the same amount of memory
consistently.
Click to expand...

Could be a bug with the component. Have you talked to the developers of
ActiveReports?

You shouldn't need to GC.collect your member explicitly.
 
James,

Can you try what happens if you place the creating and accessing of your
xTiff in a seperate method. That should be on a new stack and therefore go
out of scoop everytime.

Cor
 
I have a record set of roughly 1200 records. The logic flow is like this:

Dim ds As New DataSet
Dim da As New SqlDataAdapter("SELECT * FROM Fax_Queue", objConnection)
da.Fill(ds, "Queue")

Dim dr As DataRow
Dim rpt as FaxReport = New FaxReport

For Each dr In ds.Tables("Queue").Rows
rpt.listid = dr("List_ID")
rpt.Run(False)
Dim xTIFF As DataDynamics.ActiveReports.Export.TIFF.TiffExport = New
DataDynamics.ActiveReports.Export.Tiff.TiffExport

tiffPath = path & ".tiff"
xTIFF.Export(rpt.Document, tiffPath)
xTIFF.Dispose()
GC.Collect() ' ???

Dim del As New SqlCommand("DELETE FROM Tbl_Collection_Fax_Queue
WHERE Queue_ID = " & dr("Queue_ID"), objConnection)
objConnection.Open()
del.ExecuteNonQuery()
objConnection.Close()
Next

As this application progresses and the TIFF files are generated, the memory
consumed by this process increases to a point where it basically locks up.
How can I prevent this memory leak? Is there a logic problem or can I
forcibly release memory somehow? In my head this process should be
consuming roughly the same amount of memory consistently.

Thanks for any help you can offer,
James
Click to expand...

James,

What version of the runtime are you working with here? Also, might I
suggest that you not use a DataSet here... You are working with one
record at a time in a forward only manner, I believe you will see much
improved performance and reduce some of the memory pressure since you
won't have all 1200 records in memory at once. Also, you might want
to avoid the open/closes of the data connection. With connection
pooling, you probably aren't being hurt to much but, it seems sort of
pointless in this case. I would open the connection, run your process
and close the connection when your done. You should probably avoid
calling GC.Collect as well...

Ok... Another question? Do you run this procedure often? In rapid
succession? You might want to get ahold of a profiler. There are
several out there, and a couple of them are free or have free trials
(I've used Ants profiler in the past with very good success, but
YMMV). This may boil down to a bug in a 3rd party component, but
profiling the application will give you a good idea of where and why
the memory is being consumed.

Anyway, just some random thoughts on optimization and trouble
shooting :)

HTH
 
Hi Tom,

Good to see you again.

However, I thought I had missed something (happens often), but in my idea is
the OP only using the DataSet to get some values, not to place values in the
dataset (then he would need the memorystream and byte arrays).

Cor
 
Hi Tom,

Good to see you again.

However, I thought I had missed something (happens often), but in my idea is
the OP only using the DataSet to get some values, not to place values in the
dataset (then he would need the memorystream and byte arrays).

Cor
Click to expand...

Cor,

Long time no see.... I don't see any place where he is actually
updateing the dataset, so I think the use of a sqldatareader would be
best here. Just some random thoughts... ;)
 
I don't disagree that a DataReader would be fine, but that's somewhat
incidental. I actually believe I've solved the problem by moving the Dim
for the report inside of the For loop and setting rpt = Nothing. This
removes the "leak" for some reason. This is very much in alpha phase and I
didn't know whether I'd need to manipulate data for any reason or not. More
a proof of concept than anything.

The process itself is set in a windows service to run every 15 minutes. It
could have as many as 5000 records. 99% of the time there will be no
records and it will simply exit.

Anyway my issue now is that for some reason this logic flow is causing
problems:

<generate tiff>
<attach tiff to email>
<send>

Most of the time this works fine, but other times it seems that the
attachments are getting corrupted somehow. Gut feeling is that it's
attaching the files before they're truly done generating. Considering the
creation occurs in a class I have no control over, is there any good way to
determine that the file is 100% created before I attempt to attach it to an
email?

Thanks
 
James,

By setting it inside a loop, you are doing the same as setting it in a
method, in a loop there is most probably a stack created (I never
investigated it) as with a new method. That stack is removed at the end of
the loop and the objects can be garbaged (I have the idea that this is a
starting point for garbadging, as well something I have never investigated).

Cor
 
James said:
I don't disagree that a DataReader would be fine, but that's somewhat
incidental. I actually believe I've solved the problem by moving the Dim
for the report inside of the For loop and setting rpt = Nothing. This
removes the "leak" for some reason.
Click to expand...

If you declare the variable inside the loop or not doesn't make any
difference for the memory management. It changes the scope of the
variable, but not the usage, and the garbage collector is only concerned
with the usage of variables, not the scope.

However, if you create a new instance of the object for every iteration
in the loop instead of creating one instance outside the loop and reuse
it, that might very well make all the difference in the world. The
object might not be designed for reuse at all.
Anyway my issue now is that for some reason this logic flow is causing
problems:

<generate tiff>
<attach tiff to email>
<send>

Most of the time this works fine, but other times it seems that the
attachments are getting corrupted somehow. Gut feeling is that it's
attaching the files before they're truly done generating. Considering the
creation occurs in a class I have no control over, is there any good way to
determine that the file is 100% created before I attempt to attach it to an
email?
Click to expand...

From what I can tell, you are using the same name for all the files
that you create. If the method that sends the mail is asynchronous, you
will be replacing the contents of the file with a new image while the
previous mail is still being sent.
 
Cor said:
James,

By setting it inside a loop, you are doing the same as setting it in a
method, in a loop there is most probably a stack created (I never
investigated it) as with a new method. That stack is removed at the end
of the loop and the objects can be garbaged (I have the idea that this
is a starting point for garbadging, as well something I have never
investigated).
Click to expand...

No, a loop doesn't create a new stack frame.
 
Thanks for the reply. The file name is different each time, I just
simplified the code, which obviously ended up being detrimental. Anyway,
I'm gathering the object wasn't designed for reuse and has some type of
inherent flaw...so disposing of it and recreating it seems to "fix" what
isn't really my problem in the first place. That's my best guess at this
point.
 
Back
Top