Size of datatable...in bytes?

  • Thread starter Thread starter Jesper
  • Start date Start date
J

Jesper

Is there any way to get or estimate the memory being used by a dataset or a
datatable? I'm writing a caching class and trying to keep track of the size
of the data I'm keeping or at least an estimate, to make better decisions on
when and what to expire.

I realize that without going to unmanaged code, I'm probably only going to
be able to get an estimate, but thats ok. Right now I'm doing a rather crude
loop over all the columns of a table but I'm not including any overhead for
neither columns, rows, tables, or sets. And I'm guessing that they are quite
conciderable.
 
Jesper,

I don't think that your datatable or your dataset will be very hugh in size.
However all those datarow objects and moreover all those items where they
are again referencing too can be of course very huge. By instance a photo
item.

Just my idea

Cor
 
I only have 'primitive' data in my database. Numbers and some strings here
and there. However, the cache is so flexible that it can contain both a
single int object, a DataTable, and a full Dataset. And there can easily be
100 datasets and thousands of small int objects in the cache if the user is
paging back and forth. So I'd like to put an 'expense' factor on each cache
item in the form of (estimated) size divided by how often I expect to be
needing it, and then exipre the most 'expensive' items first.

Jesper.
 
Hi Jesper,

You might use some memory allocation profiler such as AQTime from
AutomatedQA to check out the memory hit.
Also you might use WeakReference class for your cache - it will ensure that
the memory will be reclaimed (and the cached data lost) when low on memory.
 
Thats a pretty good idea. I think I'll do that in my testing environment and
see if I can catch an approximate pattern to the size, and then replace it
with more efficient code.
 
Hi Sahil,

Just a note that serialized size might be less than actualy one since not
all data might be serialized.
And, ah, DataSet serializes into an XML file btw. It will give him way
greater memory size than actual is.
Perhaps both issues combined might give him exact size :-)
 
Miha you are right. To get around that, just use DataSetSurrogate (search
the MS KB for that).

Anyway, it will always be "proportionate" :). So it's a good design time
measure.

- Sahil Malik [MVP]
http://codebetter.com/blogs/sahil.malik/





Miha Markic said:
Hi Sahil,

Just a note that serialized size might be less than actualy one since not
all data might be serialized.
And, ah, DataSet serializes into an XML file btw. It will give him way
greater memory size than actual is.
Perhaps both issues combined might give him exact size :-)

--
Miha Markic [MVP C#] - RightHand .NET consulting & development
www.rthand.com
SLODUG - Slovene Developer Users Group www.codezone-si.info

Sahil Malik said:
Jesper - you can serialize it to a memorystream and do a
memorystrem.length. That's a good enough estimate.
Not saying that it'd be very efficient :)

- Sahil Malik [MVP]
http://codebetter.com/blogs/sahil.malik/
Click to expand...
Click to expand...
 
Miha- I haven't done a lot w/ weakreference but I'm trying to get more
familiar with it. Do you have a quick example or how you'd implement it in
this instance?

Thanks man!

--
W.G. Ryan MVP (Windows Embedded)

TiBA Solutions
www.tibasolutions.com | www.devbuzz.com | www.knowdotnet.com
Miha Markic said:
Hi Jesper,

You might use some memory allocation profiler such as AQTime from
AutomatedQA to check out the memory hit.
Also you might use WeakReference class for your cache - it will ensure that
the memory will be reclaimed (and the cached data lost) when low on memory.

--
Miha Markic [MVP C#] - RightHand .NET consulting & development
www.rthand.com
SLODUG - Slovene Developer Users Group www.codezone-si.info

Jesper said:
Is there any way to get or estimate the memory being used by a dataset or
a datatable? I'm writing a caching class and trying to keep track of the
size of the data I'm keeping or at least an estimate, to make better
decisions on when and what to expire.

I realize that without going to unmanaged code, I'm probably only going to
be able to get an estimate, but thats ok. Right now I'm doing a rather
crude loop over all the columns of a table but I'm not including any
overhead for neither columns, rows, tables, or sets. And I'm guessing that
they are quite conciderable.
Click to expand...
Click to expand...
 
Hi Bill,

I don't have any example handy, however it should be prety straightforward.
Instead of holding reference to, i.e., dataset, you hold reference to a
WeakReference instance which in turn holds reference to DataSet instance
(its property Target).
If WeakReference.Target returns you null you know that the DataSet instance
is gone.

--
Miha Markic [MVP C#] - RightHand .NET consulting & development
www.rthand.com
SLODUG - Slovene Developer Users Group www.codezone-si.info

W.G. Ryan eMVP said:
Miha- I haven't done a lot w/ weakreference but I'm trying to get more
familiar with it. Do you have a quick example or how you'd implement it in
this instance?

Thanks man!

--
W.G. Ryan MVP (Windows Embedded)

TiBA Solutions
www.tibasolutions.com | www.devbuzz.com | www.knowdotnet.com
Miha Markic said:
Hi Jesper,

You might use some memory allocation profiler such as AQTime from
AutomatedQA to check out the memory hit.
Also you might use WeakReference class for your cache - it will ensure that
the memory will be reclaimed (and the cached data lost) when low on memory.

--
Miha Markic [MVP C#] - RightHand .NET consulting & development
www.rthand.com
SLODUG - Slovene Developer Users Group www.codezone-si.info

Jesper said:
Is there any way to get or estimate the memory being used by a dataset or
a datatable? I'm writing a caching class and trying to keep track of
the
size of the data I'm keeping or at least an estimate, to make better
decisions on when and what to expire.

I realize that without going to unmanaged code, I'm probably only going to
be able to get an estimate, but thats ok. Right now I'm doing a rather
crude loop over all the columns of a table but I'm not including any
overhead for neither columns, rows, tables, or sets. And I'm guessing that
they are quite conciderable.
Click to expand...
Click to expand...
Click to expand...
 
Back
Top