space used in memory for a table and its indexes

  • Thread starter Thread starter Bernie Yaeger
  • Start date Start date
B

Bernie Yaeger

When you load a table into memory as a datatable using ADO .net, is the
space for the table's indices also loaded into memory?

Tx for any help.

Bernie Yaeger
 
The table is stored as a series of arrays--one for each column. No
server-side indexes are loaded into client memory.
hth

--
____________________________________
William (Bill) Vaughn
Author, Mentor, Consultant
MVP, hRD
www.betav.com
Please reply only to the newsgroup so that others can benefit.
This posting is provided "AS IS" with no warranties, and confers no rights.
__________________________________
 
Hi Bill,
How is the row object implemented? I want to implement my own DataTable-like
structure. If I create a collection of type specific columns, can you
suggest a way to handle the rows?

My goal is to be able to read and add/insert rows where each cell (column)
in the row is type specific. I do not want to have to box/unbox or otherwise
wrap my primitive data.

Regards,
Dave
 
Want to know exactly how it's implemented? Use Anakarino to decompile the
Framework... ;)
I can't encourage creation of "parallel but equivalent" code. As a dev
manager I don't like my people reinventing solutions for alternatives that
won't improve performance by more than a few percentage points (if that).

--
____________________________________
William (Bill) Vaughn
Author, Mentor, Consultant
MVP, hRD
www.betav.com
Please reply only to the newsgroup so that others can benefit.
This posting is provided "AS IS" with no warranties, and confers no rights.
__________________________________
 
Hi Bill,
I could really use your input and guidance. In order to justify that our
similar efforts have not been a waste of effort, here is what we have
achieved so far:

Our current project (implemented in C#) does a lot of computations and
creates a large (multi-gigabyte) results set.

The original design/implementation was done according to standard or common
..NET recommended "best practices", to the best of our ability. The app took
about 48 hours to create a certain results set. Careful, but simple changes
cut this time in half and some people thought that was the best that could
be done. (BTW, we use the ANTS profiler.)

However, using very creative approaches, but still sticking with good OO C#
code, we can now generate this size results set in 2 hrs 45 min. Saving over
21 hours is significant! It makes the app viable and competitive with unix/c
implementations. We believe this proves that C# can meet or exceed C
performance in the scientific data crunching arena.

We are doing various things to achieve this performance, but in every case,
our biggest gains have come from implementing our own code to replace stuff
that was offered in the framework. You might call this the creation of
"parallel but equivalent" code. However, we have achieved what most people
considered impossible (in terms of performance) for a C# number cruching
app.

As one small example, we implemented our own persistence (replacing .NET
binary serialization). This gave us a 100 fold improvement (ie, 10,000%).
That's not "a few percentage points".

I don't want to sound arrogant, and I'm not implying that we have all the
answers -- we don't. We are relatively inexperienced in C# and .NET. I would
really appreciate your input on this issue of how to implement a
minimalistic datatable equivalent. I doubt we can do that without some input
from folks like yourself. However, once we get started and then complete
this, I am optimistic that it will give us significant performance gains --
because we have *numerous* examples of this already. If you will help me get
started, I'll report the results in the newsgroup and we can find out
whether or not this particular implementation was a waste of time for us.

Regards,
David
 
It sure sounds like you have learned quite a bit about the framework and C#.
I doubt if I could be of much assistance.

--
____________________________________
William (Bill) Vaughn
Author, Mentor, Consultant
MVP, hRD
www.betav.com
Please reply only to the newsgroup so that others can benefit.
This posting is provided "AS IS" with no warranties, and confers no rights.
__________________________________
 
Bill,
I was just hoping to convince you (or anyone reading this thread) that that
input from people such as yourself would not be wasted. You seemed to be
saying it was a waste of time to even consider working on this, and I hoped
to show why that might not be the case. The fact is I do need some tips to
get me started in the right direction.

For example, I do not yet know the basics of how any database would
implement a row object (with typed columns) for a data table. Is there a
text book that covers this? I haven't seen any coverage in data structures
books, nor in database texts.

I guess I need a manual covering the basics of how tables and rows are
implemented in databases. Does anyone who knows how generic databases
implement tables and rows care to point me in the right direction?

Thanks!
Dave
 
Nah, I'm just saying for the mainstream customer/developer I would prefer to
stay within the boundaries of the Framework. I expect that there are those
that would love to wade into this problem, but I for one, am too strapped
for time to spend much time on it.
Sorry.

--
____________________________________
William (Bill) Vaughn
Author, Mentor, Consultant
MVP, hRD
www.betav.com
Please reply only to the newsgroup so that others can benefit.
This posting is provided "AS IS" with no warranties, and confers no rights.
__________________________________
 
Back
Top