Setting based 1 Arrays

[Herfried K. Wagner [MVP]]

At least for me an array index is more than a position only. In
addition, I consider it a numeric /key/ used to define a mapping between
numeric keys and values (f(x) = y, x in [a, ..., b]).

An array is meant to be the simplest most efficient list you can have, a
contiguous block of memory.

I'd consider it a low-level implementation detail that an array is stored as
a contiguous block of memory.

If you want to add, insert, remove items or store a key then you don't
want an array, you want a collection of some sort.

Although that's true, I didn't say that I prefer insertion of values into
arrays.

It is a lot clearer in C# than VB. In C# you cannot modify your array
making it pretty obvious to the programmer it is a fixed block of memory.
If you want a fixed block that does not change then use an array, if you
need to change it then use a collection.

The fixed block semantics are IMO an implementation detail which is
irrelevant for programming languages which do not provide pointer access to
the array data.

What do you mean there?

Arrays are so limited in functionality making them a bad choice in most
cases. Imagine a method returning an array of data. If you want to
manipulate the data (removing and inserting items), you'll either have to
create a new array and copy the items into the new array or copy them to
another data structure. This approach is not very efficient and requires
duplication of values or references.

 

[Mike]

Arrays are so limited in functionality making them a bad choice in most
cases.

Huh? Most cases? How about "some cases?"

Imagine a method returning an array of data. If you want to
manipulate the data (removing and inserting items), you'll either have
to create a new array and copy the items into the new array or copy them
to another data structure. This approach is not very efficient and
requires duplication of values or references.

So you add more WTs (Wrapper Technology)? To work with what would in
the end of the day be memory block "array" manipulation? How is that
more "efficient" than going raw?

Maybe it is somewhat inefficient in languages like VB with inherent
garbage collection ideas and you are left with a guessing game. But
now with VB.NET inheriting concepts from C/C++, there is no real why
it memory management can't be more efficient.

There is no such thing as a bad language, just bad programmers.
Conversely, there is no such thing as a good language, just good
programmers.

--

 

[Michael C]

I'd consider it a low-level implementation detail that an array is stored
as a contiguous block of memory.

It's still a detail you cannot escape as a high level programmer. You still
need to know it's going to be the most efficient method of storing the data
but probably the least efficient at modifying. Adding, removing, inserting
is slow because of the low level implementation. The same rules apply if
you're using C++ or vb.net.

Although that's true, I didn't say that I prefer insertion of values into
arrays.

Then you probably want something other than an array. The List collection is
still very efficient but allows all this.

The fixed block semantics are IMO an implementation detail which is
irrelevant for programming languages which do not provide pointer access
to the array data.

You can pretend it's irrelevant but arrays are still slow for inserting
because of the way it is implemented. Because of the way other collections
are implemented they are significantly faster. By allowing add, insert,
remove vb does the programmer an injustice because most programmers think
this is a good idea. (I presume vb allows this?)

Arrays are so limited in functionality making them a bad choice in most
cases. Imagine a method returning an array of data. If you want to
manipulate the data (removing and inserting items), you'll either have to
create a new array and copy the items into the new array or copy them to
another data structure. This approach is not very efficient and requires
duplication of values or references.

Lots of functions do return arrays and I can't say I've found it a problem.
Inserting/removing an item in an array will copy the entire array anyway but
do it for *every* insert.

Michael

 

[Cor Ligthert[MVP]]

Michael,

An array has from the first day been: Starting Memory Address, Length of
item, indexer
where the memory position for an item = memory address + (indexer * length
of item)

There has never been a possibility to change the indexer simply because of
the fact that the memory address that would be affected by increasing the
indexer could be occupied.

This has nothing to do with high or low level programming.
An array is from the time of low level programming incorporated in high
level languages.

Collections (List) are from high level languages.

Cor

 

[Mike]

I can't help with the C++ side but I wouldn't have thought it would be that
many wrappers I guess vb.net or C# will be easier.

I am having some fun now mixing it up. Using C++/CLR to create
methods that are more complex to do than in VB.NET (at least I have
not figured it out or its too complex). I don't know the C# language
strong enough yet to be productive.

So now have 2 DLLs

wcServerAPI.DLL (written in VB.NET)
wcServerAPISharp.DLL (written in C#)
WcServerAPICLR.CLL (written in C++/CLR)

I can probably just stick with the CLR version once I rewrite my C/C++
headers (break it up and provide C++/CLR prototype attributes). What
creeps me out is the tedious details you have to go thru just to do
simple I/O of some fundamental types. I know part of it is just not
knowing the language fully yet.

I need to get me a good expert .NET book for experienced c/c++
programmers.

---

 

[Michael C]

Michael,

An array has from the first day been: Starting Memory Address, Length of
item, indexer
where the memory position for an item = memory address + (indexer * length
of item)
Ok.

There has never been a possibility to change the indexer simply because of
the fact that the memory address that would be affected by increasing the
indexer could be occupied.

I'm not sure what you're trying to say there.

This has nothing to do with high or low level programming.
An array is from the time of low level programming incorporated in high
level languages.

I believe that was the point I already made.

Collections (List) are from high level languages.

Depends what you consider high and low level languages. Some consider C a
low level language and it has all sorts of collections available to it.

Michael

 

[Michael C]

I can probably just stick with the CLR version once I rewrite my C/C++
headers (break it up and provide C++/CLR prototype attributes). What
creeps me out is the tedious details you have to go thru just to do simple
I/O of some fundamental types. I know part of it is just not knowing the
language fully yet.

There is a lot of that, because you can't just include a .h file you have to
copy paste relevant parts in and modify them and then create wrappers. I
guess it gets easier as you go

 

[Herfried K. Wagner [MVP]]

Lots of functions do return arrays and I can't say I've found it a
problem. Inserting/removing an item in an array will copy the entire array
anyway but do it for *every* insert.

So, let me ask what you are typically doing with the items returned in an
array? It's often necessary to modify the returned "list" of items, which
requires either costly array copying or transferring the items to another
data structure. Not very consistent and well-designed, isn't it?

 

[Mike]

There is a lot of that, because you can't just include a .h file you have to
copy paste relevant parts in and modify them and then create wrappers. I
guess it gets easier as you go

I think so, and there is no substitute for rolling up the sleeves and
putting in the time.

Part of the problem (for me) is that I trying to reuse and leverage
what I already have, over 12 years of WIN32 RPC API development. For
example, I have a IDL for the RPC (DCE) server, which for the function
I have been using as an example in the threads:

interface Wildcat
{
typedef [context_handle] void *TWildcatContextHandle;
...
error_status_t GetAccessProfileNames(
[in] TWildcatContextHandle wch,
[in] DWORD count,
[out, ref, size_is(count)] char names[][SIZE_SECURITY_NAME]);
...
}

I am wondering if i can take this IDL file and maybe use it as feed to
a new converter tool (or maybe existing?) to create a .NET class
library. Maybe a veteran RPC, now .NET expert can give me feed back
on that. Obviously, the context handle will not be needed here for
the client wrap. I've been using the disassembler tool to learn some
basic translations .NET compilers are doing.

Once I get the basic ".net automation" worked out, then I use
preferred VB.NET to create components that people can just drop in
their forms .

--

 

[Mike]

So, let me ask what you are typically doing with the items returned in
an array? It's often necessary to modify the returned "list" of items,
which requires either costly array copying or transferring the items to
another data structure. Not very consistent and well-designed, isn't it?

What about retrieving an array for display, for selection, for
assignment to a List control, etc - a very common practice.

In general, if there is a need to manipulate an array, the efficiency
(system performance) of it depends on your loads - how large? Is speed
an issue?

If largest is required, there are other methods, such as sparse
matrices that are more efficient. But I ask, does it really make
copies when inserting? Not always. Depends on the implementation of
the System.Array which is most likely a C++ ATL class. Moving block of
memory (using machine op codes translated by your code) are a
fundamental part of the WinIntel memory system. Off hand, 5-10 op
codes are need to insert memory by moving the block, and again, if
sparse methods are using, no movement is required at at.

But I don't get something. Why is an manipulating array an issue and
not manipulating string or an XML document or any other collection?
Arrays is a fundamental element of programming, whether you use it or
not, the code behind the scene is.

---

 

[Michael Williams]

It's still a detail you cannot escape as a high level programmer.
You still need to know it's going to be the most efficient method
of storing the data but probably the least efficient at modifying.
Adding, removing, inserting is slow because of the low level
implementation. The same rules apply if you're using C++ or vb.net.

Why do you keep changing things and putting words into other people's mouths
so that you can contradict something they haven't actually said? I use
arrays all the time for lots of purposes and they are extremely useful, but
I don't usually add or remove individual elements and I use them for the
purposes that they are most suitable for. Are you suggesting that I should
not use them? In fact I seem to recall being involved in a little
"challenge" a while back involving some code to perform a specific operation
on a bitmap and I think that you were also involved in it? I did not of
course expect to beat C++ code (or was it C# code?) to perform the same task
because I was writing the code in VB6, but as I recall my VB6 code which
used VB6 arrays very close to the speed of the C implementation. Arrays are
fine, even though (like almost everything else) they are not suitable for
every purpose.

Mike

 

[Michael Williams]

An array has from the first day been: Starting Memory Address,
Length of item, indexer where the memory position for an item
= memory address + (indexer * length of item). There has never
been a possibility to change the indexer simply because of the
fact that the memory address that would be affected by
increasing the indexer could be occupied.

What do you mean by "changing the indexer"? Using VB6 it is possible to
create (for example) a zero based array and then "on the fly" change it to a
1 based array (or to any other base) without any problems. You can also
create (for example) an array of Longs, and populate it with data if you
wish, and then on the fly you can change it to an array of Bytes without
upsettings the data, so that you can use standard VB functions to read the
data as Bytes rather than Longs if you wish to do so for some specific
reason. Or do you mean something else?

Mike

 

[Michael C]

So, let me ask what you are typically doing with the items returned in an
array?

Well I could do all sorts of things with them but out of the range of things
I need to do it's not often I need to modify the list. In the case that I do
then yes I will have to copy it to another list.

It's often necessary to modify the returned "list" of items,

I'd say it's less common.

which requires either costly array copying or transferring the items to
another data structure. Not very consistent and well-designed, isn't it?

Well, you could always return a collection but the problem with that it
you're not always likely to return the right type of collection, so the
programmer will need to copy anyway. Generally when an array is returned I
am going to need to interate the list of items at least once to do something
with them (I must have requested them for a reason.

Michael

 

[Michael C]

Why do you keep changing things and putting words into other people's
mouths so that you can contradict something they haven't actually said? I
use arrays all the time for lots of purposes and they are extremely
useful, but I don't usually add or remove individual elements and I use
them for the purposes that they are most suitable for. Are you suggesting
that I should not use them? In fact I seem to recall being involved in a
little "challenge" a while back involving some code to perform a specific
operation on a bitmap and I think that you were also involved in it? I did
not of course expect to beat C++ code (or was it C# code?) to perform the
same task because I was writing the code in VB6, but as I recall my VB6
code which used VB6 arrays very close to the speed of the C
implementation. Arrays are fine, even though (like almost everything else)
they are not suitable for every purpose.

What on earth are you rambling on about this time Mike? I never said there
was a problem with arrays, they have their use in the right situations. Like
anything else you need to know their good and bad points.

Michael

 

[Mike]

Well, you could always return a collection but the problem with that it
you're not always likely to return the right type of collection, so the
programmer will need to copy anyway. Generally when an array is returned I
am going to need to interate the list of items at least once to do something
with them (I must have requested them for a reason.

Btw, in VB.NET the Array class implements ICollection.

Its all the same thing

---

 

[Michael C]

Btw, in VB.NET the Array class implements ICollection.

Underneath though it is still using a contiguous block of memory the same
way C++ does. ICollection most likely just allows you to interate the array
and not add anything.

 

[Tom Shelton]

Underneath though it is still using a contiguous block of memory the same
way C++ does. ICollection most likely just allows you to interate the array
and not add anything.

Under the covers List(Of T) and ArrayList are simply arrays. They just resize
their selves as they grow in an intelligent fashion.

 

[Michael Williams]

[Michael Williams said] Why do you keep changing things
and putting words into other people's mouths so that you
can contradict something they haven't actually said?

[Michael C mumbled] What on earth are you rambling on
about this time Mike? I never said there was a problem with
arrays, they have their use in the right situations.

What I'm "rambling on about", as you very well know, is the fact that you
regularly attempt to maintain and give credence to your own ramblings by
deliberately putting words into other people's mouths and then arguing
against them as though they had actually said those words. You do this
simply because you like the sound of your own voice and you want to pretend
that you are better than people even when you are not. For example, the
specific instance of that behaviour to which I was referring in my previous
post was when you said to Herfried K Wagner, "If you want to add, insert,
remove items or store a key then you don't want an array then you want a
collection of some sort", even though Herfried Wagner had /not/ said that he
wanted to do anything of the kind. Herfried's response to you was, "Although
that's true, I didn't say that I prefer insertion of values into arrays".

You're a troll here, just like me

Mike

 

[Michael C]

What I'm "rambling on about", as you very well know, is the fact that you
regularly attempt to maintain and give credence to your own ramblings by
deliberately putting words into other people's mouths and then arguing
against them as though they had actually said those words. You do this
simply because you like the sound of your own voice and you want to
pretend that you are better than people even when you are not. For
example, the specific instance of that behaviour to which I was referring
in my previous post was when you said to Herfried K Wagner, "If you want
to add, insert, remove items or store a key then you don't want an array
then you want a collection of some sort", even though Herfried Wagner had
/not/ said that he wanted to do anything of the kind. Herfried's response
to you was, "Although that's true, I didn't say that I prefer insertion of
values into arrays".

If I did that it was purely by mistake. I actually misread the line you
quoted thinking that he said he did prefer it. I tend to read people's
replies too quickly and sometimes mix up what 1 person has said with
another.

You're a troll here, just like me

Speak for yourself Mike, unlike you I hand out useful advice.

Michael

 

[Michael C]

Under the covers List(Of T) and ArrayList are simply arrays. They just
resize
their selves as they grow in an intelligent fashion.

Are you saying if I insert an element at the bottom of a million item List
then it will just copy a huge array up 1 element? Yikes!

Michael