Handle to pointer conversion

[Ioannis Vranos]

I have been checking C++/CLI lately by using VC++ 2005 Express Beta 1
(should be called Alpha though).

In managed extensions we could pass managed pointers to functions taking
unmanaged pointers as parameters by using __pin pointers:


__gc class whatever
{
// ...
};


template<class T>
void somefun(T *p)
{
// ...
};



int main()
{
whatever *p=__gc new whatever;

whatever __pin *pp=p;

somefun(pp);

// ...
}



How can this handle to pointer conversion take place in C++/CLI? The
following does not seem to work:


ref class managed
{
};


int main()
{
using namespace stdcli::language;

managed ^h=gcnew managed;

pin_ptr<managed>p=h;
}


C:\c>cl /clr temp.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 14.00.40607.16
for Microsoft (R) .NET Framework version 2.00.40607.16
Copyright (C) Microsoft Corporation. All rights reserved.

temp.cpp
temp.cpp(12) : error C3833: 'managed' : invalid target type for pin_ptr
Cannot declare pin_ptr to a handle type. Consider declaring
pin_ptr to one of the members of 'managed'

C:\c>






Best regards,

Ioannis Vranos

 

[Brandon Bray [MSFT]]

I have been checking C++/CLI lately by using VC++ 2005 Express Beta 1
(should be called Alpha though).

Actually, the alpha is something else entirely. We're getting very close to
publishing an update to the VC tools which has a far more complete set of
the syntax we want to support in Whidbey.

In managed extensions we could pass managed pointers to functions taking
unmanaged pointers as parameters by using __pin pointers:

[CODE SAMPLE]

How can this handle to pointer conversion take place in C++/CLI? The
following does not seem to work:

[SNIP]

temp.cpp
temp.cpp(12) : error C3833: 'managed' : invalid target type for pin_ptr
Cannot declare pin_ptr to a handle type. Consider declaring
pin_ptr to one of the members of 'managed'

The error message is correct (aside from the unfortunate use of "handle
type" which should be "ref class"). There is no way to directly pin a whole
object on the GC heap, nor is there any good reason to do so.

Since a whole object on a GC heap can never be passed to native code, it's
only the members of such an object that should be passed. Therefore, you
need to pin the member of the ref class that you want to pass to native
code.

FYI, by pinning a member of an object or an element of an array, you pin the
entire object or the entire array respectively.

 

[Ioannis Vranos]

The error message is correct (aside from the unfortunate use of "handle
type" which should be "ref class"). There is no way to directly pin a whole
object on the GC heap, nor is there any good reason to do so.


?!



Since a whole object on a GC heap can never be passed to native code, it's
only the members of such an object that should be passed. Therefore, you
need to pin the member of the ref class that you want to pass to native
code.

FYI, by pinning a member of an object or an element of an array, you pin the
entire object or the entire array respectively.

What do you mean by that. Consider a type describing a bank account:


ref class BankAccount
{
// ...
};



Suppose we have a list of some objects of this type and we have defined
comparison operators which in essence compare the money amounts of each
account, and we want to sort them based on those money with a sort
function (which could be the std::sort).


So why we can't pass these objects and only their members? This is a
rational thing to do and possible with managed extensions!


I suppose you know that passing objects to functions or methods is a
trivial thing in ISO C++!






Best regards,

Ioannis Vranos

 

[Brandon Bray [MSFT]]

Suppose we have a list of some objects of this type and we have defined
comparison operators which in essence compare the money amounts of each
account, and we want to sort them based on those money with a sort
function (which could be the std::sort).

It's useful to think of pinning as a dangerous operation. In addition to
fragility of holding pointers after they are no longer pinned, pinning also
introduce sand bars to the compaction algorithm which ultimately leads to
the garbage collector running more often which is not a good thing. (I could
spend quite a while describing how running the garbage collector too often
severely lowers the performance of your program.)

So why we can't pass these objects and only their members? This is a
rational thing to do and possible with managed extensions!

Pinning is meant to deal with data that should not move in memory because
native code is operating on that data. The only data that native code can
operate on is value types.

Using the STL algorithms is absolutely something that we want to support in
general. In fact, that is something we are planning on officially supporting
with the STL.NET library. The only difference between the standard
algorithms and the ones needed for managed programming is the usage of % to
bind references on the stack. No pinning is necessary at all.

I suppose you know that passing objects to functions or methods is a
trivial thing in ISO C++!

Of course, I do. However, pinning is not something that should be done
lightly. We designed a syntax and semantics that makes pinning rarer. C++
and all the new features in new C++ syntax support passing objects to
methods and binding references, and the new syntax even provides much better
support than the old syntax.

 

[Ioannis Vranos]

It's useful to think of pinning as a dangerous operation. In addition to
fragility of holding pointers after they are no longer pinned, pinning also
introduce sand bars to the compaction algorithm which ultimately leads to
the garbage collector running more often which is not a good thing. (I could
spend quite a while describing how running the garbage collector too often
severely lowers the performance of your program.)

OK, I understand that pinning is an inconvenient situation, however
since it is supported (and since the entire managed objects get pinned
anyway), why not being supported for the entire ref objects?


This could make the use of existing unmanaged facilities easier, like
the C++ standard library.

Pinning is meant to deal with data that should not move in memory because
native code is operating on that data. The only data that native code can
operate on is value types.

Not only. As I said previously in managed extensions you can do:


__gc class whatever
{
// ...
};


template<class T>
void somefun(T *p)
{
// ...
};



int main()
{
whatever *p=__gc new whatever;

whatever __pin *pp=p;

somefun(pp);

// ...
}

Using the STL algorithms is absolutely something that we want to support in
general. In fact, that is something we are planning on officially supporting
with the STL.NET library. The only difference between the standard
algorithms and the ones needed for managed programming is the usage of % to
bind references on the stack. No pinning is necessary at all.

Ok then, since pinning is such a nightmare, why can't you make
interior_ptr to work for managed types at least, since it does not
involve pinning?

Of course, I do. However, pinning is not something that should be done
lightly. We designed a syntax and semantics that makes pinning rarer. C++
and all the new features in new C++ syntax support passing objects to
methods and binding references, and the new syntax even provides much better
support than the old syntax.

However functions taking pointers can't be used, while it was possible
with managed extensions! At least for template functions taking T*, this
should be possible.






Best regards,

Ioannis Vranos

 

[Brandon Bray [MSFT]]

Not only. As I said previously in managed extensions you can do:

__gc class whatever
{
// ...
};

template<class T>
void somefun(T *p)
{
// ...
};

int main()
{
whatever *p=__gc new whatever;
whatever __pin *pp=p;
somefun(pp);
// ...
}

You can do the following instead:

ref class whatever { /*...*/ };

template<class T>
void somefun(T p) { /*...*/ }

template<class T>
void somefun(T* p) { /*...*/ }

Simply add an overload that does not take pointers. The other overload can
even do pointer operations and deduce handle, interior_ptr, etc. (You could
even conceive of removing the T* overload and just leave the T overload.

Ok then, since pinning is such a nightmare, why can't you make
interior_ptr to work for managed types at least, since it does not
involve pinning?

You don't need pointers to these whole objects. Many of the STL container
algorithms work on things other than pointers, including sort. The only
problem is that they try to bind references, which is not allowed unless the
value does not move in memory. Replacing the references with tracking
references (%) is the solution.

However functions taking pointers can't be used, while it was possible
with managed extensions! At least for template functions taking T*, this
should be possible.

The functions taking pointers shouldn't be taking pointers. They should be
taking handles. A template function that takes just a T (not T* or T^) can
still use pointer algebra (dereference, comparison, etc.) and can take both
a pointer and handle. Alternatively, you can partially specialize the
templates so they take a handle instead and do something different.

Hope that makes sense!

 

[Ioannis Vranos]

Simply add an overload that does not take pointers. The other overload can
even do pointer operations and deduce handle, interior_ptr, etc. (You could
even conceive of removing the T* overload and just leave the T overload.

Well templates taking handles as parameters work too. I am talking
mainly about unmanaged code, with pointer specialisations performing
operations on pointers/iterators like std::sort.

I know, MS will write standard library specialisations for handles,
however I am not MS-specific, and even in this case in the mean time we
could use these conversions to get the job done until MS produces
specialisations for all the standard library facilities.

You don't need pointers to these whole objects. Many of the STL container
algorithms work on things other than pointers, including sort.

Can you give an example with std::sort in particular?

The functions taking pointers shouldn't be taking pointers. They should be
taking handles. A template function that takes just a T (not T* or T^) can
still use pointer algebra (dereference, comparison, etc.) and can take both
a pointer and handle. Alternatively, you can partially specialize the
templates so they take a handle instead and do something different.

So the point is we can't work in mixed mode that easily any more!






Best regards,

Ioannis Vranos

 

[Brandon Bray [MSFT]]

Well templates taking handles as parameters work too. I am talking
mainly about unmanaged code, with pointer specialisations performing
operations on pointers/iterators like std::sort.

As sort works on a collection that has iterators, to make it work you can
use iterators that need not be pointers. Since template code is not
unmanaged unless you use the #pragma unmanaged facility, you will not have
unmanaged code. As such, pinning is a really bad idea.

Templates that are designed around pointers are not going to work for
handles. In the past, it was possible to reuse the templates for __gc
pointers by first pinning those pointers. That is a practice we want to
discourage.

I know, MS will write standard library specialisations for handles,
however I am not MS-specific, and even in this case in the mean time we
could use these conversions to get the job done until MS produces
specialisations for all the standard library facilities.

We're actually not writing specializations of the standard algorithms for
handles. There's no need to do so. The only change that needs to take place
is changing & to % in the _very few_ places where references are used.
Otherwise, there are no changes at all.

All of the standard algorithms work on iterators. It happens that in many
cases a pointer matches the semantics of an iterator, but an iterator does
not need to be a pointer. So any standard algorithm that requires a pointer
as an iterator is wrong.

This is why I suggested removing the overload that required a pointer.
Instead, you can just deduce the type and make sure it has the same algebra
as an iterator. Handles for the most part can do everything you want to do
with pointers (equality, assignment, dereference).

Can you give an example with std::sort in particular?

Not readily. Sort works on containers, and I don't have any managed
containers at hand that I can dump into a newsgroup message. When we get
further along with Visual C++ 2005, you'll be able to see the STL.NET
library. It introduces a number of new collections that are built on top of
ref classes, but all the template functions used for algorithms have very
little change.

So the point is we can't work in mixed mode that easily any more!

That's not quite correct. Mixed mode is better than ever. We have made it
more difficult to do some things that you should really strive to avoid at
all lengths. Pinning unnecessarily is one of those things you should avoid.
The template code you're using is not unmanaged code, so you should not pin.
It's better to look at the template code to see if it is making the right
assumptions.