x64 and BSTR allocation, what has changed?

  • Thread starter Thread starter Egbert Nierop \(MVP for IIS\)
  • Start date Start date
E

Egbert Nierop \(MVP for IIS\)

In win32 mode, a BSTR was UINT length prefixed and terminated exactly by a
zero char.

So if you allocated "Hello World" that would allocate 28 bytes.

In x64 and (IA64 as well) it would become 32 bytes, because of the fact that
a pointer, is 8 bytes instead of 4 bytes.

The length prefix -still- is a UINT however and not a UINT_PTR.

But it seems that I'm still not quite complete on par. Is there any other
info or rules that have changed on top off the win32 BSTR allocations?

Thanks!
 
In win32 mode, a BSTR was UINT length prefixed and terminated exactly by a
zero char.

So if you allocated "Hello World" that would allocate 28 bytes.

In x64 and (IA64 as well) it would become 32 bytes, because of the fact that
a pointer, is 8 bytes instead of 4 bytes.
Click to expand...

How do you come to that conclusion? Where does a pointer come into a
BSTR?

Dave
 
David Lowndes said:
How do you come to that conclusion? Where does a pointer come into a
BSTR?
Click to expand...
ok,

I'll explain this.
Not the pointer -itself- comes into but the pointer is to the UINT length
prefix.

On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
UINT_PTR length prefix.

But but... however, if you want the length, you need a UINT prefix, not a
UINT_PTR prefix.

I'm not stating this, just hesitating. Maybe someone knows the real facts...

BSTR __stdcall SysAllocString(const OLECHAR * input)
{
PWSTR retval = NULL;

if (input != NULL)
{
UINT_PTR slen = lstrlenW(input);
UINT_PTR newlen = slen * sizeof(OLECHAR)+ sizeof(UINT_PTR) +
sizeof(OLECHAR);
PWSTR temp = NULL;
temp = (PWSTR)::CoTaskMemAlloc(newlen);

if (temp != NULL)
{
UINT* plen = (UINT*)temp;
plen[0] = (UINT)slen * sizeof(OLECHAR);
retval = &temp[sizeof(UINT_PTR) / sizeof(OLECHAR)];
if (slen > 0)
CopyMemory(retval, input, (slen + 1) * sizeof(OLECHAR));
}
}
return retval;
}
 
How do you come to that conclusion? Where does a pointer come into a
ok,

I'll explain this.
Not the pointer -itself- comes into but the pointer is to the UINT length
prefix.

On X64 and IA64, the pointer is -not- to the UINT length prefix, but to a
UINT_PTR length prefix.
Click to expand...

So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
length portion rather than a 4 byte length? I'd be surprised if it
was.

I don't see any evidence for that assumption in the MSDN documentation
or the header files. SysStringLen is defined to return an UINT not
UINT_PTR.

Dave
 
Hi David!
So, you're saying that on 64-bit platforms, a BSTR has an 8 byte
length portion rather than a 4 byte length? I'd be surprised if it
was.
Click to expand...

A BSTR does not have a "length portion"... it is just a pointer...

Or did I miss something?

BSTR is just "wchar_t*"...

--
Greetings
Jochen

My blog about Win32 and .NET
http://blog.kalmbachnet.de/
 
Let's not forget that the memory must be 8-byte aligned.
I would guess it allocates 8 bytes for the length prefix, and
ignores the first four. And also let's not forget that the
allocation strategy for BSTR is officially undocumented...
Only the layout is documented.

--
=====================================
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: (e-mail address removed)
MVP VC FAQ: http://www.mvps.org/vcfaq
=====================================
 
Jochen Kalmbach said:
A BSTR does not have a "length portion"... it is just a pointer...
Click to expand...

Does too. The length in bytes is stored in the 4 bytes preceding the
byte referred to by the pointer. How do you expect SysStringLen to work
otherwise? Recall that BSTR may contain embedded NULs.
--
With best wishes,
Igor Tandetnik

With sufficient thrust, pigs fly just fine. However, this is not
necessarily a good idea. It is hard to be sure where they are going to
land, and it could be dangerous sitting under them as they fly
overhead. -- RFC 1925
 
Jochen Kalmbach said:
Hi David!

A BSTR does not have a "length portion"... it is just a pointer...

Or did I miss something?
Click to expand...

Yes, I'm saying that on 64-bit platforms, the length portion is 8 bytes!
But only the lower 4 bytes are effective.

I can prove this since I used a BSTR stub. This had special reasons. I had a
benchmark, that has proven that avoiding BSTR caching, was good on a Wintel
platform I did this by redefining all SysAlloc* strings to my own written
replacements.

On the 64-bit platform, this stub is working in a stand-alone EXE but not
inside a service and I can't figure why.
 
Alexander Nickolov said:
Let's not forget that the memory must be 8-byte aligned.
I would guess it allocates 8 bytes for the length prefix, and
ignores the first four. And also let's not forget that the
allocation strategy for BSTR is officially undocumented...
Only the layout is documented.
Click to expand...

Your guess is right about the 8 bytes length prefix.

The BSTR is just allocated by CoTaskMemAlloc and and CoTaskMemRealloc that
is using RtlHeap functions.

I have no source code, but my COM components, have been working for 3 years
on this. Except on the 64-bit platform that I am testing now.
 
Jochen Kalmbach said:
Hi David!

A BSTR does not have a "length portion"... it is just a pointer...

Or did I miss something?

BSTR is just "wchar_t*"...
Click to expand...

You missed a lot by thinking this :)
 
Oh, I know how it's allocated :). I'm just pointing out that
this is officially undocumented.

So in Win64 we have:

byte len content
-8 4 unused
-4 4 length
0 n string

Or did I miss something?

--
=====================================
Alexander Nickolov
Microsoft MVP [VC], MCSD
email: (e-mail address removed)
MVP VC FAQ: http://www.mvps.org/vcfaq
=====================================
 
Let's not forget that the memory must be 8-byte aligned.
I would guess it allocates 8 bytes for the length prefix, and
ignores the first four.
Click to expand...

OK.

.... so why does this matter to Egbert?

Dave
 
Alexander Nickolov said:
Oh, I know how it's allocated :). I'm just pointing out that
this is officially undocumented.

So in Win64 we have:

byte len content
-8 4 unused
-4 4 length
0 n string
Click to expand...
and add a '\0' after the n string.
(unicode)


Or did I miss something?
Click to expand...

It's working. I had still somewhere an invalid memory location calculation
(-4 instead of -8)

#include "stdafx.h"
#include "bstrnocache.h"

void __stdcall FreeString(BSTR * theString) throw()
{
if (theString != NULL && *theString != NULL )
{
SysFreeString2(*theString);
*theString = NULL;
}
}

int __stdcall SysReAllocStringByteLen2(BSTR * bstr, const char* input, UINT
cch) throw()
{
// ATLASSERT(bstr != NULL);
if (*bstr == NULL)
*bstr = SysAllocStringByteLen(input, cch);
else
{
UINT copyIntoLen = input != NULL ? lstrlenA(input): 0;
if (copyIntoLen > cch)
copyIntoLen = cch;
UINT_PTR newlen = cch + sizeof(OLECHAR) + sizeof(UINT_PTR);

PCHAR temp = (PCHAR)(*bstr) - sizeof(UINT_PTR);

temp = (PCHAR)::CoTaskMemRealloc(temp, newlen);

if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrection] = cch ; //asign bytelength for BSTR
if (copyIntoLen > 0) //copy chars
memcpy(&plen[1],
input,
(copyIntoLen + 1));

temp[sizeof(UINT_PTR) + cch ] = 0; //terminate LPWSTR with a wide_string
temp[sizeof(UINT_PTR) + cch + 1] = 0;
*bstr = (BSTR)&temp[sizeof(UINT_PTR)];
}
}
return *bstr == NULL ? FALSE : TRUE;
}
int __stdcall SysReAllocStringLen2(BSTR * bstr, const OLECHAR* input, UINT
cch) throw()
{
//ATLASSERT(bstr != NULL);
if (*bstr == NULL)
{
*bstr = SysAllocStringLen2(input, cch);
}
else
{
UINT_PTR copyIntoLen = input != NULL ? lstrlenW(input): 0;
if (copyIntoLen > cch)
copyIntoLen = cch;
UINT_PTR newlen = cch * sizeof(OLECHAR) + sizeof(OLECHAR) +
sizeof(UINT_PTR);

PWSTR temp = *bstr - sizeof(UINT_PTR) / sizeof(OLECHAR);

temp = (PWSTR)::CoTaskMemRealloc(temp, newlen);

if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrection] = cch * sizeof(OLECHAR); //asign bytelength for
BSTR
if (copyIntoLen > 0) //copy values
memcpy(&temp[sizeof(UINT_PTR) / sizeof(OLECHAR)],
input,
(copyIntoLen + 1) * sizeof(OLECHAR));

temp[sizeof(UINT_PTR) / sizeof(OLECHAR) + cch ] = 0; //terminate LPWSTR
*bstr = &temp[sizeof(UINT_PTR) / sizeof(OLECHAR)];

}

}
return *bstr == NULL ? FALSE : TRUE;
}
BSTR __stdcall SysAllocStringLen2(const OLECHAR * input, UINT cch) throw()
{
UINT_PTR copyIntoLen = input != NULL ? lstrlenW(input) : 0;
if (copyIntoLen > cch)
copyIntoLen = cch;

UINT_PTR newlen = cch * sizeof(OLECHAR) + sizeof(OLECHAR) +
sizeof(UINT_PTR);

PWSTR temp = (PWSTR)::CoTaskMemAlloc(newlen);
if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp;
UINT*plen2 = (UINT*)temp;
plen2[0] =0;
plen2[sizePtrCorrection] = cch * sizeof(OLECHAR);
if (copyIntoLen > 0)
memcpy(&plen[1],
input,
(copyIntoLen + 1) * sizeof(OLECHAR));

temp[sizeof(UINT_PTR) / sizeof(OLECHAR) + cch ] = 0;
return &temp[sizeof(UINT_PTR) / sizeof(OLECHAR)];
}
else
return NULL;

}

BSTR __stdcall SysAllocStringByteLen2(const char* input, UINT cch)
throw()
{
UINT_PTR copyIntoLen = input != NULL ? lstrlenA(input) : 0;
if (copyIntoLen > cch)
copyIntoLen = cch;

UINT_PTR newlen = cch + sizeof(OLECHAR) + sizeof(UINT_PTR);

PWSTR temp = (PWSTR)::CoTaskMemAlloc(newlen);
if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrection] = cch;
if (copyIntoLen > 0)
memcpy(&temp[sizeof(UINT_PTR) / sizeof(OLECHAR)],
input,
copyIntoLen + 1);
char* zeroit = (char*)temp;
zeroit[sizeof(UINT_PTR) + cch ] = 0; //terminate LPWSTR with a wide_string
zeroit[sizeof(UINT_PTR) + cch + 1] = 0;
//temp[sizeof(unsigned int) / sizeof(wchar_t) + cch ] = 0;
return &temp[sizeof(UINT_PTR) / sizeof(OLECHAR)];
}
else
return NULL;

}


///<summary>
/// Identical to SysAllocString
///</summary>
BSTR __stdcall SysAllocString2(const OLECHAR * input) throw()
{
PWSTR retval = NULL;

if (input != NULL)
{
UINT_PTR slen = lstrlenW(input);
UINT_PTR newlen = slen * sizeof(OLECHAR)+ sizeof(UINT_PTR) +
sizeof(OLECHAR);
PWSTR temp = NULL;
temp = (PWSTR)::CoTaskMemAlloc(newlen);

if (temp != NULL)
{
UINT* plen = (UINT*)temp;
plen[sizePtrCorrection] = (UINT)slen * sizeof(OLECHAR);
retval = &temp[sizeof(UINT_PTR) / sizeof(OLECHAR)];
if (slen > 0)
memcpy(retval, input, (slen + 1) * sizeof(OLECHAR));
}
}
return retval;
}

int __stdcall SysReAllocString2(BSTR * bstr, const OLECHAR * input) throw()
{
//ATLASSERT(bstr != NULL);
bool didEmpty = FALSE;
if (*bstr == NULL)
{
*bstr = SysAllocString2(input);
}
else
{

if (input == NULL)
{
FreeString(bstr);
didEmpty = true;
}
else
{
UINT_PTR slen = lstrlenW(input);
UINT_PTR newlen = slen * sizeof(OLECHAR) + sizeof(OLECHAR) +
sizeof(UINT_PTR);

BSTR temp = *bstr - sizeof(INT_PTR) / sizeof(OLECHAR); //get real address
temp = (BSTR)::CoTaskMemRealloc(temp, newlen);

if (temp != NULL)
{
UINT_PTR* plen = (UINT_PTR*)temp;
UINT* plen2 = (UINT*)temp;
plen2[sizePtrCorrection] = (UINT)slen * sizeof(OLECHAR);
*bstr = &temp[sizeof(UINT_PTR) / sizeof(OLECHAR)];
memcpy(*bstr, input, (slen + 1) * sizeof(OLECHAR));
}
}

}
return *bstr == NULL && didEmpty == false? FALSE : TRUE;
}

UINT __stdcall SysStringByteLen2(BSTR theString) throw()
{
UINT retval = 0;
if (theString != NULL)
{
UINT* temp = (UINT*)theString;
temp--;
retval = *temp;
}
return retval;

}

UINT __stdcall SysStringLen2(BSTR theString) throw()
{
UINT retval = 0;
if (theString != NULL)
{
UINT* temp = (UINT*)theString;
temp--;
retval = *temp / sizeof(OLECHAR);
}
return retval;
}
 
Hi Egbert!
You missed a lot by thinking this :)
Click to expand...

No problem for me... I never rely on undecomented features...

(As "Alexander said: "And also let's not forget that the
allocation strategy for BSTR is officially undocumented...")

Greetings
Jochen
 
David Lowndes said:
OK.

... so why does this matter to Egbert?
Click to expand...

The prefix was concernig my question, the alignment not.

CoTaskMemAlloc nowhere states in the docs that you must 'align' memory. I
think that RtlHeap* etc do align the allocations by themselves but I'm not
that guru to know this.
 
Jochen Kalmbach said:
Hi Egbert!


No problem for me... I never rely on undecomented features...
Click to expand...

Sure, it's your party, but BSTR is _not_ just a wchar_t* is documented!

(As "Alexander said: "And also let's not forget that the
allocation strategy for BSTR is officially undocumented...")
Click to expand...

This is not rocket science :)
If you need an 11 wchar length string, what else must you store but the
length _and_ the compatibility with wchar_t* ie a zero terminator?

The other thing you need to know, is that all oleautomation memory allocs,
should go through CoTaskMemAlloc. Now I'm a rocket scientist :)

I started to hesitate on the x64 because of GPs but now I fixed it.
 
... so why does this matter to Egbert?
The prefix was concernig my question, the alignment not.
Click to expand...

I still don't see why any of this was of any significance to you. What
are you doing that you need to know this for?

Dave
 
Jochen said:
No problem for me... I never rely on undecomented features...
Click to expand...

But you do realize that you must create, manipulate and destroy a BSTR
only in the documented ways, essentially using the Sysxxx functions? You
can provide a BSTR in instances where a LPCWSTR is expected, but not the
other way round. The preceding length value is an implementation detail,
and therefore we should not make any assumptions -- agreed. But if you
fail to remember that a BSTR is /not/ just a wchar_t array, you'll get
into trouble.
 
Back
Top