BUG: CallingConvention not followed

  • Thread starter Thread starter Fernando Cacciola
  • Start date Start date
F

Fernando Cacciola

I have the following C# declaration:

[DllImport("mylib.dll", CallingConvention = CallingConvention.StdCall)]
extern static bool SomeFunc( IntPtr a
,int [] b
,byte [] c
,PointF[] d
);

which calls into some unmanaged C++ function.

I'm using VS.NET 2003

The dissasembly output shows:

000000b9 push dword ptr [ebp-1Ch]
000000bc push dword ptr [ebp-14h]
000000bf mov edx,dword ptr [ebp-18h]
000000c2 mov ecx,dword ptr [ebp-4]
000000c5 call dword ptr ds:[09092E40h]

But this is wrong!!!!
only the last two parameters are being pushed onto the stack
the other two are passed in the edx and ecx registers.

naturally, it crashes, because the C++
functions expected the four arguments in the
stack.

I noticed that this has been reported here before:

http://tinyurl.com/4mgak

TIA

Fernando Cacciola
 
Fernando,
The dissasembly output shows:

000000b9 push dword ptr [ebp-1Ch]
000000bc push dword ptr [ebp-14h]
000000bf mov edx,dword ptr [ebp-18h]
000000c2 mov ecx,dword ptr [ebp-4]
000000c5 call dword ptr ds:[09092E40h]

But this is wrong!!!!
only the last two parameters are being pushed onto the stack
the other two are passed in the edx and ecx registers.
Click to expand...

Keep in mind that all P/Invoke calls go through some thunk code that
handles marshalling and stuff. So the stack that's set up here isn't
necessarily the same as what it will look like when the actual native
function is called. The CLR tends to use the __fastcall calling
convention internally, so the assembler looks correct to me.

If calls to __stdcall functions were this fundamentally broken, I
think it would have been found a long time ago, because they are used
everywhere. Most of the Win32 API, on which the BCL depends heavily,
use __stdcall.

naturally, it crashes, because the C++
Click to expand...

Perhaps you should look for other reasons for the crash.



Mattias
 
Mattias Sjögren said:
Fernando,
The dissasembly output shows:

000000b9 push dword ptr [ebp-1Ch]
000000bc push dword ptr [ebp-14h]
000000bf mov edx,dword ptr [ebp-18h]
000000c2 mov ecx,dword ptr [ebp-4]
000000c5 call dword ptr ds:[09092E40h]

But this is wrong!!!!
only the last two parameters are being pushed onto the stack
the other two are passed in the edx and ecx registers.
Click to expand...

Keep in mind that all P/Invoke calls go through some thunk code that
handles marshalling and stuff. So the stack that's set up here isn't
necessarily the same as what it will look like when the actual native
function is called. The CLR tends to use the __fastcall calling
convention internally, so the assembler looks correct to me.
Click to expand...
Indeed...
I watched the stack memory contents, stepped into the "call" assembler
instruction, and there, right at the "push bp" of the C++ function prolog,
the stack suddenly appeared with a lot more stuff than what it had "just 1
instruction before".
Clearly, the Visual Studio dissasembly window is shortcutting and hiding
code... the relevant parts of the code in this case, unfortunately.
If calls to __stdcall functions were this fundamentally broken, I
think it would have been found a long time ago, because they are used
everywhere. Most of the Win32 API, on which the BCL depends heavily,
use __stdcall.
Click to expand...
Right... I thought it had to do with this very particular function, but then
I tried with a simple test function and saw the same pattern.
Unfortunately, I can't see the "real code"...
Is there any "lower" level debugger I can use? or even better, any
documentation on the _details_ of P/Invoke?
TIA
Perhaps you should look for other reasons for the crash.
Click to expand...
Well, for whatever reason it worked when I added [Out] to the parameter:
PointF[] d

In the C++ side this is: PointF const* d

Thanks

Fernando Cacciola
SciSoft
 
Fernando,
Unfortunately, I can't see the "real code"...
Is there any "lower" level debugger I can use?
Click to expand...

VS can do this, it just takes some extra work. What I usually do when
I want to step through the CLR code is to call DebugBreak() to attach
to the debugger, and make sure I select to only use native debugging.



Mattias
 
Mattias Sjögren said:
Fernando,


VS can do this, it just takes some extra work. What I usually do when
I want to step through the CLR code is to call DebugBreak() to attach
to the debugger, and make sure I select to only use native debugging.
Click to expand...
Oh, thanks!

Fernando Cacciola
SciSoft
 
Back
Top