Unsafe native methods

[Andrew Jarvis]

I have a VC++/CLR method that calls some assembly code.

At a point where the assembly codes tries to load the ebp register with a
value, execution switches to a function called
System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop
never to return.

What do I have to do to allow my code to execute unmolested?

 

[Pavel A.]

I have a VC++/CLR method that calls some assembly code.

At a point where the assembly codes tries to load the ebp register with a
value, execution switches to a function called
System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop
never to return.

What do I have to do to allow my code to execute unmolested?

Save and restore registers correctly? Tell whoever wrote these assembly
codes to fix them?

--pa

 

[Andrew Jarvis]

Tell whoever wrote these assembly codes to fix them?

I wrote the code!

These are the first few instructions...

======
_SWExportPixelsB PROC

push ebx
push edi
push esi
push ebp

cld
mov esi,[esp+srcData]
mov edi,[esp+dstData]
mov ebp,[esp+lutData]
======

The exception happens on the last line - mov ebp,[esp+lutData].
Since the routine exits abnormally, it would not matter if the registers
were not saved.

If I use ebx instead of ebp, its works fine but then ebx has two roles and
additional loading and saving is required.

AJ

 

[Andrew Jarvis]

I should emphasise that my code performs correctly.
It is just that it takes a detour via the 'unsafe method' trap. I don't know
exactly what is happening here, but the trap eventually returns and
execution continues.
I wanted to know if there is a way to stop these checks. I found the
SuppressUnmanagedCodeSecurity attribute but my code is being called during a
message handler so the managed/native transition seems beyond my control.

 

[Pavel A.]

Tell whoever wrote these assembly codes to fix them?

I wrote the code!

These are the first few instructions...

======
_SWExportPixelsB PROC

push ebx
push edi
push esi
push ebp

cld
mov esi,[esp+srcData]
mov edi,[esp+dstData]
mov ebp,[esp+lutData]
======

The exception happens on the last line - mov ebp,[esp+lutData].
Since the routine exits abnormally, it would not matter if the registers
were not saved.

If I use ebx instead of ebp, its works fine but then ebx has two roles and
additional loading and saving is required.

AJ

Do you single step in a debugger? If so, which debugger?
Maybe some other debugger won't be confused by unexpected change of ebp...

--pa