IntPtr != HANDLE conversion problem

  • Thread starter Thread starter Shawn B.
  • Start date Start date
S

Shawn B.

Greetings,

Me again.


I have (roughly) the following code:

HANDLE hConsoleOutput;
HANDLE hConsoleInput;

hConsoleOutput = GetStdHandle(STD_OUTPUT_HANDLE);
hConsoleInput = GetStdHandle(STD_INPUT_HANDLE);


after executing, each of the hConsole* variables contain 'void'. Yet, if I
declare them as IntPtr instead, they each contain a proper number. The
problem is, immediately after I call:

GetConsoleScreenBufferInfo(hConsoleOutput, &ConsoleInfo);

And one of two things happens:

1) If hConsole* variables declared as HANDLE, then I get a first-chance
exception.
2) If hConsole* variables declared as IntPtr, I get a
'GetConsoleScreenBufferInfo' : cannot convert parameter 1 from
'System::IntPtr __gc *' to 'HANDLE'
compile error.

(laughing out loud) I thought "it just works".

Anyway, how can I coerce an IntPtr into a HANDLE pararameter on an API
function? Once again, Google hasn't provided any obvious answers.


Thanks,
Shawn
 
Shawn B. said:
I have (roughly) the following code:

HANDLE hConsoleOutput;
HANDLE hConsoleInput;

hConsoleOutput = GetStdHandle(STD_OUTPUT_HANDLE);
hConsoleInput = GetStdHandle(STD_INPUT_HANDLE);

after executing, each of the hConsole* variables contain 'void'. Yet, if
I declare them as IntPtr instead, they each contain a proper number. The
problem is, immediately after I call:
Click to expand...
Handles are defined as pointers even though they are not. It's only
an abstraction used by the OS API.

You should really use HANDLE.
GetConsoleScreenBufferInfo(hConsoleOutput, &ConsoleInfo);

1) If hConsole* variables declared as HANDLE, then I get a first-chance
exception.
Click to expand...
A first-chance exception is not necessarily a bad thing.

That's how the OS Exception Handling interacts with the debugger:
- when an exception is raised (either by software or by hardware)
the OS first notifies the debugger that an exception occured
(first-chance)
- The debugger passes back control to the debuggee
(VC continues by default)
- If the process handles the exception everything's fine.
If not the OS notifies the debugger again.
(VC does not continue the process)

So for
try { throw 1; }catch ( int ) {}
You'd get a first-chance exception even though everything's fine.

-hg
 
Handles are defined as pointers even though they are not. It's only
an abstraction used by the OS API.

You should really use HANDLE.

A first-chance exception is not necessarily a bad thing.

That's how the OS Exception Handling interacts with the debugger:
- when an exception is raised (either by software or by hardware)
the OS first notifies the debugger that an exception occured
(first-chance)
- The debugger passes back control to the debuggee
(VC continues by default)
- If the process handles the exception everything's fine.
If not the OS notifies the debugger again.
(VC does not continue the process)

So for
try { throw 1; }catch ( int ) {}
You'd get a first-chance exception even though everything's fine.
Click to expand...

Thanks for the reply. The problem is, that everytime I run this, I'm
running without a debugger or in Release mode, and it always displays a
popup saying that the Just-in-time debugger could not be loaded and then
closes my console. In any case, the functionality doesn't work as intended,
especially when I encounter GetConsoleMode(hConsoleInput, (DWORD *)mode); it
really craps out the application because hConsoleMode is <void> (is my
guess).

In any case, there's a deeper problem that I need to figure out how to
solve. When I call GetConsoleScreeBufferInfo(hConsoleOutput, &ConsoleInfo)
in unmanaged C++, it returns the proper handle. When I call it from Managed
C++, the exact same lines of code return void. When I use IntPtr, I get the
proper handle, but I can't pass it into any API calls because I don't know
how to convert them into HANDLE, which is what the SDK header files expect.
I could redeclare the functions using DllImport, but I'm hoping to learn how
to solve the problem rather than just working around it. I'd rather
understand what I'm doing and get to the heart of it.


Thanks,
Shawn
 
Thanks for the reply. The problem is, that everytime I run this, I'm
running without a debugger or in Release mode, and it always displays a
popup saying that the Just-in-time debugger could not be loaded and then
Click to expand...
That's an unhandled exception (a real problem).
closes my console. In any case, the functionality doesn't work as
intended, especially when I encounter GetConsoleMode(hConsoleInput, (DWORD
*)mode); it really craps out the application because hConsoleMode is
<void> (is my guess).
Click to expand...
What do you mean with "hConsoleMode is <void>"? void is a type
not a value. Do you see that in the debugger? Is that optimized
code?
In any case, there's a deeper problem that I need to figure out how to
solve. When I call GetConsoleScreeBufferInfo(hConsoleOutput,
&ConsoleInfo) in unmanaged C++, it returns the proper handle. When I call
it from Managed C++, the exact same lines of code return void. When I use
IntPtr, I get the proper handle, but I can't pass it into any API calls
because I don't know how to convert them into HANDLE, which is what the
SDK header files expect.
Click to expand...

I'm afraid I don't quite understand. How is a handle returned? I don't see
that
in the interface contract. BTW: You can always use C-style casts in the
expression evaluator. E.g. QuickWatch-> "(int)hConsoleMode,x"
or "(void*)hConsoleMode".

-hg
 
Thanks for the reply. The problem is, that everytime I run this, I'm
That's an unhandled exception (a real problem).
Click to expand...
Yes. But one that only happens with managed code, the exact same code in an
unmanaged class works fine. I'm postulating, because of the reasons
explained below.
What do you mean with "hConsoleMode is <void>"? void is a type
not a value. Do you see that in the debugger? Is that optimized
code?
Click to expand...
When I place a break point in the managed class function, and inspect the
value of hConsoleInput immediately after the line of code:

hConsoleOutput = GetStdHandle(STD_OUTPUT_HANDLE);
hConsoleInput = GetStdHandle(STD_INPUT_HANDLE);

, after inspecting the values, hConsoleOutput = {void} and hConsoleInput =
{void}. Yes, this is during a breakpoint, in debug mode.
I'm afraid I don't quite understand. How is a handle returned? I don't see
that
in the interface contract. BTW: You can always use C-style casts in the
expression evaluator. E.g. QuickWatch-> "(int)hConsoleMode,x"
or "(void*)hConsoleMode".
Click to expand...

As I explained above... if I place the code in an unmanged function:

hConsoleOutput = GetStdHandle(STD_OUTPUT_HANDLE);
hConsoleInput = GetStdHandle(STD_INPUT_HANDLE);

and then inspect the contents of hConsoleOutput, it reads 0x00000003, and
hConsoleInput reads 0x00000007. When I place those exact lines of code in a
managed function, the contents of each, after inspecting them (by hovering
the mouse over the variable during a breakpoint) (and via QuickWatch) reads
"{void}" instead of the hex number returned when the code is in an unmanaged
fuction.

As far as the casting is conserned, the casting doesn't solve the problem.
When I try your "(void *)hConsoleMode" anywhere, including as a quickwatch
expression, it crashes the IDE and restarts it. When I try
(int)hConsoleMode, it says "Error: (int) cannot be performed on a {void}.

Thus, for some reason, compiling it in a managed class changes its
behaviors. I don't know why, but I need to solve the problem.


Thanks,
Shawn
 
I don't seem to have problems running the functions from managed code. This
may be a dumb question, but do you have a console associated with your
process using something like /SUBSYSTEM:CONSOLE in linker options or calling
AllocConsole() or AttachConsole()?
 
James Park said:
I don't seem to have problems running the functions from managed code. This
may be a dumb question, but do you have a console associated with your
process using something like /SUBSYSTEM:CONSOLE in linker options or
calling AllocConsole() or AttachConsole()?
Click to expand...

The console is actually a C# Console application, calling on the managed
code.


I'd like to know how it works for you, because I've tried on 5 different
machines with each very different characteristics between them, and it is a
consistent problem. One other option, I could post the project file so you
can examine the unique circumstance of my project?


Thanks,
Shawn
 
I don't seem to have problems running the functions from managed code. This
may be a dumb question, but do you have a console associated with your
process using something like /SUBSYSTEM:CONSOLE in linker options or
calling AllocConsole() or AttachConsole()?
Click to expand...

If I create this class in unmanaged code, it works perfect, like a champ.
Only when I make the API calls (exact same copy-paste code) in a managed
class, do I have the problem.

I could try to just have the managed class call the unmanaged class and have
everything work, but then I start running into all the other problems... I
pass in an int or IntPtr or StringBuilder through my managed functions, I
still have to convert them into something the unmanaged API header file
understands. One way or another, this doesn't seem to be working with
managed code.


Thanks,
Shawn
 
The projects appears to be working fine for me (no exceptions). Even though
it's showing {void} for hConsoleInput/Output, they are indeed being assigned
correctly.

On another subject, the reason why the one commented line isn't working is
because you're trying to use a mananged pointer in place of an unmanaged
one. One way you can go about fixing that is to use a pinning pointer
instead:

CONSOLE_SCREEN_BUFFER_INFO __pin * pCool = &ConsoleInfo;
 
Back
Top