Why can not hook CloseHandle() file system API use detours method?

  • Thread starter Thread starter Vic.Dong
  • Start date Start date
V

Vic.Dong

Hello All:

I use detours method of MS kit to hook CloseHandle() file systme API and
inject this hook.dll into NotePad process by static registry key value:
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows
NT\CurrentVersion\Windows\AppInit_DLLs with one INI file that specified
only one NotePad application below is:

INI file:
[Scope]
UseWindowsHook=Yes
HookAll=No
Hook=notepad
Protect=MSDEV
[Trace]
Enabled=Yes

In my hook.dll:
use detours function:
DetourFunctionWithTrampoline((PBYTE)Real_CloseHandle,(PBYTE)Mine_CloseHandle);
DetourRemove((PBYTE)Real_CloseHandle,
(PBYTE)Mine_CloseHandle);
.....
based MS detours sample.

But I found when my hooked dll was loaded in NotePad process that TXT file
can not be opend, and appears a lot of trace string in my Mine_CloseHandle()
function. But another function e.g.: CreatFile(), ReadFile() .... can be
hooked and open TXT file.

Why can not I hook CloseHandle() API?

B.R.

Vic
 
I do not know your code, but what I suspect:

1. CloseHandle is called SIGNIFICANTLY more frequently that CreateFile
function. CloseHandle may be call after OpenProcess, DuplicateHandle,
CreateFileMapping, CreateFileForMapping and ...
It is why "appears a lot of trace string in my"

2. Also I suspect that DetourFunctionWithTrampoline substitute the
function addresses only when it was called. However notepad in some
situation (for example, when you open file dialog) loads new DLLs and I
think it is necessary to update substitutions in those mew loaded DLLs.

If you run NOTEPAD under standard MSDEV debugger you may see when
notepad loads or unloads DLLs.

My opinion is instead of using DetourFunctionWithTrampoline implement
your debugger application. Put break points at CreateFile, CloseHandle
and so on and hook all these functions. It is simple because
kernel32.dll is not loaded/unloaded dynamically.
I am doing hooking in this way in my product at http://ircdb.org
Other example of hooking is at http://smike.ru
 
Back
Top