Windows hook function knowledge and Assembly example

Windows hook functions can be considered as one of the main features of windows. They allow you to capture events that occur in your own processes or other processes. With "hook-up", you can give Windows a callback function to process or filter events. This function is also called "hook function". When an event you are interested in occurs, windows will call this function. There are two types of hooks: local and remote.

• Local hooks only hook events of your own processes.
• Remote hooks can also hook events that occur in other processes. There are two remote hooks:
  • Based on the thread, it will capture events of a specific thread in other processes. In short, it can be used to observe the events that will happen to a specific thread in other processes.
  • System-wide event messages will be captured by all processes in the system.

Installing the hook function will affect the system performance. The system hooks for monitoring "system-scope events" are particularly obvious. Because the system calls your hook function when processing all related events, your system will obviously slow down. Therefore, use it with caution and uninstall it immediately after use. In addition, because you can intercept messages from other processes in advance, once your hook function has a problem, it will certainly affect other processes. Remember: being powerful also means taking responsibility for use.
Before correctly using the hook function, we will first explain how the hook function works. When you create a hook, Windows will first create a data structure in the memory, which contains information about the hook, then add the struct to an existing hook linked list. The new hook is added to the front of the old one. When an event occurs, if you install a local Hook, the hook function in your process will be called. If it is a remote Hook, the system must insert the hook function into the address space of other processes. To achieve this, the hook function must be in a dynamic link library, if you want to use a remote Hook, you must put the hook function in the dynamic link library. Of course, there are two exceptions: The work Log Hook and the work log playback hook. The hook functions of these two hooks must be in the thread where the hooks are installed. The reason is: These two hooks are used to monitor relatively low-level hardware events. Since they are recorded and played back, all events are of course sequential. Therefore, if the callback function is placed in the DLL and the input events are recorded in several threads, we cannot guarantee the correct order. The solution is to put the hook function in a single thread, such as the thread for installing the hook.
There are a total of 14 types of hooks. The following are the times when they are called:

• Wh_callwndproc when sendmessage is called
• Wh_callwndprocret when sendmessage is returned
• Wh_getmessage when getmessage or peekmessage is called
• Wh_keyboard when getmessage or peekmessage is called to query wm_keyup or wm_keydown messages from the Message Queue
• Wh_mouse when getmessage or peekmessage is called to query mouse event messages from the Message Queue
• Wh_hardware when getmessage or peekmessage is called to query non-mouse and keyboard messages from Message Queue types
• Wh_msgfilter: when a message is to be processed in a dialog box, menu, or scroll bar. This hook is local. It is designed for the control objects that have their own message processing processes.
• Wh_sysmsgfilter is the same as wh_msgfilter, but it is only within the system range.
• Wh_journalrecord when Windows receives messages from the hardware queue
• Wh_journalplayback when an event is requested from the system's hardware Input Queue
• Wh_shell when a Windows Shell event occurs, for example, a task bar needs to re-draw its button.
• Wh_cbt when a computer-based training (CBT) event occurs
• Wh_foregroundidle is used by windows, and is rarely used by general applications.
• Wh_debug is used to debug the hook function

Now we know some basic theories. Now we will explain how to install/uninstall a hook.
To install a hook, you can call the setjavaswhookex function. The function is prototype as follows:

Setwindowshookex proto hooktype: DWORD, phookproc: DWORD, hinstance: DWORD, threadid: DWORD

• Hooktype is one of the values listed above, such as wh_mouse and wh_keyboard.
• Phookproc is the address of the hook function. If a remote hook is used, it must be placed in a DLL; otherwise, it should be placed in its own code.
• The instance handle of the DLL where the hinstance hook function is located. If it is a local Hook, the value is null.
• Threadid is the ID of the thread you want to monitor after installing the hook function. This parameter determines whether the hook is local or system-wide. If the value is null, the hook will be interpreted as within the system range, so that it can monitor all processes and their threads. If you specify a thread ID in your own process, this hook is a local hook. If the thread ID is the ID of a thread in another process, the hook is a global remote hook. There are two special cases: wh_journalrecord and wh_journalplayback always represent local system range hooks. The reason is that they are local because they do not need to be placed in a DLL. Wh_sysmsgfilter is always a system-wide remote hook. In fact, it is similar to the wh_msgfilter hook. If the threadid parameter is set to 0, they are exactly the same.

If the function is successfully called, the hook handle is returned in eax; otherwise, null is returned. You must save the handle because we need it to uninstall the hook later.

To uninstall a hook, call the unhookwidowhookex function. This function has only one parameter, that is, the handle of the hook to be detached. If the call is successful, a non-0 value is returned in eax; otherwise, null is returned.
Now that you know how to install and uninstall a hook, let's take a look at the hook function ..
As long as the message event type of the hook you install occurs, Windows will call the hook function. For example, if the hook you installed is of the wh_mouse type, the hook function will be called when a mouse event occurs. Regardless of the hook type you installed, the hooks are prototype the same:

Hookproc proto ncode: DWORD, wparam: DWORD, lparam: DWORD
 
• Ncode specifies whether to process the message
• Wparam and lparam include additional messages for the message

Hookproc can be seen as a placeholder for a function name. As long as the function prototype is consistent, you can give the function any name. The meanings of the preceding parameters and returned values are different for different types of hooks. For example:

Wh_callwndproc

• Ncode can only be hc_action, which indicates that a message is sent to a window.
• If wparam is not 0, it indicates the message being sent.
• Lparam pointer to the cwpstruct struct variable
• Return Value: unused. 0 is returned.

Wh_mouse

• Ncode is hc_action or hc_noremove
• Wparam contains the mouse event message
• Lparam pointer to the mousehookstruct struct variable
• Return Value: If not processed, 0 is returned; otherwise, a non-0 value is returned.

Therefore, you must query your Win32 API guide to obtain detailed definitions of parameters of different types of hooks and the meaning of their return values. There is another problem: All the hooks are stored in a linked list, and the recently added hooks are placed in the head of the linked list. When an event occurs, Windows calls it from the head of the linked list to the end of the linked list. Therefore, your hook function has the responsibility to transmit messages to the hook function in the next chain. Of course, you may not do this, but you 'd better understand the reason for doing so. In most cases, it is best to pass the message event so that other Hooks have the opportunity to process the message. Call the next hook function to call callnexthookex. The function is prototype as follows:

Callnexthookex proto hhook: DWORD, ncode: DWORD, wparam: DWORD, lparam: DWORD

• Hhook is the handle of your own hook function. You can use this handle to traverse the hook chain.
• Ncode, wparam and lparam you only need to pass the passed parameters to callnexthookex.

Note: For remote hooks, hook functions must be placed in the DLL, which will be reflected in other process spaces. When Windows maps DLL to other process spaces, data segments are not mapped. In short, all processes only share the DLL code. As for the data segment, each process will have its own copy. This is a problem that is easily overlooked. You may assume that the values saved in the DLL can be shared among all the processes mapped to the DLL. Generally, because every process mapped to the DLL has its own data segment, your program runs well in most cases. But not the hook function. For hook functions, the DLL data segment must be the same for all processes. In this way, you must set the data segment to shared, which can be achieved by specifying the segment attribute in the Link switch. In MASM, you can do this:

/Section: <section name>, S

The initial segment name is. Data, And the uninitialized segment name is. BSS. 'Add the DLL that you want to write a hook function and share its uninitialized data segments among all processes. You must do this:

Link/section:. BSS, S/dll/subsystem: Windows ..........

S indicates that the segment is a shared segment.

Example:

There are two modules: one is the GUI part and the other is the DLL for installing and uninstalling the hook.

; ------------------------------------------- Source code of the main program --------------------------------------
. 386
. Model flat, stdcall
Option Casemap: None
Include masm32demodewindows. inc
Include masm321_deuser32. inc
Include masm321_dekernel32. inc
Include mousehook. inc
Includelib mousehook. Lib
Includelib masm32libuser32. Lib
Includelib masm32libkernel32. Lib

Wsprintfa proto C: DWORD,: DWORD,: vararg
Wsprintf textequ <wsprintfa>

. Const
Idd_maindlg equ 101
Idc_classname equ 1000
Idc_handle equ 1001
Idc_wndproc equ 1002
Idc_hook equ 1004
Idc_exit equ 1005
Wm_mousehook equ wm_user + 6

Dlgfunc proto: DWORD,: DWORD

. Data
Hookflag dd false
Hooktext DB "& hook", 0
Unhooktext DB "& unhook", 0
Template DB "% lx", 0

. Data?
Hinstance dd?
Hhook dd?
. Code
Start:
Invoke getmodulehandle, null
MoV hinstance, eax
Invoke dialogboxparam, hinstance, idd_maindlg, null, ADDR dlgfunc, null
Invoke exitprocess, null

Dlgfunc proc hdlg: DWORD, umsg: DWORD, wparam: DWORD, lparam: DWORD
Local hlib: DWORD
Local Buffer [128]: byte
Local buffer1 [128]: byte
Local rect: rect
. If umsg = wm_close
. If hookflag = true
Invoke uninstallhook
. Endif
Invoke enddialog, hdlg, null
. Elseif umsg = wm_initdialog
Invoke getwindowrect, hdlg, ADDR rect
Invoke setwindowpos, hdlg, hwnd_topmost, rect. Left, rect. Top, rect. Right, rect. Bottom, swp_showwindow
. Elseif umsg = wm_mousehook
Invoke getdlgitemtext, hdlg, idc_handle, ADDR buffer1, 128
Invoke wsprintf, ADDR buffer, ADDR template, wparam
Invoke lstrcmpi, ADDR buffer, ADDR buffer1
. If eax! = 0
Invoke setdlgitemtext, hdlg, idc_handle, ADDR Buffer
. Endif
Invoke getdlgitemtext, hdlg, idc_classname, ADDR buffer1, 128
Invoke getclassname, wparam, ADDR buffer, 128
Invoke lstrcmpi, ADDR buffer, ADDR buffer1
. If eax! = 0
Invoke setdlgitemtext, hdlg, idc_classname, ADDR Buffer
. Endif
Invoke getdlgitemtext, hdlg, idc_wndproc, ADDR buffer1, 128
Invoke getclasslong, wparam, gcl_wndproc
Invoke wsprintf, ADDR buffer, ADDR template, eax
Invoke lstrcmpi, ADDR buffer, ADDR buffer1
. If eax! = 0
Invoke setdlgitemtext, hdlg, idc_wndproc, ADDR Buffer
. Endif
. Elseif umsg = wm_command
. If lparam! = 0
MoV eax, wparam
MoV edX, eax
SHR edX, 16
. If dx = bn_clicked
. If AX = idc_exit
Invoke sendmessage, hdlg, wm_close, 0, 0
. Else
. If hookflag = false
Invoke installhook, hdlg
. If eax! = NULL
MoV hookflag, true
Invoke setdlgitemtext, hdlg, idc_hook, ADDR unhooktext
. Endif
. Else
Invoke uninstallhook
Invoke setdlgitemtext, hdlg, idc_hook, ADDR hooktext
MoV hookflag, false
Invoke setdlgitemtext, hdlg, idc_classname, null
Invoke setdlgitemtext, hdlg, idc_handle, null
Invoke setdlgitemtext, hdlg, idc_wndproc, null
. Endif
. Endif
. Endif
. Endif
. Else
MoV eax, false
RET
. Endif
MoV eax, true
RET
Dlgfunc endp

End start

; ----------------------------------------------------- DLL source code section --------------------------------------
. 386
. Model flat, stdcall
Option Casemap: None
Include masm32demodewindows. inc
Include masm321_dekernel32. inc
Includelib masm32libkernel32. Lib
Include masm321_deuser32. inc
Includelib masm32libuser32. Lib

. Const
Wm_mousehook equ wm_user + 6

. Data
Hinstance dd 0

. Data?
Hhook dd?
Hwnd dd?

. Code
Dllentry proc hinst: hinstance, reason: DWORD, reserved1: DWORD
. If reason = dll_process_attach
Push hinst
Pop hinstance
. Endif
MoV eax, true
RET
Dllentry endp

Mouseproc proc ncode: DWORD, wparam: DWORD, lparam: DWORD
Invoke callnexthookex, hhook, ncode, wparam, lparam
MoV edX, lparam
Assume edX: PTR mousehookstruct
Invoke windowfrompoint, [edX]. Pt. X, [edX]. Pt. Y
Invoke postmessage, hwnd, wm_mousehook, eax, 0
Assume edX: Nothing
XOR eax, eax
RET
Mouseproc endp

Installhook proc hwnd: DWORD
Push hwnd
Pop hwnd
Invoke setwindowshookex, wh_mouse, ADDR mouseproc, hinstance, null
MoV hhook, eax
RET
Installhook endp

Uninstallhook proc
Invoke unhookwindowshookex, hhook
RET
Uninstallhook endp

End dllentry

; -------------------------------------------- DLL makefile ----------------------------------------------

Name = mousehook
$ (Name). dll: $ (name). OBJ
Link/section:. BSS, S/dll/DEF: $ (name). DEF/subsystem: Windows/libpath: C: masmlib $ (name). OBJ
$ (Name). OBJ: $ (name). ASM
ML/C/COFF/CP $ (name). ASM

Analysis:

The main window of the application contains three editing controls, which display the Class Name of the window where the current mouse cursor is located, the window handle, and the address of the window process respectively. There are also two buttons: "Hook" and "eixt ". When you press the hook, the application hooks the event message entered by the mouse, and the text of this button changes to "unhook ". When you move the mouse over a window, messages about the window are displayed in the main window. When you press "unhook", the application will uninstall the hook. The main window uses a dialog box as its main window. It defines a message wm_mousehook, which is used to transmit messages between the main window and the DLL. When the message is received in the main window, wparam contains the handle of the window where the cursor is located. Of course, this is our arrangement. I did this for convenience. You can use your own methods to communicate between the main application and DLL.

. If hookflag = false
Invoke installhook, hdlg
. If eax! = NULL
MoV hookflag, true
Invoke setdlgitemtext, hdlg, idc_hook, ADDR unhooktext
. Endif

The application has a global variable, hookflag, which is used to monitor the hook status. If the hook is installed, it is true; otherwise, it is false. When you press the hook button, the application checks whether the hook has been installed. If not, it will call the installhook function in the DLL to install it. Note that the handle of the Main Dialog Box is passed to the DLL, so that the hook dll can pass the wm_mousehook message to the correct window. When the application is loaded, the hook DLL is also loaded. Once the main program is loaded into the memory, the DLL is loaded immediately. The DLL entry point function is executed before the first statement of the main program is executed. So when the main program is executed, the DLL has been initialized. Let's load the port point and put the following code:

. If reason = dll_process_attach
Push hinst
Pop hinstance
. Endif

This section of code stores the DLL instance handle in a global variable. Hinstance is always valid because the entry point function is executed before all functions are called. We put the variable in. data so that every process has its own value. This is because when you move the cursor over a window, the hook DLL is mapped to the address space of the process. If other dll has been loaded at the default DLL load address, the hook dll will be mapped to another address. Hinstance is updated to another value. When you press unhook and then the hook, setwindowshookex will be called again. This time, it uses the new address as the instance handle. In the example, this is incorrect, and the DLL loading address has not changed. This hook will become a local one. You can only hook the mouse events that occur in your window, which is hard to satisfy.

Installhook proc hwnd: DWORD
Push hwnd
Pop hwnd
Invoke setwindowshookex, wh_mouse, ADDR mouseproc, hinstance, null
MoV hhook, eax
RET
Installhook endp

The installhook function is very simple. It saves the passed window handle in hwnd for future use. Call the setwindowshookex function to install a mouse hook. The return value of this function is stored in the global variable hhook and will be used in unhookwindowshookex in the future. After setwindowshookex is called, the mouse hook starts to work. No matter when a mouse event occurs, the mouseproc function will be called:

Mouseproc proc ncode: DWORD, wparam: DWORD, lparam: DWORD
Invoke callnexthookex, hhook, ncode, wparam, lparam
MoV edX, lparam
Assume edX: PTR mousehookstruct
Invoke windowfrompoint, [edX]. Pt. X, [edX]. Pt. Y
Invoke postmessage, hwnd, wm_mousehook, eax, 0
Assume edX: Nothing
XOR eax, eax
RET
Mouseproc endp

The hook function first calls the callnexthookex function to allow other hooks to process the mouse event. Then, call the windowfrompoint function to obtain the window handle at the given screen coordinate position. Note: We use the point member variable in the mousehookstruct struct variable pointed to by lparam as the current mouse position. Call the postmessage function to send the wm_mousehook message to the main program. One thing you must remember is: Do not use the sendmessage function in the hook function, which will cause a deadlock. The definition of mousehookstruct is as follows:

Mousehookstruct struct DWORD
PT point <>
Hwnd DWORD?
Whittestcode DWORD?
Dwextrainfo DWORD?
Mousehookstruct ends

• PT is the position of the screen where the current mouse is located.
• Hwnd is the handle of the window that will receive the mouse message. Usually it is the window where the mouse is located, but if the window calls setcapture, the mouse input will be directed to this window. Because we use the windowfrompoint function instead of the member variable.
• Whittestcode specifies the hit-test value, which gives more mouse position values. It specifies the place where the mouse is located in the window. For a complete list of values, see the wm_nchittest message in the Win32 API guide.
• Dwextrainfo this value contains related information. Generally, this value is set by the mouse_event function and can be obtained by calling getmessageextrainfo.

 

When the wm_mousehook message is received in the main window, it uses the window handle in the wparam parameter to query the window message.

. Elseif umsg = wm_mousehook
Invoke getdlgitemtext, hdlg, idc_handle, ADDR buffer1, 128
Invoke wsprintf, ADDR buffer, ADDR template, wparam
Invoke lstrcmpi, ADDR buffer, ADDR buffer1
. If eax! = 0
Invoke setdlgitemtext, hdlg, idc_handle, ADDR Buffer
. Endif
Invoke getdlgitemtext, hdlg, idc_classname, ADDR buffer1, 128
Invoke getclassname, wparam, ADDR buffer, 128
Invoke lstrcmpi, ADDR buffer, ADDR buffer1
. If eax! = 0
Invoke setdlgitemtext, hdlg, idc_classname, ADDR Buffer
. Endif
Invoke getdlgitemtext, hdlg, idc_wndproc, ADDR buffer1, 128
Invoke getclasslong, wparam, gcl_wndproc
Invoke wsprintf, ADDR buffer, ADDR template, eax
Invoke lstrcmpi, ADDR buffer, ADDR buffer1
. If eax! = 0
Invoke setdlgitemtext, hdlg, idc_wndproc, ADDR Buffer
. Endif

To avoid text jitter during text re-painting, we compare the text that has been edited in the space midline with the text we will display. If they are the same, they can be ignored. Get the class name, call getclassname, get the window process, call getclasslong, input the gcl_wndproc flag, format them into text strings, and put them in the relevant editing space.

Invoke uninstallhook
Invoke setdlgitemtext, hdlg, idc_hook, ADDR hooktext
MoV hookflag, false
Invoke setdlgitemtext, hdlg, idc_classname, null
Invoke setdlgitemtext, hdlg, idc_handle, null
Invoke setdlgitemtext, hdlg, idc_wndproc, null

After you press unhook, the main program calls the uninstallhook function in the DLL. This function calls the unhookwindowshookex function. Then, it switches the text of the button back to "Hook", sets the hookflag value to false, and then clears the text in the editing control.
The switch options of the linker are as follows:

Link/section:. BSS, S/dll/DEF: $ (name). DEF/subsystem: Windows

It specifies the. BSS segment as a shared segment so that all processes mapped to the DLL share uninitialized data segments. If you do not need this switch, the hooks in your dll will not work properly.