Compilation Tutorial: Win32 Debugging API (3)

In this chapter, we will continue to explore the Win32 debugging API. In particular, we will learn how to track the debugged program.

Theory:

If you've used a debugger before, you should be familiar with tracking. When tracking a program, the program stops after each instruction executes, which gives you the opportunity to check Register/memory values. This one-step operation is officially defined as tracking (tracing).

The characteristics of a single-step operation are provided by the CPU itself. The 8th bit of the flag register is called the TRAP flag trap flag. If the bit is set, the CPU runs in Single-step mode. The CPU will generate a debug exception after each instruction. When debug exception is generated, the trap flag is automatically cleared. With the Win32 debugging API, we can also step into the debugged program. The method is as follows:

Call GetThreadContext, specify Contextflags as Context_control to obtain the value of the flag register

Set the trap flag bit in the context structure member flag Register Regflag

Call SetThreadContext

Waits for the modal event. The debugger will execute in single step mode, and after each instruction, we will get the debug event, the U.exception.pexceptionrecord.exceptioncode value is Exception_single_step

If you want to track the next instruction, you need to set the trap flag bit again.

Cases:

.386
. Model Flat,stdcall
Option Casemap:none
Include \masm32\include\windows.inc
Include \masm32\include\kernel32.inc
Include \masm32\include\comdlg32.inc
Include \masm32\include\user32.inc
Includelib \masm32\lib\kernel32.lib
Includelib \masm32\lib\comdlg32.lib
Includelib \masm32\lib\user32.lib
. Data
AppName db "Win32 Debug Example No.4", 0
Ofn OpenFileName <>
Filterstring db "Executable Files", 0, "*.exe", 0
DB "All Files", 0, "*.*", 0,0
Exitproc db "The debuggee exits", 0dh,0ah
DB "Total instructions executed:%lu", 0
Totalinstruction DD 0
. Data?
Buffer db DUP (?)
StartInfo Startupinfo <>
Pi Process_information <>
Dbevent debug_event <>
Context <>
. Code
Start
MOV ofn.lstructsize,sizeof ofn
mov ofn.lpstrfilter, OFFSET filterstring
mov ofn.lpstrfile, OFFSET buffer
MOV ofn.nmaxfile,512
mov ofn. Flags, ofn_filemustexist or ofn_pathmustexist or ofn_longnames or Ofn_explorer or ofn_hidereadonly
Invoke GetOpenFileName, ADDR ofn
. If Eax==true
Invoke Getstartupinfo,addr StartInfo
Invoke CreateProcess, addr buffer, NULL, NULL, NULL, FALSE, debug_process+ debug_only_this_process, NULL, NULL, addr start info, addr Pi
. While TRUE
Invoke Waitfordebugevent, addr dbevent, INFINITE
. If dbevent.dwdebugeventcode==exit_process_debug_event
Invoke wsprintf, addr buffer, addr Exitproc, totalinstruction
Invoke MessageBox, 0, addr buffer, addr AppName, mb_ok+mb_iconinformation
. break
. ElseIf dbevent.dwdebugeventcode==exception_debug_event. If dbevent.u.exception.pexceptionrecord.exceptioncode== Exception_breakpoint
MOV context. Contextflags, Context_control
Invoke GetThreadContext, Pi.hthread, addr context
or context.regflag,100h
Invoke Setthreadcontext,pi.hthread, addr context
Invoke Continuedebugevent, Dbevent.dwprocessid, Dbevent.dwthreadid, dbg_continue
. continue
. ElseIf Dbevent.u.exception.pexceptionrecord.exceptioncode==exception_single_step
Inc totalinstruction
Invoke GETTHREADCONTEXT,PI.HTHREAD,ADDR context or context.regflag,100h
Invoke Setthreadcontext,pi.hthread, addr context
Invoke Continuedebugevent, Dbevent.dwprocessid, dbevent.dwthreadid,dbg_continue
. continue
. endif
. endif
Invoke Continuedebugevent, Dbevent.dwprocessid, Dbevent.dwthreadid, dbg_exception_not_handled
. ENDW
. endif
Invoke Closehandle,pi.hprocess
Invoke Closehandle,pi.hthread
Invoke ExitProcess, 0
End Start
Analysis:

The program first displays an open file dialog box, and when the user selects an executable file, it steps through the program and records the number of instructions executed until the debugger exits.

. ElseIf dbevent.dwdebugeventcode==exception_debug_event. If dbevent.u.exception.pexceptionrecord.exceptioncode== Exception_breakpoint

Use this opportunity to set the debugger as Single-step mode. Remember that Windows sends a EXCEPTION_BREAKPOINT message before executing the first instruction in the debugger.

MOV context. Contextflags, Context_control
Invoke GetThreadContext, Pi.hthread, addr context

Call GetThreadContext to populate the context structure with the current register contents of the debugger in particular, we need the current value of the flag register.

or context.regflag,100h

Set the trap bit of the Flag Register image (8th bit)

Invoke Setthreadcontext,pi.hthread, addr context
Invoke Continuedebugevent, Dbevent.dwprocessid, Dbevent.dwthreadid, dbg_continue
. continue

Then call SetThreadContext to overwrite the value of the context. The continuedebugevent is then invoked by Dbg_continue to restore the running of the debugged program.

. ElseIf Dbevent.u.exception.pexceptionrecord.exceptioncode==exception_single_step
Inc totalinstruction

When an instruction in the debugger executes, we receive the Exception_debug_event debug event and must check U. The value of the Exception.pExceptionRecord.ExceptionCode. If the value is Exception_single_step, the debug event is caused by a single-step mode. In this case, Totalinstruction plus one, because we know exactly when the debugger is executing an instruction.

　　
Invoke GETTHREADCONTEXT,PI.HTHREAD,ADDR context or context.regflag,100h
Invoke Setthreadcontext,pi.hthread, addr context
Invoke Continuedebugevent, Dbevent.dwprocessid, dbevent.dwthreadid,dbg_continue
. continue

Because the trap flag is automatically cleared after a debug exception, you must set the trap flag bit if we need to continue with the Single-step mode.

Warning: Do not use this example in this tutorial to debug large programs: Tracking is slow. You may need to wait more than 10 minutes to shut down the debugger.