Debugging in Windows

Debugging in Windows

1. There are two basic objectives for future debugging:
(1) discover where the program crashed
(2) identify the cause of program crash
2. debugging technology arranged in the order of priority:
(1) Use the debug version for local debugging
(2) Use a release version with debugging symbols for local debugging
(3) Use the debug version for remote debugging
(4) use a released version with debugging symbols for remote debugging
(5) use the dr. Watson log file for post-event debugging
(6) use the information in the crash dialog box for post-event debugging
3. the return value of a Windows API function of the bool type is not necessarily 0 or 1. Therefore, when compiling a C ++ program with the debugging function, comparing the returned value with true is a very risky task. avoid doing so. If an error occurs in a Windows API function whose return type is handle, an empty handle or invalid_handle_value (value:-1) is usually returned ). For Windows API functions whose return type is long or DWORD, 0 or-1 is usually returned. If an API function cannot have an error, it returns void. You can get the corresponding error code through getlasterror.
4. in Visual C ++, you can enter "@ err, HR" in the observation window to monitor the returned values of getlasterror.
5. If you want to display the error code in the error message, you can use the formatmessage API function to convert the error code to the text format.
6. Never relocate third-party DLL used by other programs
7. The optimized version usually does not use EBP as the stack base address pointer. This optimization type is called frame pointer omitting (FPO)
8. Because the optimized version may not use the stack base address pointer, if the function prototype declaration is inconsistent, it may cause a crash when the function returns.
9. Use the debug Disassembly window to view how your source code is converted into assembly code
10. create the most useful ing files, usually using the/MapInfo: lines and/MapInfo: exports project options. Archive The ing files of all modules of your released program.
11. Use the Visual C ++ name resolution tool (undname) to convert the mixed name to the original name.
Undname? Randomexception @ ygxhhhh @ Z

Output :? Randomexception @ ygxhhhh @ z = randomexception

-F option displays the entire function prototype
Undname-F? Randomexception @ ygxhhhh @ Z

Output :? Randomexception @ ygxhhhh @ z = void _ stdcall randomexception
11. the runtime function library of the debug version tracks the memory allocation and allows users to check for memory leaks. Write the 0xcd byte mode in the allocated memory, this helps to identify errors in using uninitialized data; write the 0xcd byte mode in the released memory, which helps to detect the use of released memory; four bytes of protected data are allocated on both sides of the buffer and initialized in the byte mode of 0xfd to check the top overflow and bottom overflow of the write memory; the source code file name and line number are recorded in each memory allocation, which helps you locate the memory allocation in the source code. Therefore, multiple memory errors can be found in the debug version.
12. Frame pointer omitting (FPO) hides the function prototype Mismatch Error, which only causes the function to crash when the function is returned in the debug version.
13. All variables in the debug version are volatile. In a released version, if a variable is not set to volatile, there will be an optimization-related error. If multiple threads are used, there is a high possibility of program problems.
15. Variable Optimization issues in the released version:
Void stackattack (){
Int optimizedout1, optimizedout2;
Tchar bustext [16], * bugs = _ T ("this function has bugs! ");
_ Tcscpy (bustext, bugs );
}
In this function, the length of the bustext buffer cannot receive the bugs string. Unnecessary variables optimizedout1 and optimizedout2 will protect the stack content from corruption in the debug version, but these variables will be removed in the released version. The result is that the buffer overflow destroys the return address of the function returned by the stack, and the released program crashes, but not in the debug version. Generally, the optimized variables are not so obvious.
12. It is best to create debugging symbols for your executable program and archive the obtained PDB documents, even if the program belongs to the released version.
13. Create a debugging symbol for a specific version of the program. Set the Visual C ++ project corresponding to the program as follows:
(1) Open the Project Settings dialog box, and select the desired version (for example, Win32 release) in the setting... dialog box ).
(2) In the project control tree, click the root node to select the entire project.
(3) Select the general class in the C/C ++ label. In the debugging information, if it is a release version, select program database. If it is a debug version, select program database for edit and continue (note: the edit continue option is incompatible with the optimized connection, it also increases the length of the executable file, which is not suitable for the release version ).
(4) Select the debug class in the Link label. Select debug info and Microsoft format. Remember not to select the separate types option so that all debugging information will be merged into a separate PDB file. In addition, if you need to debug the ing file after you do something, remember to select the generate mapfile option.
(5) for the release version, select the link tag and add "/OPT: ref" at the end of the Project Options dialog box ". This option prevents reference functions and data from appearing in the executable file, thus avoiding the file unafraid increase. Do not use this option for debugging versions because it will disable incremential linking)
(6) use the rebuild all command to re-compile the entire project.
Note: If you find that the executable files with debugging symbols are much larger than those without debugging characters, you may have forgotten to add the/OPT: ref link option.
14. To further control debugging, you need to use the disassembly code window for code debugging.
15. It is best not to use the separate types option unless you want to compile a very large project on a very slow computer.
16. The Tib (thread information block) structure can be displayed in the observation window. Use @ TiB and add the following code to the program:
# Ifdef _ debug
# Include "Tib. H"
Ptib
# Endif
In the observation window, use ptib = @ Tib to view Tib content.
17. Use autoexp. dat
18. Set the system call breakpoint in Windows2000:
(1) determine the module that contains the API function.
Findstr MessageBox win32api.csv
(2) determine that the debugging symbol corresponding to the module has been loaded
(3) determine the real function name
Dumpbin-Symbols user32.dbg | findstr MessageBox
Return "_ messageboxa @ 16 ". If the debugging symbol is not loaded, run the following command:
Dumpbin-Exports user32.dll | findstr MessageBox
Return "messageboxa". Note that "MessageBox" will only be seen by the Preprocessor. It will convert the name to "messageboxa" or "messageboxw". "A" indicates ANSI, "W" represents a wide character or Unicode.
(4) set a breakpoint in the breakpoint dialog box. If the debugging symbol is loaded, enter
{, User32.dll} _ messageboxa @ 16
If the debugging symbol is not loaded, enter
{, User32.dll} messageboxa
If the debugging symbols are not loaded, you need to set the load coff & export option in the debug label of the Options dialog box. This option allows you to set breakpoints on the output function without debugging symbols.
Note: If the findstr.exe tool is not available, you can use the find in files command of Visual C ++.
19. if the returned value is no more than 32 bits, enter "@ eax" in the observation window. If the returned value is 64 bits, the 32-bit low value is placed in eax, and the 32-bit high value is placed in EDX. If the returned value is greater than 64 bits, a pointer pointing to the returned value will be placed in eax. type conversion can be performed in the observation window. For example, if a crect is returned, you can type "(crect *) @ eax "displays the result, or directly type eax in the address column of the Memory window to view the return value.
20. Use the API function getasynckeystate to debug the wm_mousemove message.
21. Use spy ++ to debug message-related issues.
22. Use callback to help you debug windows code: the callback allows you to access windows to see what it is doing.
23. Difference Between postmessage and sendmessage: postmessage is only responsible for placing messages in the message queue. It is not determined when and whether sendmessage is processed until the return code (DWORD type) that is processed by the message is sent.
24. You can use the memory alignment rules of X86 to determine whether a pointer is valid. The stack and heap pointers are both double-character alignment. Therefore, their last position should be hexadecimal 0, 4, 8, and C. When the x86 command can be of any size, the last digit of the command pointer can be any number. The function is 16-byte alignment, so the last bit of the function pointer should always be zero.
25. To find access to the released memory, you must leave the released pointer empty.