Notes for C ++ multi-thread debugging and Testing

In the last post, I talked about the cross-platform problem of C ++ multithreading. Later, I felt like I was not doing anything. Today, some precautions for debugging and testing when developing C ++ multi-threaded applications. The following precautions are mainly for C ++, but some are also applicable to other languages.

★About breakpoint setting and one-step execution
Many people rely heavily on the breakpoint and single-step functions of the debugger. This is fine in the case of a single thread (however, some single-threaded but Gui-related programs may also be a little troublesome ). As for multi-threaded program debugging, these two methods are the beginning of a nightmare. Most of the main problems caused by multithreading are related to race condition. For more information, see "here. However, when you set a breakpoint or a single-step trailSevere interferenceCompetition among multiple threads. What you see is an illusion. For example, two threads run concurrently and some discord bugs exist (caused by competition ). Once you set a breakpoint in a thread, the thread stops at the breakpoint, and only the other thread is running. At this time, the concurrency scenario has been completely destroyed, as you can see through the debuggerPossibleIs a harmonious scenario.
Run the question a little. This is similar to the "Uncertainty Principle" of Quantum Mechanics. The observer's observation behavior interferes with the object to be measured, causing the observer to see an interference phenomenon.

★About log output
Since breakpoint and single step are not easy to use. What should I do? An alternative solution is to output log. It can effectively reduce the side effects caused by breakpoints and individual steps (for competitive conditions.
◇ Traditional log mechanism problems
The traditional log output is mainly printed to the screen or to the file. For C ++, the built-in classes and functions (such as cout, printf, and fputs) in the standard library may have thread security issues (related to the specific implementation of the compiler ). Especially for the eight global objects of the standard stream Library (iostream), you must be careful when using them. If it is light, the output log text is mixed. If it is heavy, the program crashes.
For the above reasons, we should try to use the log mechanism built in the third-party thread library to handle the log output function. For example, Ace's built-in ace_log_msg.
◇ Log functions must be short and concise
In many cases, we wrap a common function to implement the log output function. Then, the function calls the log class/function of the thread library. In order not to affect the Race Condition of the thread, this log function should be as simple as possible: do not involve too many trivial matters, do not perform time-consuming operations, and try not to operate some global variables.
◇ Log side effects
However, even if the log function is short and concise, it may affect the race condition (after all, log also has overhead and consumes CPU time ).
In case the race condition is affected by the log, it will be tricky. I have encountered this kind of situation before: Adding log, the program is normal; removing log, the program crashes randomly. In this case, there are two possible reasons: either the log function itself is faulty, or the race condition of the program is very sensitive (even the log overhead will be affected ).
At this time, you can only rely on the naked eye and the human brain. First, let's take a closer look at the relevant code and documentation several times (we 'd better find another experienced person to review the Code together), and then let's start thinking about it.

★About the debug and release versions
C ++ programs often have different versions of debug and release. In some cases, this may also lead to multithreading issues.
The debug version contains some debugging information and enables some debugging mechanisms (such as assert macros ). SoPossibleThis affects the competition of multiple threads. In bad luck, the debug version works normally, and the release version program crashes randomly. To avoid this situation, consider the following two methods:
◇ Discard the debug version
You can simply discard the debug version. In this case, you need to consider replacing debugging-related macros such as assert with your own set of macros so that they can take effect in non-DEBUG Versions.
◇ Two versions of synchronization test
With this method, programmers can use the debug version for self-testing, but testers must use the release version for routine testing. The specific operation steps can be assisted by daily build (for details about daily build, refer to "Here "). Be sure to avoid: In normal times, only debug version testing will be performed, and the release version will be created on the eve of the release. This is very dangerous!

★About the tested machine (hardware)
It's a personal experience and impressive thing.
When Ace was used to develop cross-platform programs, the company's development and testing environments were single-CPU machines. At that time, the multi-core machine was not available, and the multi-CPU machine was expensive, and the company was not willing to spend money on configuration.
After the software is developed, the tester has gone through several rounds of regression tests and has not found any major problems. However, running in the customer's environment often causes random crashes. Because debugging cannot be performed in the customer environment, and the Environment is okay, several developers in the development team have to make full use of the functions of the naked eye and the human brain (staring at the code and design documentation ). After N for a long time, I almost broke my head and realized that the customer's machine was multi-CPU. Then I quickly borrowed a multi-CPU machine from other departments, installed the software for debugging, and finally found out there was a problem with a third-party library. After that, I immediately came up with various methods to apply for multiple CPU machines for testing.
Due to the lessons learned above, I strongly recommend that youEveryBoth programmers and testers configure multi-core/multi-CPU machines. After all, now multi-core machines are very popular, even if multiple CPU machines, the price is still reasonable. There is really no need to introduce development risks to save that little money (not only will the development/testing staff waste time, but may also increase the implementation and maintenance costs ).
In addition, some may ask "How to compress hyper-threading machines ?" If you are interested in the differences among multiple CPUs, multiple cores, and hyper-threading, you can refer to "here ". I personally feel that hyper-threading is not as good as multi-core and multi-CPU.

The reprinted statement must contain this statement to ensure the completeness of this article. The author's programming preferences and original addresses are shown in hyperlink form:
Http://program-think.blogspot.com/2009/04/debug-test-multithreaded-applications.html