Top 10 embedded software testing methods

In the embedded software development process, generally, the ratio of time spent in testing and coding is (in fact
Can be more ). This ratio keeps decreasing as your programming and testing level increases, but whatever the case, the software
Testing is important to most people. Many years ago, a developer tried to have a deeper understanding of embedded systems.
Q: How can I know what my system is doing?
?

Oracle was a little surprised at this problem, because no one asked this question at the time, while the contemporary embedded
Most of the questions asked by developers are centered on "How can I make the program run faster" and "What compiler is best"
And other superficial problems. Therefore, Oracle is delighted and earnest in the face of this unusual but almost mature problem.
Reply to him: your question is very in-depth and mature, because it is possible to continue to raise questions only after constant in-depth understanding.
High level. Oracle, in order to encourage this persistent programmer
The secret tells him:

1. Understand how to use tools
2. Detect memory problems as soon as possible
3. in-depth understanding of code optimization
4. Do not make yourself haystack
5. Reproduce and isolate problems
6. Move back
7. Determine test integrity
8. Improving code quality means saving time
9. discover it, analyze it, and solve it.
10. Use the beginner's thinking

These 10 secrets have been widely spread in the industry, benefiting many people. This article focuses on these 10 tips.

1. Understand how to use tools

Generally, embedded systems have high requirements on reliability. Failure of embedded system security may cause disaster recovery
Even if it is a non-secure system, mass production can cause serious economic losses. This is
Strict testing, validation and verification of embedded systems, including embedded software, is required. As more and more
The field uses software and microprocessor to control a variety of embedded devices, and quickly introduces complicated embedded software.
Effective Testing becomes more and more important.
Just like the tools needed for car repair, good programmers should be able to use various software tools skillfully. Different Jobs
With different use scopes and different functions. Using these tools, you can see what your system is doing
What, what resources it occupies, And what external things it deals. Depressing you for several days
It may be easy to do with a tool, but you just don't know. So why are so many people always
How can I use a testing tool after I get stuck? There are many reasons. There are two main reasons. One is fear, the other
Is inert. Fear is that you may need skills to add test tools or test modules to Code while introducing new ones.
Errors, so they always prefer to eliminate bugs by constantly modifying the re-compilation code, but the results are far from satisfactory.
Events. Lazy because they are used to simple testing methods such as printf. The following describes some embedded
Common testing tools.

. Source Code-level debugger [Source-level debugger]
This type of Debugger generally provides single-or multi-step debugging, breakpoint settings, memory detection, variable viewing, and other functions,
Is the most fundamental and effective debugging method for Embedded debugging. For example, GDB provided by VxWorks tornadoii belongs to this
Type.

. The simple and practical printing and display tool [printf] printf or other similar printing and display tools are estimated to be the most
Flexible and simple debugging tools. Printing various variables during code execution allows you to know how code is executed.
Status. However, printf imposes a lot of interference on normal code execution (generally, printf takes a long CPU usage)
, Need to be used with caution, it is best to set the printing switch to control printing.

. Ice or JTAG debugger [in-circuit emulator] Ice is a device used to simulate the CPU core. It can
To detect the internal operation of the CPU in real time without interfering with the normal operation of the timer. Desktop debugging
The functions provided by the software include complex conditional breakpoints, advanced real-time tracking, performance analysis, and port analysis,
It can also provide. Ice generally has a special CPU, called a bond-out CPU. This is
And can access the internal signal of the CPU through special connections.
When the CPU is encapsulated, it cannot be "seen. When used together with powerful debugging software on the workstation
, Ice provides the most comprehensive debugging function you can find. However, ICE also has some disadvantages: expensive; no
Working at full speed. Similarly, not all CPUs can be used as external CPUs. from another perspective
CPU integration is unlikely to be promptly replaced by the new CPU. JTAG (joint test Action Group) Although
It was initially developed to monitor IC and circuit connections, but this serial interface extends its use, including Tuning
Trial Support .. Rom monitor [rom monitor] rom monitor is a small program that resides in the embedded system ROM.
Through serial or network connection and debugging software running on the workstation. This is a cheap method
Of course, it is also the lowest-end technology. In addition to a communication port and a small amount of memory space
It has any dedicated hardware. It also provides the following functions: Download Code, Run Control, breakpoint, step-by-step
And observe and modify registers and memory. Because Rom monitor is part of the operating software, only when your application
It only works when the program runs. If you want to check the CPU and application status, you must stop the application.
Program to go To the ROM monitor again.

. Data Monitor [Data Monitor] This kind of monitor not only can be displayed without stopping CPU operation
Specify the variable content. You can also collect and graphically display the change process of each variable.

. OS monitor [operating system monitor] operating system monitor can display tasks such as Task Switching
, Semaphore sending and receiving, interruption, and other events. On the one hand, these monitors can present the relationship between events and
On the other hand, it can also diagnose problems such as priority inversion of semaphores, deadlocks, and interruption latency.
.

The performance analysis tool [profiler] can be used to test the CPU consumption. The profiler tool allows
You know where the system bottleneck is, CPU usage, and where optimization is needed.

The memory test tool [Memory teseter] can locate memory usage problems, such as memory leakage.
Memory fragments, memory crashes, and other issues. If the system encounters unpredictable or intermittent problems,
You should use the memory testing tool for testing.

. Run the tracker [execution tracer] to display the functions, calls, and parameters executed by the CPU.
And when to call. This tool is mainly used to test the code logic and can be released in a large number of events.
Abnormal.

The overwrite tool [Coverage tester] mainly displays the code executed by the CPU and lets you know
Some code branches are not executed. This helps improve code quality and eliminate useless code.

. Gui testing tool [GUI tester] Many embedded applications carry some form of graphical user interface for Interaction
Mutual, some system performance tests are performed based on the user input response time. Gui testing tools can be used as scripts
The tool runs test cases in the development environment. Its functions include recording and playback of operations and capturing screen display.
Later analysis and comparison, setup and Management of the test process (robot of rational and LoadRunner of mercury)
Tool is an outstanding representative ). Many embedded devices do not have a GUI, but they can often be inserted into the embedded devices.
Run the GUI test script. Although this method may require changes to the tested code, it saves the function test cost.
Time of the test and regression test.

. Self-made tool [home-made tester] in embedded applications, sometimes for a specific purpose, you need
Compile some tools to achieve some testing purpose. I have compiled a video stream recording and display tool to test the video.
Meeting data flow and changes helped the company find several hidden bugs.

2. Early Detection of memory problems is very harmful and difficult to troubleshoot. There are three main types: Memory leakage
Exposure, memory fragmentation, and memory crash. The attitude towards memory problems must be clear, that is, early detection and early treatment"
. In the software design, memory leakage is the most famous, mainly because the constantly allocated memory cannot be timely
Release, over time, the system's memory is exhausted. Even careful Programmers sometimes encounter memory leaks
Question. A friend who has tested the memory leak is expected to have a profound experience, that is, the memory leak problem is generally hidden.
Deep, it is difficult to find through code reading. Some memory leaks may even occur in the database. It is possible that
It is a bug in the library, or it may be caused by errors caused by the programmer's failure to correctly understand their interface instructions.

In many cases, most Memory leakage problems cannot be detected, but may result in random faults. Program
Employees often blame this phenomenon for hardware problems. If the system stability is not high, restart
System problems are not big either. However, if the user is very stable to the system, such faults may cause the user
Product loss of confidence also means that your project is a failed project. Memory leakage causes great harm.
There are already many tools to solve this problem. These tools are used to find unreferenced or reused code blocks,
Spam memory collection, database tracking, and other technologies to detect memory leaks. Each tool has advantages and disadvantages.
It is better to use it than to use it. In short, the responsible developers should test the memory leakage to prevent
Before they happen.
Memory fragments are more hidden than memory leaks. As the memory is constantly allocated and released, the memory size is continuously divided.
It is resolved to a small block of memory to form fragments. Over time, when you need to apply for a large block of memory, it may fail.
. If the system memory is large enough, it will take a long time to stick to it, but it will not escape the bad luck of allocation failure.
. In systems with dynamic allocation, memory fragments often occur. Currently, the most effective solution to this problem is
Is to use a tool to display the memory usage in the system to find out who is the culprit of memory fragmentation, however
And then improve the corresponding part.

Due to various problems in dynamic memory management, many companies simply disable malloc/
Free.
Memory crash is the most serious result of memory usage, mainly because the array access is out of bounds and the write has been released
Memory, pointer calculation error, and stack access address out of bounds. The system failure caused by this memory crash is random,
And it is difficult to find out. Currently, few tools are provided for troubleshooting.
In short, if you want to use memory management units, you must be careful and strictly abide by their usage rules.
For example, who is assigned and who is released.
3. in-depth understanding of code optimization
When talking about system stability, people will think of real-time and speed more, because the code efficiency
It is too important. Knowing how to optimize code is a skill that every embedded software developer must possess. Just like
A girl needs to lose weight. At least she knows where she needs to lose weight before she can buy diet pills or equipment to lose it.
. It can be seen that the premise of code optimization is to find the place that really needs optimization, and then take the right medicine to optimize the corresponding part.
. The profile mentioned above (performance analysis tools, some fully functional ides provide such built-in work
Can record various situations, such as the CPU usage of each task, whether the priority of each task is properly assigned,
How many times a data has been copied, how many times the disk has been accessed, whether the network receiving and receiving program has been called, and the test code
Whether it is disabled or not.
However, the profile tool is insufficient to analyze the real-time system performance. On the one hand, people
The profile tool is often used after a system problem occurs, that is, the CPU is exhausted, and the profile tool occupies the CPU
The profile may not work in this situation. Any test based on the Heisenberg effect
Means will change the system operation more or less, which is also applicable to profiler! In short, improve operation efficiency
The premise is that you must know what the CPU has done.

4. Do not make yourself haystack
Haystack is just a vivid metaphor for debugging. I often hear that someone in the group is debugging the code.
Say shit! It can be understood that because the code is not written by him, he has enough reason to go to the shit bug code,
As long as he does not write such code, otherwise, other people in the same group may also shit the Code he wrote one day.
. Why is there a needle in a haystack? Someone must have dropped the needle into the sea. Why did the needle fall into the sea? Yes
Someone is not careful or hasty. So when you complain about the difficulty of finding a needle, have you ever thought about your own grass?
Loss Rate. Similarly, when debugging is half-dead, have you ever wondered whether you want to reflect on
Seeking shortcuts may not strictly abide by good coding design specifications, nor detect positive conditions or Algorithms
Are you sure you want to mark some codes that may have problems? For more information about how to write high-quality data, see Lin Rui's
High quality c ++/C programming guide or 0x8 book on C (http://blog.sina.com.cn/u

/4a317b79010004mc ).

If you have dropped the needle in the ocean, you must do one to prevent yourself from being stabbed before finding it.
Some preventive work, such as wearing safety gloves. Similarly, to fully expose and capture the root cause of the problem, we can
Design a comprehensive error tracking code. How can this problem be solved? Handle every function call failure as much as possible,
Try to check the validity of input and output of each parameter, including pointer and check whether a certain parameter is called too many or too few times.
Process. The Error Tracking will let you know where you probably dropped the needle.

5. Reproduce and isolate problems
If you don't drop the needle in the sea, but in the straw, you can do it better. Because at least we can
Divide the straw heap into multiple blocks and find them one by one. For large-scale projects with independent modules, the isolation method is usually
The last method to deal with hidden bugs. If the problem occurs intermittently, we need to try to remove it.
Now it records the entire process of making it reproduce in order to reproduce the problem with these conditions in the next time. If you
We are sure we can use the recorded conditions to reproduce the problem, so we can proceed to isolate the problem. How to isolate
? We can use # ifdef to close some code that may have nothing to do with the problem, so as to minimize the number of systems that can still
Reproduce the problem. If the problem persists, open the toolbox. Yes
Try to use ice or data monitor to view changes to a suspicious variable. You can use the tracking tool to obtain function calls.
Usage includes passing parameters, checking whether memory crashes, and stack overflow.

6. Move back
In order not to lose himself in the forest, the hunter often shed some marks on the trees for himself
You can find the way out based on these tags one day. Track past code changes and record future changes
Debugging after a problem occurs is very helpful. Assume that one day, after the last modified program ran for a long time
Then, what is the first reflection of your change, because before the last modification
Okay. So how can we check the last modification? Yes, the code control system SCS or version control.
System VCs (concurrent version control, CVS is the evolution version of VCs ). Check the previous version
After the in is down, it is compared with the current test version. The comparison tool can be the built-in diff tool of SCS, VCs, and CVS or its
It is a more powerful comparison tool, such as beyondcompare and examdiff. Record all changes through comparison
To analyze all the suspicious code that may cause problems.

7. Determine test integrity
How do you know how comprehensive your test is? Coverage testing can answer this question.
Question. The overwrite test tool can tell you which code is executed by the CPU. A good coverage tool can usually tell
You have no problem with code between 20% and 40%, and other bugs may exist. The coverage tool has different test levels
You can select a level based on your needs. Even if you are confident that your unit test is complete and
If there is no dead code, the overwriting tool can still point out some potential problems for you. See the following code: If (
I> = 0 & (almostalwayszero = 0 | (last = I) If almostalwayszero is not 0
, The Last = I value assignment statement is skipped, which may not be what you expected. This problem is solved by overwriting the tool's
The conditional test function can be easily detected. In short, coverage testing is very helpful for improving the code quality.

8. Improving code quality means saving time
Some research shows that more than 80% of software development time is used in the following aspects:. debug your own code (Unit
Testing). debug yourself and other related code (Inter-Module Testing). debug the entire system (System Testing) worse
It may take 10-times for you to find a bug, which may be easy at the beginning.
. A small bug may cost you a lot, even if the bug does not have performance for the entire system
It has a big impact, but it is likely to affect what you can see. Therefore, we must develop a good
Coding and testing methods for higher code quality, so as to shorten the debugging code.

9. discover it, analyze it, and solve it.
There is no panacea in this world. The memory monitor is good when the profile is bigger and bigger.
When the coverage tool is easy to use or cannot be covered. Some Hidden questions
Even if you use all the tools, you may not be able to find the root cause of the problem.
External phenomena or data output to discover the rules or exceptions. Once any exceptions are found,
Be sure to thoroughly understand and trace the root cause until it is solved.

10. Use the beginner's thinking
Someone said: "Some things may have various situations in the minds of beginners.
May be very single in mind ". Sometimes, some simple problems may be complicated, and some simple systems
The design is complicated because of your "expert thinking ". When you are in trouble, turn off your computer and go out.
Let's take a walk and talk about your problem with your friends and even your puppies. Maybe they can give you an unexpected start.
.

Conclusion: Embedded debugging is also an art. Just like other arts, if you want to succeed, you
Knowledge, experience, and knowledge of tools are essential. As long as we can understand the ten secrets of Oracle,
I believe that we can succeed in embedded testing.