The method of generating code coverage for C + + project under Linux

Recently done a series of unit testing related work, in addition to various specifications and testing framework, the discussion is more about the production of code coverage, C + + with some other high-level language or scripting language, such as Java,. NET and Php/python/perl/shell, because there is no reflection of these high-level languages and scripting languages, the process of code coverage is slightly more complicated. Found that many students on C + + coverage of how the production is not very clear, here to do a simple introduction.

I. Basic methods of Use

C/D + + development on Linux generally uses gcc/g++ as the main compiler, if need to generate coverage data need to be in the makefile or scons file to do the following compile link settings,

• When compiling, add-fprofile-arcs-ftest-coverage or –coverage;
• When linking, add-fprofile-arcs or –lgcov;
• Open the –G3 option to remove the code optimization options above the-O2 level, or the compiler will do some optimizations to the code, such as row merging, which affects the row coverage results;

The basic requirements on the above three points, but there is a suggestion, in order to use the above several compilation options do not affect the normal compilation process (otherwise it will greatly affect the efficiency of the program). By using parameter passing in makefile to support coverage generation, you can use the following method in Makefile

Ifeq ($ (coverage), yes)

Cxxflags + =-fprofile-arcs-ftest-coverage

Linkercxx + =-fprofile-arcs-ftest-coverage

Opt_flags =-g3

endif

In this way, you can use make Coverage=yes to introduce these compilation options without affecting the normal compilation (scons).

Second, a simple example

Here is a simple program to do the test, consisting mainly of three files: Rectangle.cpp, RectangleTest.cpp, Makefile.

1) Rectangle.cpp is the code being measured, which defines a simple class Rectangle (Rectangle) with three methods:

• Set_values (), sets the length and width of the rectangle object;
• Area (), to find the size of the rectangle;
• Lenth (), to find the circumference of the long-form;

2) RectangleTest.cpp is a simple test program, in order to use the demo, and do not use cppunit/gtest such a unit test framework, directly using the main () function to invoke rectangle inside the method;

The code for Rectangle.cpp and RectangleTest.cpp, for example,

3) Makefile is relatively simple, mainly supports the parameter support in Coverage=yes. You can use the-fprofile-arcs-ftest-coverage option here in order to simplify the use of –coverage.

The process of coverage generation is shown in the following four steps, with steps 3 and 4, which can be used as needed.

1. Compile the source code with the coverage parameters;

2. Run the test procedure;

3. Use Gcov to obtain coverage data in the form of text;

4. Use Lcov to obtain the coverage data in HTML form;

The following is a step-through demonstration of this process for this example.

1. Compile the source code with the coverage parameters;

Since the Coverage=yes option is already supported in Makeifle, run "make Coverage=yes" directly, this time will produce the test program, and also generate Gcno file (detailed explanation of Gcno file, see the principle behind the third part), for example,

2. Run the test procedure;

Run the./rectangletest test program, after the run, the corresponding *.GCDA file will be generated for all CPP source code files (for a detailed explanation of the gcda file, see the principle behind the third section), as

3. Use Gcov to obtain coverage data in the form of text;

It is important to note that this step is not required, but if you need the coverage result of the text Format (*.gcov), take this step. If you want to see the results in HTML format, skip this step directly. The Gcov is a result generation tool with GCC's own coverage, eliminating the need for separate installation.

For a source code file, such as Rectangle.cpp, executing "gcov Rectangle.cpp" produces Rectangle.cpp.gcov files.

This is a saved text file that can be opened via vim and see detailed line coverage data, as follows

4. Use Lcov to obtain the coverage data in HTML form;

Sometimes it's necessary to use the data presentation of the HTML results to make it look more intuitive. IBM Open Source Lcov This tool, more see http://ltp.sourceforge.net/coverage/lcov.php

Tools to use, such as,

Manually copy the Cc_result directory to the Htdocs directory of servers such as Http/apache, which can be viewed through the browser to see the coverage results, as below,

The entire coverage generation process can be done in four steps above. The following section makes a brief description of its principle.

Third, the basic principle

1. Terminology interpretation

Before you understand the rationale behind this, you need to have a simple understanding of some of the concepts of coverage technology. The main blocks are basic block, basic block graph, line coverage (lines coverage), branch coverage (branch coverage), etc.

• Basic block, "a basic block is a sequence of instructions with only entry and only one exit.  If any one of the instructions is executed, they'll all is executed, and in sequence from first to last. " Here you can think of the basic block as a whole line of code, the code in the basic block is linear, or all run, or not run;
• Basic block graph, the last statement of the basic block is generally going to jump, otherwise a statement will be counted as part of the base block. If a jump statement is conditional, a branch (ARC) is created, and the base block has two base blocks as the destination. If each base block is treated as a node, then all the basic blocks in a function form a forward graph called the basic block graph. And as long as you know the number of BB or arc of the execution times can deduce all the BB and all the arc execution times;
• Piling, which means adding a counter between valid base blocks, calculates the number of times the base block is run, and the position of the piling is on the effective side of the basic block diagram;
• Line coverage, the ratio of the number of active lines of source code to the line of code being executed;
• Branch coverage (branch coverage), where there is a decision statement, there will be 2 branches, the entire program through the branch and all branches of the ratio is the branch coverage. Note that there is a slight difference between the conditional coverage (condition coverage), and the conditional coverage is more finely divided on the combination of the decision statements.

2. gcc/g++ compilation options

GCC needs to statically inject the target program compilation options, add 2 options (-FTEST-COVERAGE-FPROFILE-ARCS) When compiling the link, generate the *.gcno file after the compilation, and after the static injection of the target program at "Normal End", The *.GCDA data file is generated in the run directory, and the coverage data results are generated through the Gcov tool.

-ftest-coverage

produce a Notes file that the Gcov code-coverage utility (see GCOV-A Test coverage Program) can use to show program Coverage. Each of the source file ' s note file is called Auxname.gcno. Refer to The-fprofile-arcs option above for a description of auxname and instructions on how to generate test coverage da Ta. Coverage data matches the source files more closely if you does not optimize. Let the compiler generate a. gcno file with the same name as the source code (note file) that contains a rebuild base block dependency graph and The necessary information to associate the source code with the base block;

-fprofile-arcs

ADD code So, program flow arcs is instrumented. During execution The program records what many times each branch and call are executed and how many times it's taken or RET Urns. When the compiled program exits it saves this data to a file called AUXNAME.GCDA for each source file. The data is used for profile-directed optimizations (-fbranch-probabilities), or for test coverage analysis (-ftest-co Verage). Each object file's Auxname is generated from the name of the output file, if explicitly specified and it's not the final Executable, otherwise it is the basename of the source file. In both cases any suffix are removed (e.g. FOO.GCDA for input file dir/foo.c, ORDIR/FOO.GCDA for output file specified as- o dir/foo.o). See cross-profiling.

Let the compiler statically inject the code that operates on the counters associated with each source code line and link in the link stage to the LIBGCOV.A, which contains the logic to generate the *.GCDA file at the end of the program;

The following through the source code analysis to explain exactly what these 2 options do. With the g++-S option, the assembly language Rectangle.s and Rectangle_cc.s (add –coverage option) are generated, the command is as follows,

g++-c-o rectangle.s rectangle.cpp-g-wall-s

g++-c-o Rectangle_cc.s rectangle.cpp-g-wall–coverage-s

Vimdiff RECTANGLE.S and Rectangle_cc.s, such as

Through this assembly language comparison, we can see that gcc through the 2 parameters, the piling process completed.

More in-depth content, for example, if you want to know the format of the GCNO/GCDA file, you can refer to an article in @livelylittlefish, GCC coverage Code Analysis-.GCDA/.GCNO file and its format analysis (http:// blog.csdn.net/livelylittlefish/article/details/6448885).

Iv. expansion of the topic through the above three parts of the introduction, I believe that the vast majority of coverage problems can be solved, the following 2 issues are we in the actual operation of the process encountered, but also share.

1. The result of coverage is that only the files that are tested are displayed, not all compiled code is used as the denominator of coverage

In fact, it can be seen that the entire coverage generation process is a 4-step process, typically through peripheral scripting, or Makefile/shell/python, to automate the entire process. 2 ideas to solve this problem, all through the outer camouflage. The first, is to modify the Lcov App.info, intermediate files, find other files and coverage information where, combined with makefile, all the compiled source program check whether stored in App.info, if not, add in. The second camouflage, is camouflage *.gcda, there is no source coverage information because the file is not called to, no response to the GCDA file generated. Toast (http://toast.taobao.org/) is achieved through the first camouflage, and more understanding of the need to see the open source code. 2. Background process coverage data collection;

In fact, the above coverage information can not only be used for unit testing, but also for functional testing. However, functional testing, the general Linux C + + is the implementation of a daemon process, and coverage of the condition is that the program needs to exit normally, that is, the user code call exit Normal end, the Gcov_exit function is called, it continues to call __gcov_flush Function output statistics to the *.GCDA file. The same 2 ideas can solve this problem,

First, add a signal handler to the test program, intercept common forced exit signals such as SIGHUP, SIGINT, Sigquit, SIGTERM, and actively call exit or signal function output statistics in handler __gcov_flush Results. But this needs to modify the program under test. This is also our previous general practice. But after attending a lecture with no classmates, I found the second better method.

Second, using the dynamic library preloading technique and the constructor attribute of the GCC extension, we can encapsulate the Signalhandler and its registration process into a separate dynamic library and implement the signal interception registration when the dynamic library is preloaded. In this way, you can simply use the following command line to achieve the exception exit when the statistical results output.

Other programming languages in our engineering practice, there are other programming languages, all related to the production of coverage, our engineering practice recommended the following methods,

• C + +, the method described in this article;
• Java, Maven Cobertura plugin;
• Python, Pyunit + coverage.py;
• PHP, PHPUnit +–coverage-html;
• Perl, Test::class and Devel::cover;
• Shell, ShUnit2 + Shcov;

How to generate code coverage for C + + projects under Linux