Tselect (test prioritization tool) using Intel Compiler (5) PGO (7) PGO Tool)

Reference manual:

Http://software.intel.com/sites/products/documentation/studio/composer/en-us/2011Update/compiler_c/index.htm

Note: this series of articles is personal notes. If any of them is incorrect, please refer to the relevant official documents. If any errors are found, I will try to update and modify them as much as possible. In addition, the following content does not guarantee that all versions of the compiler are correct, and the implementation of the compiler may also have some changes. For details, refer to the official documentation.

For more information, see the additional description in http://blog.csdn.net/gengshenghong/article/details/7082448.

Summary: The tool here is tselect, test prioritization tool. Literally, the test priority tool should also be related to the test, so the tester may be easier to understand, the following describes the content and usage of tselect. I do not know much about the testing theory.

For test priority, refer:

Http://blog.csdn.net/lengyue_112/article/details/375342

Http://edu.21cn.com/ruankao/g_185_695002-1.htmand so on

(1) Role of test priority

For more information, see.

(2) Introduction to the use of the PGO test priority tool tselect

1. required files

Similar to the codecov tool, tselect involves the following files: SPI files compiled using the/qprof-gen: srcpos option, DPI files merged by Dyn files.

2. Use tselect

Use: tselect-dpi_list File

-Dpi_list is a required option, and there are other Optional options.

The following is a typical model used by the tselect tool:

For a simple understanding, the first step is to use/qprof-gen: srcpos to obtain the SPI file and auxiliary executable file,Multiple timesRun the auxiliary executable files with different test data to obtain their DPI files, put the DPI files in a text file in the form of a list, and use the tselect tool.

(3) instance:

Run the following code ):

// File: tselect.cpp  #include <stdio.h>    void foo()  {      printf("This is foo\n");  }    void bar()  {      printf("This is bar\n");  }    void foobar()  {      printf("This is foobar\n");  }    int main(int argc, char* argv[])  {      int count = argc;      switch(count)      {          case 0:              printf("Case0\n");              break;          case 1:              printf("Case1\n");              foo();              break;          case 2:              printf("Case2\n");              bar();              break;          case 3:              printf("Case3\n");              foo();              bar();              break;          case 4:              printf("Case4\n");              foobar();              break;      }  } 

@echo OFF  rem #File: build.bat  set exeargs=arg1 arg2 arg3rem #can set env vars in command line or in this file before run this bat file.    if exist *.obj del *.obj  if exist *.exe del *.exe  if exist *.dyn del *.dyn  if exist *.lock del *.lock  if exist *.dpi del *.dpi  if exist *.spi del *.spi  if exist *.spl del *.spl    icl tselect.cpp -o tselect.exe /Od /Qprof-gen:srcpos /nologo  tselect.exe %exeargs%  profmerge

The DPI and SPI corresponding to the execution of case1234 are obtained respectively, as follows:

Pgopti. SPI
Pgopti1.dpi
Pgopti2.dpi
Pgopti3.dpi
Pgopti4.dpi

The list.txt file is created as follows:

pgopti4.dpipgopti3.dpipgopti2.dpipgopti1.dpi

Then run:

tselect -dpi_list list.txt

The output is as follows:

Intel(R) C++/Fortran Compiler test prioritizer for applications running on Intel(R) 64, Version 12.1 Build 20111128Copyright (C) 1985-2011 Intel Corporation.  All rights reserved.WARNING: no SPI file name was specified, assumed .\pgopti.spi.Total number of tests   =  4Total block coverage    ~  91.67Total function coverage ~ 100.00 num  %RatCvrg  %BlkCvrg  %FncCvrg  Test Name @ Options ---  --------  --------  --------  -------------------   1    50.00     45.83     75.00   pgopti3.dpi   2    72.73     66.67    100.00   pgopti4.dpi   3    86.36     79.17    100.00   pgopti2.dpi   4   100.00     91.67    100.00   pgopti1.dpi

Note: Running tselect requires a list file and multiple DPI files, as well as an SPI file (pgopti by default. SPI), no source code (this is different from codecov, codecov requires source code, otherwise the result is incorrect ).

1. Meanings of Related Basic parameters

The output parameters of the preceding tselect statement are described as follows:

(Total number of testsparts: the total number of DPI files listed in list.txt is determined by list.txt.

(Total block coverage) the block coverage rate is 91.67%.

(Total function coverage) the total function coverage rate is 100%.

The following table corresponds to each test in the list.

The following information is displayed from the above results. The test corresponding to running pgopti3.dpi covers the basic block of program 45.83%, it accounts for 50% of the total block coverage rate available for all four tests (45.83%/91.67% = 50%). In addition, the corresponding test covers 75% of the function coverage (a total of Foo, bar, foobar, and main functions, while case3 covers Foo, bar, and main functions, so it is 75%); after running the test corresponding to pgopti4.dpi,AccumulationIt covers 66.67% of the basic blocks of the program, and accounts for 72.73% (66.67%/91.67% = 72.73%) of the block coverage of the total four tests. In addition, it corresponds to the testAccumulationCovers 100% of the function coverage (case4 will overwrite the foobar function, so all functions will be overwritten); other similar.

Note:

A. Here is a sequence problem. The subsequent tests are the cumulative results. In addition, the sequence of pgopti310442.1622.161is as definite as possible, and it does not matter with the list order of list.txt. No matter what order of listing, the obtained tselect results are in the order of pgopti3, 4, 2, and 1! Why? (Update: See the following content to get the answer)

B. Because cumulative results are displayed, if a test does not change the data such as coverage of the previous test results, the data is not displayed in the following table. For example, list.txt:

pgopti4.dpipgopti3.dpipgopti2.dpipgopti1.dpipgopti4.dpipgopti3.dpipgopti2.dpipgopti1.dpi

The result is as follows:

Total number of tests   =  8Total block coverage    ~  91.67Total function coverage ~ 100.00 num  %RatCvrg  %BlkCvrg  %FncCvrg  Test Name @ Options ---  --------  --------  --------  -------------------   1    50.00     45.83     75.00   pgopti3.dpi   2    72.73     66.67    100.00   pgopti4.dpi   3    86.36     79.17    100.00   pgopti2.dpi   4   100.00     91.67    100.00   pgopti1.dpi

The total number of tests is 8, but the table has only four rows.

2. Estimate the minimum number of tests. The display sequence is irrelevant to the list order of list.txt. Why? To put it bluntly, I didn't understand exactly what the tselect tool is! Next, I will analyze my understanding of this tool. PS: The Role of tselect may be different from the test priority concept mentioned above. It should be understood or started from the document. The above example shows the basic function of tselect. The document is "minimizing the number of tests ". The reason why I didn't understand it is because of my understanding of the meaning of minimize, which has a meaning of "... "If you know the meaning of this layer, the above sequence problem is easy to understand. Its role is to" estimate the minimum number of tests ". My understanding is: based on the data of Multiple tests, tselect analyzes how to perform the tests (in what order the tests are executed) to achieve the highest coverage rate as soon as possible, this is a question of "test priority. For example, in the above example, it is clear that pgopti3.dpi (executes case3), which calls the Foo and bar functions, so that more functions can be overwritten as soon as possible, if you run pgopti4.dpi first, the maximum function coverage rate is 100%. If you run pgopti1.dpi or pgopti2.dpi first, the function coverage rate is not changed (of course, the block coverage rate is changed ). In short, tselect gives a recommended priority to execute these tests to achieve maximum coverage. For some tests, the coverage will not change and will not be listed in the results.

3. Estimate the minimum execution time

With the above understanding, the following content is simple. You can add time data after each test row in list.txt, and then use the-mintime option to estimate the sequence of the minimum execution time for all tests (if the-mintime option is not used, you can also write a time after each row is tested. This is not required and it has no impact on writing. If the-mintime option is used, each row must have a time, this time must be in a certain format ). The following is an example of list.txt:

pgopti4.dpi 00:00:40:04pgopti3.dpi 00:00:30:03pgopti2.dpi 00:00:20:02pgopti1.dpi 00:00:10:01pgopti1.dpi 00:00:10:01

Note: there can be any number of spaces between the DPI file name and the subsequent time, but there cannot be tabs (this is a bit unfriendly ).

Run:

tselect -dpi_list list.txt -mintime

Get:

Total number of tests   =  5Total block coverage    ~  91.67Total function coverage ~ 100.00Total execution time    =     1:50:11 num  elapsedTime   %RatCvrg  %BlkCvrg  %FncCvrg  Test Name @ Options ---  -----------  --------  --------  --------  -------------------   1        10:01    36.36     33.33     50.00   pgopti1.dpi   2        30:03    59.09     54.17     75.00   pgopti2.dpi   3      1:00:06    77.27     70.83     75.00   pgopti3.dpi   4      1:40:10   100.00     91.67    100.00   pgopti4.dpi

Therefore, a total of five tests, the total execution time only needs to reach the coverage rate of the primary node at 1:50:11, And the sequential sequence is sequential from less to more. The last pgopti1.dpi (fifth test) in list.txt does not need to be performed because its execution has no impact on the coverage rate. (Of course, we intentionally use the same DPI data here, So we certainly won't change the coverage rate, but different DPI can also not change the coverage rate ).

4. Other options

Here, we have basically understood the role and usage of tselect. Next we will look at other options.

-Help: displays help.
-Dpi_list file: The only required option that specifies the list file.
-SPI file: Specify the SPI file.
-O file: Specifies the report file output by tselect. It is output to the console by default. After-O is specified, it is output directly to the file, but not to the console.
-Mintime: estimate the minimum execution time.
-Nototal: the report does not contain the total information.
-Verbose: the tool displays more log information during running and more detailed information in the report.

-Cutoff value: the parameter is a value. It specifies that the tselect tool will terminate when the cumulative block coverage rate reaches the value % of the expected total block coverage rate. For example, tselect-dpi_list list.txt-cutoff 10.00 indicates that the block coverage is terminated as long as the block coverage rate reaches 10%. The following is a column:

Tselect-dpi_list list.txt-cutoff 49

Get:

Total number of tests   =  5Total block coverage    ~  91.67Total function coverage ~ 100.00 num  %RatCvrg  %BlkCvrg  %FncCvrg  Test Name @ Options ---  --------  --------  --------  -------------------   1    50.00     45.83     75.00   pgopti3.dpi

In the same example, use tselect-dpi_list list.txt-cutoff 51 to obtain:

Total number of tests   =  5Total block coverage    ~  91.67Total function coverage ~ 100.00 num  %RatCvrg  %BlkCvrg  %FncCvrg  Test Name @ Options ---  --------  --------  --------  -------------------   1    50.00     45.83     75.00   pgopti3.dpi   2    72.73     66.67    100.00   pgopti4.dpi

-Srcbasedir dir:
-Comp: