Web-side PHP code function coverage test solution-php Tutorial

Web-side PHP code function coverage test solution 1. code coverage

There are many levels to measure code coverage, such as row coverage, function/method coverage, class coverage, and branch coverage. Code coverage is also an important criterion for measuring testing quality. for a black box test, if you are not sure whether your test case has actually run every line of code in the system, discounts are always required for Test integrity. Therefore, almost all programming languages in the industry have their own code coverage solutions. PHP, the most beautiful language in the world, is no exception. PHPUnit and Spike PHPCoverage provide a set of xdebug-based code coverage testing solutions. In this article, I will introduce my own solutions for testing the coverage rate of PHP code functions for specific business scenarios.

2. Business Background

Suppose we develop a website online and hand it over to our business testing colleagues for functional testing. So how did they test it? Generally, the developer deploys the website, and the tester tries all the functions on the Internet, including abnormal usage. For business testing, as long as I have tested all the functional points and tested all the abnormal usage conditions, this is done. But for the development, I'm curious: have you ran all the code I wrote? Will there be some code that can be triggered only in special circumstances, and you have never been able to detect these situations? In this case, code coverage may be required.

In fact, I first thought of xdebug to test the coverage rate. we only need two or three functions, as shown below:

Xdebug_start_code_coverage (); // starts to collect code row overwrites. xdebug_get_code_coverage (); // gets the code file name and row number xdebug_stop_code_coverage () that has run so far (); // stop collecting code line overwrites

The interface provided by xdebug can be used to test row coverage. can this meet the requirements? In fact, row coverage is fine-grained. in actual projects, developers may fine-tune the code. For example, you ran. the PHP file contains 10th rows. php is fine-tuned. php adds two lines of code between lines 9th and 10th. Therefore, the original 10th rows were changed to 12th rows, while the xdebug row overwrite information only recorded the row number ...... Isn't the previous data inaccurate... I think function coverage is a good granularity. In relatively mature projects, there are few large-scale function changes. However, xdebug does not provide function override interfaces.

As a result, we now encounter the following scenarios:

[1] you want to measure the list of all functions covered in a test, know the total number of functions in the project, and check whether the coverage rate is high enough.

[2] After the test is completed, a coverage report should be generated to visualize the coverage of the code.

[3] The complete test process is as follows:

The plug-in means some preparations before the test is executed.

3. Function coverage solution (1) principle

Xdebug naturally provides support for row coverage. we need to calculate the function coverage rate by ourselves. Function coverage requires two data points: one is which functions are executed and the other is the total number of functions in the file.

The total number of functions in the file. since we cannot execute all the functions, this part can only be implemented through static code scanning. If it is in C ++ or Java, lexical analysis tools may be required. However, in the face of the most beautiful language PHP, we do not need to be so complicated. Since PHP4.3, PHP Zend Engine has a built-in tokenizer function to help developers perform source code lexical analysis. We only need to find the lexical law corresponding to the function defined in PHP, so that we can easily get all the functions in the specified PHP file.

The interfaces defined by tokenizer are also very simple:

array token_get_all (string $source)

This function parses the file and splits the php source code into an array composed of tokens.

string token_name (int $token)

Convert the token in the integer form to the string form. Similar to the strerror function in C. With tokenizer, you can design a finite state machine based on the laws and formats defined by php functions to complete full function parsing. I have written this part of the code, which is relatively simple. I will take it out separately for your reference only: PHPFunctionParser

Another difficulty in calculating function coverage is obtaining the list of executed functions. This place has led us to some detours. In the beginning, the simplest method is to obtain the executed row through xdebug, and we can use the row number to reverse push which function the row belongs. However, the row number information obtained by each request is very large. if a request executes 1000 rows, it is necessary to make 1000 judgments, and the efficiency will be relatively poor. After some research, we found that xdebug provides the function trace function, which can obtain the function call relationship in a request. However, the function name cannot be obtained, but the file where the function is located. As a result, I did another research and found that Reflection, given the method name and class name, can be inferred in which file it is defined. So we use function trace to save the function call relationship to a temporary file, and then parse the file to get the name of the function to be executed (if it is a class method, it is "class name :: function name), and then use the reflection mechanism to roll out the file that defines this function. I once again realized the power of the most beautiful language in the world.

(2) insert pile

To lower the usage threshold, it is better to change the PHP source code as little as possible. The principle of xdebug information collection is to call xdebug_start_code_coverage and xdebug_stop_code_coverage respectively to control the start and end of coverage information collection. Therefore, you must inevitably change the source code. The solution here is to register xdebug_stop_code_coverage with register_shutdown_function as a program (similar to the C language atexit function) that must be run before the php program ends, and encapsulate it into a file, then, in the first line of the source code, you can use the require file. If your PHP framework is CodeIgniter and all requests have a uniform index. php frame, you only need to change this file and only one line of source code is changed! In fact, basically all PHP frameworks currently use an index. PHP file as the inbound port for all requests.

We only added a sentence to the first line of index. php in the entry file to change the source code:

Phpcoverage. php core code logic is roughly as follows:

 
  
(3) Information Storage
  
Our function coverage test has an idea. use the function trace of xdebug to obtain the call relationships of all the functions in a request, obtain all the functions that have been executed, and output them to the file, use file parsing and reflection to obtain the name of the executed function and the file where the function is located. Store the information in a database or file.
    
  
When we tried Spike before, we found that the information was saved to files in xml format, and the data redundancy was high. as a result, the files were already very large after several tests. This is clearly not what we want to see. Therefore, when storing data, we directly serialize the data in json format. strings exist in the file, greatly reducing the file size. At the same time, we separate the Request source IP addresses and dates to store different files. In this way, the request data from each machine can be clearly viewed every day, taking another step toward "precision", allowing the tester to precisely monitor each request. Is part of the data files we collect in business practices:
    
  
  
    
  
In this way, the row and function information covered by each Web request from any IP address will be recorded in the file. In general project tests, only a few testers are in use, so some performance issues do not need to be considered.
4. report generation
  
The principle of generating coverage data is described above. However, we have obtained only one copy of the data file. how can we summarize the data into a complete report? Therefore, we need to write a script to parse the generated data file. Our practice is to use the template of the open-source tool spike phpcoverage and add our own code logic, especially the function coverage statistics that this tool does not have. The report generated on the web page we tested is as follows:
    
  
  
    
  
The figure shows the row coverage rate, function coverage rate, and total coverage rate of each file. If you need more accurate data, you can click the File connection to check which lines of code are overwritten (blue indicates overwriting, and red indicates not overwriting ):
    
  
  
5. Summary
  
During Web testing, code coverage is an important indicator to measure the testing quality. We hope to use this method to make it as accurate as possible. after the test is executed, we can see exactly which line of code has been executed and which line has not been executed. Analyze the cause of the failure to improve the test case. The process of using the tool is also very simple, plug-in => test => collect data => report. This solution minimizes the impact on the business code. you only need to change the code line. Even if there is a problem in the middle, you can quickly restore the code to its original form. Reliable testing and development.
    
  
However, it should be emphasized that the test is complete without overwriting all the code. If it is not covered, it must be incomplete. So the biggest significance of this solution is to find some of the missing code in the test and find some problems. In fact, it can also help new employees understand the entire project code structure. we can clearly understand what code each browser request is running on the server.