Web-side PHP code function coverage test solution

1. About Code Coverage

There are many levels of code coverage, such as line coverage, function/method coverage, class coverage, branch coverage, and so on. Code coverage is also an important measure of test quality, and for black-box testing, if you're not sure if your test case actually runs through every line of code in your system, you'll always have to discount the integrity of the test. As a result, the industry has its own set of code coverage solutions for almost every programming language. The most beautiful language in the world PHP is certainly no exception. PHPUnit and Spike Phpcoverage provide a suite of xdebug-based code coverage testing scenarios. In this article, I'm going to tell you about the solution to the specific business scenario that I've come up with for my PHP code function coverage test.

2. Business background

Suppose we developed a Web site online and handed it to the business test colleagues for functional testing. And how did they test it? Typically, the developer deploys the site, and the testers try out all the features on the web, including some unusual use cases. For business testing, as long as I have all the functional points measured, and all the abnormal use of the test, then it is done. But for development, I'm curious to see if you've run all the code I've written. Will there be some code that can be triggered only under very special circumstances, and you've never been able to measure the situation? At this point, you may need code coverage to do the same.

In fact, I first thought of xdebug to test coverage, only need two or three functions, as follows:

Xdebug_start_code_coverage (); Start collection of code line coverage xdebug_get_code_coverage (); Get the code name and line number Xdebug_stop_code_coverage () as of the current run; Stop collection of code line overrides

Can the interface provided by Xdebug be used to test line coverage, and does this satisfy the requirements? In fact, the line coverage granularity is a bit thin, and the developer may fine-tune the code in the actual project. For example, this test, you ran across the 10th line of the a.php file, but one day I tweaked a.php, adding two lines of code between lines 9th and 10th of a.php. As a result, the original line 10th becomes the 12th row, and the Xdebug line overwrite information only records the line number ... So the data before is not accurate ... Thinking twice, I think the function overlay is a good particle size. In relatively mature projects, there are few cases of large-scale function changes. The problem, however, is that Xdebug does not provide an interface for function overrides.

So, the scene we are now encountering is:

"1" Want to test the list of all the functions covered in a particular testing, know how many functions the project has, and calculate whether coverage is high enough.

After the "2" test is complete, generate a coverage report to visualize the coverage of your code.

The process for the "3" complete test is as follows:

The insertion of piles means some preparatory work before the test is performed.

3. Function Coverage Solution

(1) principle

Xdebug is born with support for line coverage, and we want to figure out the function coverage ourselves. Function coverage requires two points of data, one for which functions are executed, and one for the total number of functions in the file.

The total number of functions in the file, since we cannot execute all the functions again, this part can only be implemented by code static scanning. If you are in C + + or Java, you may need lexical analysis tools, but in the most beautiful language in front of PHP, we do not need to be so complex. Starting with PHP4.3, PHP Zend engine has built-in tokenizer function to help developers do the source lexical analysis. We just need to find the lexical law that defines the function in PHP, so we can easily get all the functions in the specified PHP file.

The interface defined by Tokenizer is also very simple:

Array Token_get_all (String $source)

This function does file parsing and splits the PHP source code into an array of tokens.

String token_name (int $token)

Converts a token in integer form to a string. Similar to the Strerror function in C language. With the Tokenizer, we can design a finite state machine according to the law and format defined by PHP function, and then complete the analysis of the whole function. This part of the code, I wrote a relatively simple, take it out alone, for your reference: Phpfunctionparser

Another difficulty in finding function coverage is to get a list of functions that are executed. This place let us go a few detours. At first, the simplest way, since we get the line through Xdebug, we can use the line number to push back to which function this line belongs to. However, each request to obtain the number of lines of information is very large, if a plea executed 1000 lines, it will be 1000 times to judge, the efficiency will be relatively poor. After a survey, it was found that Xdebug provides function trace, which can get the function call relationship in one request, but get the function name, but can't get the file. So, once again, the discovery of Reflection, given the method name and class name, can be rolled back to which file it is defined. So we use function trace to put a temporary file in the call relationship, and then through the file parsing, get the function name executed (if it is a class method, it is "class name: Function name" form), and then through the reflection mechanism to introduce the definition of the function of the file can be. Once again, we are experiencing the power of the most beautiful languages in the world.

(2) Insert pile

In order to reduce the use threshold, we can change the PHP source code as little as possible. Xdebug collects information by invoking Xdebug_start_code_coverage and Xdebug_stop_code_coverage, respectively, to control the start and end of coverage information collection, so it is unavoidable to change the source code. Our workaround here is to register xdebug_stop_code_coverage through Register_shutdown_function as a program (similar to the C-atexit function) that must run before the PHP program ends. Encapsulate it in a file, and then require the file in the first line of the source code. If your PHP framework is codeigniter all requests have a unified portal index.php framework, then only need to change this one file, the source code only one row of changes! In fact, basically all PHP frameworks currently use a index.php file as a portal for all requests.

Our changes to the source code only the first line of the entry file index.php added a sentence:

The phpcoverage.php core code logic is roughly as follows:

     
  
(3) Information storage
    
  
Our function coverage test has the idea of using Xdebug's function trace to get the call relationships of all functions in a request, to get all functions executed, to output to a file, to obtain the function name and the file where the function is executed through file parsing and reflection. Store this information in a database or file.
    
  
Before trying out spike, we found that this information was stored in XML format, the data redundancy is very high, resulting in a few tests, the file is very large. This is obviously not what we want to see. Therefore, when the data is stored, we directly serialize the data in JSON format, the file in the form of a string, greatly reducing the file size. At the same time, we store different files separately by requesting the source's IP and date as separate. In this way, the request data from each machine can be viewed at a glance, and a step in the direction of "precision" can be accurately monitored for each request of the tester. Are some of the data files we collect in our business practices:
    
  
  
    
  
In this way, every Web request from any IP will be logged to the file with the row and function information it overwrites. For general project testing, there are only a few testers in use, so there are no performance issues to consider.
    
  
4. Report generation
    
  
It says the principle of generating coverage data, but what we get here is just a copy of the data file, how do we summarize it into a full report? This requires us to write a script to parse the data file that was just generated. Our approach is to draw on the template of the Open Source tool spike phpcoverage and add our own code logic, especially the function coverage statistics that the tool does not have. The report generated by our own test Web page is as follows:
    
  
  
    
  
You can see the line coverage, function coverage, and total coverage statistics for each file in the diagram. If you need more accurate data, you can click the file connection to see what lines of code are covered (blue is overlay, red is not covered):
    
  
  
    
  
5. Summary
    
  
When doing web testing in a business test, the coverage of the code is an important indicator of the quality of the test. We want this method to be as "precise" as possible, to see exactly which line of code has been executed and which has not been executed after the test is executed. Analyze the reasons for not being executed, thus improving the test case. The process of using the tool is also simple, and the pile-and-test = collect the data and report it. And this solution minimizes the impact on business code, only one line of code is needed to change it. Even if there is a problem in the middle, you can quickly restore the code to its original appearance. Let the test rest assured, let development also rest assured.
    
  
The last thing to emphasize, though, is not that all the code is covered and that the test is complete. But not covered, it must be incomplete. So the big point of this solution is to be able to find some missing code in the test and find some problems. In fact, it can also help new employees understand the entire project code structure, we can clearly know, each of their browser requests, exactly what code on the server to run.

  
