Php-tokenizer's learning experience sharing

In a project, you need to analyze the PHP code to isolate the corresponding function call (and the location of the source code). While this is also possible, it is not the best way to consider both efficiency and code complexity.

In a project, you need to analyze the PHP code to isolate the corresponding function call (and the location of the source code). While this is also possible, it is not the best way to consider both efficiency and code complexity.

Query the PHP manual, found that in fact, PHP has built-in parser interface, that is, PHP tokenizer, this tool is what I want. The use of PHP Tokenizer can be simple, efficient, accurate analysis of the composition of PHP source code.

Instance

The official site has few documents for Tokenizer, but that doesn't affect our understanding of it. The Tokenizer component contains only two functions: Token_get_all and Token_name, which are used to parse the PHP code and get the identifier name corresponding to the code.

The following is a simple example of how to use these two functions:

The following are the referenced contents:

$code = ' <?php echo ' string1 '. string2 ";?> ';
$tokens = Token_get_all ($code);
foreach ($tokens as $token) {
if (Is_array ($token)) {
Line number, identifier literal, corresponding content
printf ("%d-%s\t%s\n", $token [2], token_name ($token [0]), $token [1]);
}
}

The corresponding output is

The following are the referenced contents:

1-t_open_tag <?php
1-t_echo ECHO
1-t_whitespace
1-t_constant_encapsed_string "String1"
1-t_constant_encapsed_string "string2"
1-t_whitespace
1-t_close_tag?>

By the way, $token if you are an array, then the corresponding three array members are token identifiers (which can be literal by token_name), the corresponding source code content, and the corresponding line number.

In the case of $token as a string, one of the possible cases is a constant of t_constant_encapsed_string, which should be noted when parsing the code. If you care about this, consider using the code here.

Yes, the invocation is very simple, and of course our ambitions are far greater than the simple cycle of writing. We can use this component to do practical things, such as the following code to "compress" the PHP code, to remove the line break, blank and comment

The following are the referenced contents:

/**
* "COMPRESSED" PHP source code
*
* @see Http://c7y.phparch.com/c/entry/1/art,practical_uses_tokenizer
*/
Class Compactcode
{
static protected $out;
static protected $tokens;

static public Function Compact ($source)
{
Parsing PHP Source Code
Self:: $tokens = Token_get_all ($source);
Self:: $out = ';

Reset (self:: $tokens);

Recursively judge the type of each marker
while ($t = current (self:: $tokens)) {
if (Is_array ($t)) {
Filter whitespace, annotations
if ($t [0] = = T_whitespace $t [0] = = T_doc_comment $t [0] = = t_comment) {
Self::skipwhiteandcomments ();
Continue
}
Self:: $out. = $t [1];
} else {
Self:: $out. = $t;
}

Next (self:: $tokens);
}

Return self:: $out;
}

Static Private Function skipwhiteandcomments ()
{
Add a space to split the keyword
Self:: $out. = ';
while ($t = current (self:: $tokens)) {
Again greedy find
if (Is_array ($t) && ($t [0] = = T_whitespace $t [0] = = T_doc_comment $t [0] = = t_comment)) {
Next (self:: $tokens);
} else {
Return
}
}
}
}

The invocation is simple and requires only the use of the

The following are the referenced contents:

Compactcode::compact ($source _code);

, the returned string is the content after the compression.