Remember the pit of the Zend engine once.

The open platform has a notification requirement that requires a consuming process to constantly read the data in buffer and then consume and delete it. So, I use cron every minute from a PHP process to read the database, and do the work of notification. Also, a file lock is added to ensure that only one process is working at a time.

One day, suddenly found that the data in the buffer unabated, the process of the work log has not been output. PS look at the process, or in the running state,ltrace and strace are empty, it looks like a while (true) logic, looked at the CPU, indeed, has soared to 98%. GDB to this process, find the process tangled in Php_shutdown_handler this function, PHP error log shows out of memory error. At this point, the cause of the accident has been taken into account.

1. Cause of the accident

We registered a callback function Php_shutdown_handler in the exit event of the PHP process, and we did the report monitor and alarm in the function. We can imagine what would happen if the process exited because of a low memory condition.

As imagined, the Zend engine entered the dead loop state shown, because the memory is not enough to cause the process to exit, before exiting into our registered Php_shutdown_handler function, in this function requires a process to leave a note, but the last words to half, once again out of memory, Once again, the exit event is triggered and entered the Php_shutdown_handler function, thus entering the infinite recursive loop, and can no longer go to the logic of proper exit. Which leads to the phenomena that we have observed before.

2. Workaround

2.1 Using Ini_set () to increase the memory limit of the process

This is the first solution I think of. Since there are hundreds of processes running on the same machine in CGI mode, Zend to protect system resources by default, each PHP process has a memory limit, and each time Zend needs to request memory, it checks the Memory_limit in the configuration item and exits the process if the limit is exceeded. This configuration item can be changed by modifying the parameters in the php.ini, or by invoking the Ini_set (' Memory_limit ') function to dynamically modify the process execution. So, this idea is naturally generated, that is, calling Ini_set in Php_shutdown_handler to extend the original memory limit (MEMORY_LIMIT) to twice times.

$mem _limit= Ini_get (' Memory_limit ');//64m$num= Intval (substr ($mem _limit,0, strlen ($mem _limit)-1));//64$last=$mem _limit[Strlen ($mem _limit)]-1;//m$mem _limit=$num*2.$last;//128mIni_set (' Memory_limit ',$mem _limit);

The inconvenient thing is that the ini_get () call to get the Memory_limit is a "64M" format of the string, want to double words, need to do some string transformation, and these string transformation operations still need to request memory. So the above method is not feasible.

Well, let's use a violent point, just change the memory limit to 1G.

ini_set(‘memory_limit‘‘1024M‘);

Tested, still did not solve the problem. We know that PHP configuration items are stored in a configuration hash table in the Zend, although we only modify one of the values, but also take up some memory (actually measured down to about 500B), in the extreme case still cannot solve the problem.

2.2 A pit, then release.

Since the method of adding memory has failed, let's try another method. A more intuitive method is to occupy a hole first, the need for memory, the release of the pit, and then use this part of memory on the line.

Step1:

Pre-allocating a 1M of memory in a global variable before registering the function.

//预先非配内存$_SERVER[‘php_shutdown_mem_pre_alloc‘] = str_repeat(‘*‘10241024 );//注册回调函数register_shutdown_function(‘php_shutdown_handler‘);

Step2:

reclaims memory in the callback function and reclaims the pre-allocated global variables.

//PHP exception error handling function  function  php_shutdown_handler   ()  { unset  ($_server  [     ' Php_shutdown_mem_pre_alloc ' ]);  $error  = Error_get_last ();    //if that's true. Returns null  Phpshutdownhandler::report ( $error );}  

OK, let's test it again. The results were predictable and failed.

Recall that it was a bit too hasty at that time, do you think PHP unset is a delete operation? That's a mistake, and in general PHP's magical scripting language has its own garbage collection mechanism. If you want to enable GC now, you need a little bit of action.

//PHP exception error handling function  function  php_shutdown_handler   ()  { unset  ($_server  [  ' Php_shutdown_mem_pre_alloc ' ]); gc_collect_cycles ();    $error  = Error_get_last (); //if that's true. Returns null  Phpshutdownhandler::report ( $error );} 

The problem is solved by proactively calling gc_collect_cycles () to enable garbage collection immediately.

3. Some optimizations

Of course, such code cannot be published immediately, because the production environment code is registered in the PHP auto prepend to exit the callback function, which means that all PHP scripts will execute this code. If both the pre-allocation and the process end are logical, the resulting time-cost may be wasted on all server performance optimizations. In addition, because each process is pre-allocated 1M of memory, if a machine is configured with 500 PHP-FPM processes, then when they execute simultaneously, the pre-allocated useless memory accounted for 500M. This is not tolerated in the peak memory consumption.

3.1 Pre-allocated memory time consumption test

Our pre-allocated function, such as str_repeat (), has been tested, averaging around 0.7ms, which is negligible relative to the CGI of a production environment.

3.2 Escalation method Memory consumption test

The results of the test tests for memory consumption show that a single escalation consumes about 260KB of memory. If a machine is enabled for 300 php-fpm, and each pre-allocates 400KB of memory, the pre-allocated memory peak is approximately 120M. After discussion, this kind of overhead is within a reasonable range, and the problem is solved.

In summary, in the version of PHP 5.3, Zend does not guarantee that _onexit registered functions can only be executed at most once (whether the subsequent version fixes the bug has not been verified), and PHP as a scripting language is not easy to directly manipulate memory, so that the problem is not particularly intuitive solution.

Record the pit event of the Zend engine once