Once Nginx 504 Gateway time-out error troubleshooting, resolution Records _nginx

Remember a puzzling site lost response to the troubleshooting. The site has been using Nginx to do proxy back-end Apache running PHP to provide services. Apache often does not have a regular time to be unable to service loss of response, and then nginx appear "504 Gateway time-out"
Looking at the error log also does not see anything that is the Apache bug (in fact not, the following will say why).

Perhaps older people do not love tossing, willing to stay the same, using monitoring tools, each received the alarm after the restart of the Apache barely maintained. Finally one day I am bored, is not handles the PHP, I do not use the Apache row, Rage uses the source installation php-fpm to move to the PHP-FPM to run the PHP. Installing PHP is not troublesome, the use of source installation is still very smooth, the only thing to do is to set up the PHP worker worker process log output PHP error log.

When everything is ready, change the original proxy_pass to Fastcgipass.

Copy Code code as follows:

Upstream apachephp {
Server www.jb51.net:8080; #Apache1
}

....
Proxy_pass http://apachephp;

Replace into

Copy Code code as follows:

Upstream PHP {
Server 127.0.0.1:9000;
}

...
Fastcgi_pass PHP;

You can move Apache running PHP to PHP-FPM to run.
I thought this would be a peace of the day, and the migration is really no problem, but if you don't analyze the root cause of the problem. The problem will come to the door, the next day Nginx also reported 504 of the gateway timeout. This time there is no Apache, Apache has finally cleared the relationship.

That should still be on nginx and PHP-FPM, check out the Nginx error log, you can see

Copy Code code as follows:

[ERROR] 6695#0: *168438 upstream timed out (110:connection timed out) while reading response header from upstream,
...
Request: "get/kd/open.php?company=chinapost&number=pa24977020344 http/1.1", Upstream: "fastcgi:// 127.0.0.1:9000 ", Host:" Www.jb51.net "

See here basically ruled out nginx suspicion, Nginx is waiting for PHP processing "get/kd/open.php?company=chinapost&number=pa24977020344 http/1.1" timeout out.

Restart PHP-FPM immediately, the problem is gone, the website can be accessed.

Visit the page again, still no response, but at the same time access to other pages normal, the page refresh a few times, the entire site is bad Gateway timeout.

The problem is narrowed down to the PHP script.

Copy Code code as follows:

Netstat-napo |grep "PHP5-FPM" | Wc-l

To view the PHP process has reached the upper limit of 10 in the configuration file, there is a feeling that everyone has been open.php this script jammed.

What is this script for? This script is to collect The courier information, which used the Php_curl.

PHP script will force out if the execution time exceeds the configuration item in php.ini max_execution_time not result.

Check out the php.ini max_execution_time is really good, the value is 30.

Universal Google put in use, after the continuous Google to get the following words

The Set_time_limit () function and the configuration instruction Max_execution_time only affect the time that the script itself executes. Any execution of a script that occurs in a system call, such as a systems (), a stream operation, a database operation, and so on, is not included when the script is run.

This means that if the script performs other operations in the time that is not counted in the script run time, if you do not set the timeout, then PHP will wait for the result of the call.

View open.php source file a look, sure enough, did not set the Curl timeout time.

Add the following two lines and refresh again after the problem is resolved.

Copy Code code as follows:

curl_setopt ($ch, Curlopt_connecttimeout, 10); Timeout on Connect
curl_setopt ($ch, Curlopt_timeout, 10); Timeout on response

Of course, in addition to this method, PHP-FPM also provides parameters for us to force the killing of long, sterile processes, except that the parameter is not turned on by default.

A PHP-FPM configuration file can be set up with a parameter request_terminate_timeout, a timeout to request termination, and kill if the request executes more than this time.

It also has a parameter request_slowlog_timeout, which is used to record the slow request log.

command line to run PHP, you can use this code

Copy Code code as follows:

$real _execution_time_limit = 60; Time limit

if (Pcntl_fork ())
{
Some long time code which should is
Terminated after $real _execution_time_limit seconds passed if it ' s not
Finished by
}
Else
{
Sleep ($real _execution_time_limit);
Posix_kill (Posix_getppid (), SIGKILL);
}