Nginx 502 Bad Gateway error issue collection

Nginx 502 Bad Gateway error issue collection(2010-11-18 13:51:37) reproduced

Tags: gossip

Category: Work

Nginx 502 Bad Gateway error issue collection

Because Nginx and lighttpd documents are really rare, not to mention the Chinese documentation, so collect some and 502 related errors here, keep the source address, suggest to look at the source address of the content.

502 is a problem with fastcgi, so start with the fastcgi configuration.

1. Please check if your fastcgi process is started

2.FastCGI Process Not enough to use
Please Netstat-anpo by executing | grep "php-cgi" | Wc-l judge whether it is close to the fastcgi process you started, close to your settings, indicating that the process is not enough

Source: http://blog.s135.com/post/361.htm

3. Execution Timeout
Please put
Fastcgi_connect_timeout 300;
Fastcgi_send_timeout 300;
Fastcgi_read_timeout 300;
The value of these items is increased

Source: http://blog.s135.com/post/361.htm

4.FastCGI buffering is not enough
Nginx, like Apache, has a front-end buffering limit
Please put
Fastcgi_buffer_size 32k;
Fastcgi_buffers 8 32k;
The value of these items is increased

Source: Http://www.hiadmin.com/nginx-502-gateway-error A Case/

5.Proxy buffering is not enough
If you use the proxying, please
Proxy_buffer_size 16k;
Proxy_buffers 4 16k;
The value of these items is increased

Source: http://www.ruby-forum.com/topic/169040

6.https Forwarding Configuration error
The correct configuration method
server_name www.mydomain.com;

Location/myproj/repos {

Set $fixed _destination $http _destination;
if ($http _destination ~* ^https (. *) $)
{
Set $fixed _destination http$1;
}

Proxy_set_header Host $host;
Proxy_set_header X-real-ip $remote _addr;
Proxy_set_header Destination $fixed _destination;
Proxy_pass http://subversion_hosts;
}

Source: http://www.ruby-forum.com/topic/173455

7.php script Execution time is too long
Change the 0s of php-fpm.conf <value name= "Request_terminate_timeout" >0s</value> to a time

Source: http://blog.nowans.com/content/log.2008.10.29.229.html

/////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////////

Nginx PHP-FPM fast-cgi 502 Bad Gateway error is fastcgi problem, causing Nginx 502 error more likely.
Make a list of some of the problems and troubleshooting methods found on the web that are related to 502 bad gateway errors, starting with the FASTCGI configuration:

Whether the 1.FastCGI process has started

2.FastCGI worker process number is not enough
See how many php-cgi processes are open on the server by command
Ps-fe |grep "PHP" | Grep-v "grep" | Wc-l
Use the following command to see how many php-cgi processes are already being used to process a TCP request
Netstat-anop | grep "PHP" | Grep-v "grep" | Wc-l
Close to the number set in the configuration file, indicating too few worker processes
See also:http://blog.s135.com/post/361.htm

3.FastCGI execution time is too long
Increase the following parameter values according to the actual situation
Fastcgi_connect_timeout 300;
Fastcgi_send_timeout 300;
Fastcgi_read_timeout 300;

4.FastCGI Buffer Not Enough
Nginx, like Apache, has a front-end buffering limit and can adjust the buffering parameters
Fastcgi_buffer_size 32k;
Fastcgi_buffers 8 32k;
See also:http://www.hiadmin.com/nginx-502-gateway-error a case/

5.Proxy Buffer Not Enough
If you use a proxying, adjust
Proxy_buffer_size 16k;
Proxy_buffers 4 16k;
See also:http://www.ruby-forum.com/topic/169040

6.https Forwarding Configuration Error
The correct configuration method

server_name www.mydomain.com;
Location/myproj/repos {
Set $fixed _destination $http _destination;
if ($http _destination ~* ^https (. *) $)
{
Set $fixed _destination http$1;
}
Proxy_set_header Host $host;
Proxy_set_header X-real-ip $remote _addr;
Proxy_set_header Destination $fixed _destination;
Proxy_pass http://subversion_hosts;
}
See also:http://www.ruby-forum.com/topic/169040
http://bizhi.knowsky.com/
7. View Php-fpm.log
Error please see:http://www.admin99.net/read.php/396.htm
Http://hi.baidu.com/dugu2008/blog/item/0d9e9bf8e8c13b08d8f9fd14.html

Bug in 8.php
Please see:http://bugs.php.net/bug.php?id=41593

Configuration of the 9.php-fpm.conf
Please refer to http://www.jefflei.com/post/232.html
Http://www.php-oa.com/2008/06/05/php-fpm.html

10.nginx.conf's Rewrite-url rules, etc.

11.php-fpm.pid
My humble/php/sbin/php-fpm inside
Php_fpm_pid=/data1/php/logs/php-fpm.pid modified into
#php_fpm_PID =/data1/php/logs/php-fpm.pid
It is assumed that the user rights issue has been php-fpm.conf in the user started the service estimate problem will disappear

In addition, the reason and solution of the error request of Nginx
In nginx.conf, both Client_header_buffer_size and large_client_header_buffers are adjusted to alleviate this problem.
The main configuration is client_header_buffer_size this, the default is 1k, so the header is less than 1k will not be the problem.
As I now configure is:
Client_header_buffer_size 16k;
Large_client_header_buffers 4 64k;
This configuration can receive headers below 16k, the maximum number of bytes in the browser cookie is very large, so it is not good to use the largest value.
This article is reproduced from the "North Drifting Stone Blog"
http://www.niutian365.com/blog/

/////////////////////////////////////////////////////////////////////////////////////////////////////////////// ////////////////////////////////////////////////////

First, NGINX 502 error Troubleshooting
Nginx 502 Bad Gateway error is a fastcgi problem, resulting in Nginx 502 error more likely. Make a list of some of the problems and troubleshooting methods found on the web that are related to 502 bad gateway errors, starting with the FASTCGI configuration:
whether the 1.FastCGI process has started
2.FastCGI worker process number is not enough
Run Netstat-anpo | grep "php-cgi" | Wc-l determine if close to the fastcgi process, close to the value set in the configuration file, indicating that the number of worker processes is set too low
See also: http://blog.s135.com/post/361.htm
3.FastCGI execution time is too long
Increase the following parameter values according to the actual situation
Fastcgi_connect_timeout 300;
Fastcgi_send_timeout 300;
Fastcgi_read_timeout 300;
4.FastCGI Buffer Not enough
Nginx, like Apache, has a front-end buffering limit and can adjust the buffering parameters
Fastcgi_buffer_size 32k;
Fastcgi_buffers 8 32k;
See also: Http://www.hiadmin.com/nginx-502-gateway-error a case/
5.Proxy Buffer Not enough
If you use a proxying, adjust
Proxy_buffer_size 16k;
Proxy_buffers 4 16k;
See also: http://www.ruby-forum.com/topic/169040
6.https Forwarding Configuration error
The correct configuration method
server_name www.mydomain.com;
Location/myproj/repos {
Set $fixed _destination $http _destination;
if ($http _destination ~* ^https (. *) $)
{
Set $fixed _destination http$1;
}
Proxy_set_header Host $host;
Proxy_set_header X-real-ip $remote _addr;
Proxy_set_header Destination $fixed _destination;
Proxy_pass http://subversion_hosts;
}
See also: http://www.ruby-forum.com/topic/169040
Of course, it depends on what type of fastcgi you use, I have used php-fpm, the flow is about a single machine 400,000 PV (Dynamic page), and now basically did not touch 502.

7.php script Execution time is too long
Change the 0s of php-fpm.conf <value name= "Request_terminate_timeout" >0s</value> to a time
Source: http://blog.nowans.com/content/log.2008.10.29.229.html

Second, Nginx 413 error Troubleshooting: Modify the Upload file size limit
In the upload, Nginx returned 413 error, view the log file, the error message displayed is: "413 Request Entity Too Large", so on the Internet to find the next "Nginx 413 error" found to do the following settings:
In nginx.conf increase the client_max_body_size of the relevant settings, this value by default is 1m, can be increased to 8m to increase the increase in file size limit;
If you are running PHP, then also check the php.ini, this size client_max_body_size to and php.ini in the following values of the maximum value of the same or slightly larger, so that the submission of data is not inconsistent with the size of the error.
Post_max_size = 8M
Upload_max_filesize = 2M

Third, Nginx 400 error Troubleshooting: HTTP header/cookie too large
Today, someone reported Nginx HTTP400 error, and this HTTP400 error does not occur every time, check the discovery nginx400 error is due to the request header is too large, usually because the cookie is written in a longer string caused by.
The workaround is not to record too much data in the cookie, and to consider adjusting the client_header_buffer_size (default 1k) in nginx.conf if necessary.
If the cookie is too large, you may also need to adjust the Large_client_header_buffers (default 4k), which is described below:
The request line will report an HTTP 414 error if it exceeds buffer (URI Too Long)
Nginx accepts the longest HTTP header size must be larger than one buffer, or it will report a 400 HTTP error (bad Request).

/////////////////////////////////////////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////

The implication of Nginx 502 Bad Gateway is that the requested php-cgi has been executed, but the php-cgi process is terminated for some reason (generally, the problem of reading the resource) is not completed.
The implication of Nginx 504 Gateway time-out is that the requested gateway is not requested, simply that there is no request to the php-cgi that can be executed.

To solve these two problems is actually need to think comprehensively, in general Nginx 502 Bad Gateway and php-fpm.conf settings, and Nginx 504 Gateway time-out is related to the nginx.conf settings.
The correct setting takes into account multiple factors such as the performance of the server itself and the number of visitors.
With my current server as an example CPU is Ben four 1.5G, Memory 1gb,centos system, the visitor is about 50 people at the same time online.
But the people on the line need to request php-cgi to do a lot of processing, so I set the nginx.conf to:
Fastcgi_connect_timeout 300s;
Fastcgi_send_timeout 300s;
Fastcgi_read_timeout 300s;
Fastcgi_buffer_size 128k;
Fastcgi_buffers 8 128k; #8 128
Fastcgi_busy_buffers_size 256k;
Fastcgi_temp_file_write_size 256k;
Fastcgi_intercept_errors on;
The most important setting here is the first three, which is
Fastcgi_connect_timeout 300s;
Fastcgi_send_timeout 300s;
Fastcgi_read_timeout 300s;
This specifies the php-cgi connection, send and read time, 300 seconds is sufficient, so my server rarely appears 504 Gateway time-out this error. The most critical is the php-fpm.conf setting, which will lead directly to 502 bad Gateway and 504 Gateway time-out.
Let's take a closer look at some important parameters of php-fpm.conf:
Php-fpm.conf has two key parameters, one is "Max_children" and the other is "Request_terminate_timeout"
The value of my two settings is "40″, one is" 900″, but this value is not generic, but it needs to be calculated by itself.
The method is calculated as follows:
If your server performance is good enough, and the broadband resources are sufficient, PHP scripts do not have loops or bugs, you can set the "request_terminate_timeout" directly to 0s. The meaning of 0s is to let php-cgi go on without time limit. And if you can't do this, that means your php-cgi may have a bug, or your broadband is not enough or other causes your php-cgi to be able to feign death then it is recommended that you assign a value to "Request_terminate_timeout". This value can be set based on the performance of your server. Generally, the better the performance, the higher you can set, 20 minutes-30 minutes. Because my server PHP script needs to run for a long time, some may be more than 10 minutes so I set 900 seconds, so that does not cause php-cgi dead and 502 bad gateway this error.

And how does the value of "Max_children" be calculated? This value is in principle the bigger the better, the php-cgi process is much faster, the queue requests will be very small. Setting "Max_children" also needs to be based on the performance of the server, in general, a server normally consumes about 20M of memory per php-cgi, so my "Max_children" I set to 40, 20m*40= 800M means that at peak time all php-cgi are within 800M, below my valid memory 1Gb. And if my "Max_children" set the smaller, such as 5-10, then php-cgi will be "very tired", processing speed is also very slow, waiting for a longer time. If the request has not been processed for a long time, the 504 Gateway time-out this error, while the very tired of the php-cgi are dealing with the problem will appear 502 Bad Gateway this error.

/////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////////////

Nginx configuration PHP fastcgi Group to solve the inexplicable 502 Bad Gateway error

Common Nginx with PHP in this way:

Location ~ \.php$ {
Proxy_pass http://localhost:9000;
Fastcgi_param Script_filename/data/_hongdou$fastcgi_script_name;
Include Fastcgi_params;
}

This method can only be connected to a set of spawn-fcgi open fastcgi,502 Bad gateway errors often occur when the server load is slightly higher.

At first suspected that this is the process of php-cgi too little, the increase is still reflected often wrong, occasionally found that php-cgi will report such errors:

Zend_mm_heap corrupted

It appears that php-cgi has a problem executing some code, so that the thread aborts.

On the server, you may also see that the php-cgi process is getting less, and it is estimated that the php-cgi process is faulty.Automatically exits.

PHP's problem is not always easy toSolve, so in the nginx to think of ways, Nginx's advantage is that it can always burst out some strange way out.

In the Nginx proxy, the way to avoid the inexplicable error is only proxy to a upstream server group, and then configure Proxy_next_upstream, let Nginx encounter some kind of error code, automatically jump to the next backend. In this way, the application server is unstable, but after nginx it becomes a stable service. Think of Nginx fastcgi and Proxy is a thing, so proxy can use the experience, transplanted to FastCGI can also run up.

According to this idea, using spawn-fcgi to open the same set of PHP process, the difference is only the port:

Spawn-fcgi-a 127.0.0.1-p 9000-u nobody-f php-cgi-c 100
Spawn-fcgi-a 127.0.0.1-p 9001-u nobody-f php-cgi-c 100

Then change this configuration of the fastcgi to use the upstream way:

Upstream Backend {
Server 127.0.0.1:9000;
Server 127.0.0.1:9001;
}

Location ~ \.php$ {
Fastcgi_pass backend;
Fastcgi_param Script_filename/data/_hongdou$fastcgi_script_name;
Include Fastcgi_params;
}

Check the configuration result is correct, can run up, at the same time on the server Netstat-n|grep 9000 and grep 9001 have records, prove the connection is correct; Check the page at the front desk, everything is working.

This configuration is the simplest configuration, since can be connected to the upstream, then it is obvious that some things upstream can be used, such as Ip_hash, weight, max_fails and so on.

Such configuration in a single machine can not be shared session, no test, if there is a problem, you can add Ip_hash, or configure PHP to save the session into the memcached.

Then is the configuration of Fastcgi_next_upstream, Nginx wiki does not introduce this configuration, checked, in the Nginx changes mentioned, and the birth date is the same as Proxy_next_upstream. That being the case, it should be the same as Proxy_next_upstream. In general, the default value of error timeout can work, because PHP 502 error exception is the return of 500 error, so I set the Fastcgi_next_upstream as:

Fastcgi_next_upstream error timeout Invalid_header http_500;

With this configuration, you can basically eliminate any frequent 500 errors, the probability of the problem will be much smaller, if the customer is still intense, then add a few more groups of fastcgi process.

The above configuration can eliminate the "inexplicable" 502 errors caused by PHP, and make nginx with PHP more powerful than before. If Nginx still returns 502 error, then this time must be the server hangs off or other serious problems.

Nginx 502 Bad Gateway error issue collection