Nginx load balancing and reverse proxy-"the core technology of billion-level traffic website Architecture"

When our application single instance does not support the user request, this time needs to expand, from one server to two, dozens of, hundreds of units.
However, when the user accesses is accessed through such as http://www.XX.com, when the request, the browser will first query the DNS server to obtain the corresponding IP,
The corresponding service is then accessed through this IP.
Thus, one way is to map multiple IPs to the www.XX.com domain name, but there is one simplest problem, assuming that a server restarts or fails,
DNS has a certain cache time, a long switchover after a failure, and no mechanism for heartbeat checking and failure retry of the backend service.
Therefore, the extranet DNS should be used to implement traffic scheduling with GSLB (Global load Balancing),
If you assign a user to a server closest to him to elevate the experience.
And when there is a problem in the engine room of an area (such as when the optical cable is cut off), the service can be made available by DNS pointing to the IP in the other zone.

can use "DNS query" in Webmaster's home, query c.3.cn can see results similar to the following.
That is, different operators return the public network IP is not the same.

For intranet DNS, simple polling load balancing can be achieved.
However, in that sentence, there is a certain cache time and there is no failure retry mechanism.
Therefore, we can consider choosing such as Haproxy and Nginx.
And for the general application, there is nginx can be. However, Nginx is generally used for seven-layer load balancing, and its throughput is limited.
In order to improve overall throughput, the access layer is introduced between DNS and Nginx,
such as the use of LVS (software load balancer), F5 (hard load balancer) can do four levels of load balancing,
That is, DNS resolves to lvs/f5 first, then lvs/f5 to Nginx, and then to backend real Server by Nginx.

For general business developers, we only need to care about the nginx level is enough, lvs/f5 generally by the system/operations engineer to maintain.
Nginx currently offers HTTP (Ngx_http_upstream_module) seven-layer load balancing,
The 1.9.0 version also began to support TCP (Ngx_stream_upstream_module) four-tier load balancing.


A few more concepts are clarified here.
Two-tier load balancing is the MAC address of the upstream server by overwriting the destination MAC address of the message, the source IP address and destination IP address are unchanged,
The Load Balancer server and the real server share the same VIP, such as LVSDR working mode.
A four-tiered load balancer forwards messages to the upstream server (different IP addresses + ports) based on the port, such as Lvsnat mode, HaProxy,
Seven-tier load balancing is based on the port number and application layer protocol such as the HTTP protocol hostname, URL, forwarding messages to the upstream server (different IP address + port), such as Haproxy, Nginx.


Here again to introduce the LVS Dr work mode, its work in the data link layer, LVS and upstream servers share the same VIP, by overwriting the destination MAC address of the message for the upstream server MAC address to achieve load balancing, the upstream server directly respond to the message to the client, without the LVS, thereby improving performance. However, since LVS and upstream servers must be on the same subnet, in order to solve cross-subnet problems without impacting load performance, you can choose to hang haproxy behind LVS to address cross-network and performance issues through a four-to seven-tier load Balancer haproxy cluster.
The two "semi-finished" things complement each other, and together they become a "full" load balancer. Now the Nginx stream also supports TCP, so Nginx is also a four-to seven-layer load balancer, the general scenario can be replaced with Nginx haproxy.


Explain a few terms. Access layer, reverse proxy server, load balancer server,
In this article, if there is no special description, it refers to Nginx. Upstream server is the upstream
Refers to the server that the Nginx load balances to the processing business, also can be called real server, namely the real processing Business Server.


Here are a few things we should be concerned about for load balancing.

Upstream server configuration: Use upstream server to configure the upstream server.
Load Balancing algorithm: Load balancing mechanism when configuring multiple upstream servers.
Failure retry mechanism: Configures whether additional upstream servers need to be retried when timeouts or when the upstream server does not survive.
Server heartbeat check: Health Check/heartbeat check of the upstream server.


The load balancing provided by Nginx can achieve load balancing, failover, failure retry, fault tolerance, health check, etc. of the upstream server.
When there are problems with some upstream servers, you can move requests to other upstream servers to ensure high availability and to achieve more intelligent load balancing through openresty.
such as the hot and non-hot spot flow separation, normal flow and reptile flow separation.
Nginx Load balancer itself is also a reverse proxy server, the user request through Nginx Proxy to an upstream server in the intranet processing,
The reverse proxy server can cache, compress, and process the response results to improve performance.

2.1 Upstream Configuration

The first step is to configure an upstream server for Nginx, the server that is load balanced to the real processing business, by configuring upstream under the HTTP directive.

Upstream Backend {

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2;

}

Upstream server main configuration.

IP address and port: Configures the IP address and port of the upstream server.

Weight: Weight is used to configure weights, the default is 1, the higher the weight, the more requests that are allocated to this server
(as configured on one request per three requests forwarded to 9080, the remaining two requests are forwarded to 9090),
You need to set weights based on the actual processing power of the server (for example, physical servers and virtual machines require different weights).


We can then configure the following proxy_pass to handle user requests.

Location/{

Proxy_pass Http://backend;

}

When Nginx is accessed, the request is reversed to the backend configuration of the upstream Server. Next we look at the load balancing algorithm.



2.2 Load Balancing algorithm
Load balancing is used to resolve how to select upstream server for processing when a user request arrives, by default Round-robin (polling), and several other algorithms are supported.
Round-robin: Polling, the default load-balancing algorithm, which forwards requests to the upstream server in a polling manner, enables weight-based polling by cooperating with the weight configuration.
Ip_hash: Load Balancing According to customer IP, that is, the same IP will load balanced to the same upstream Server.

Upstream Backend {

Ip_hash;

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2;

}

hash key [consistent]: hashes a key or uses a consistent hash algorithm for load balancing. The problem with the hash algorithm is that when a server is added/removed, it causes many keys to be re-loaded to different servers (which may cause problems at the backend), so it is recommended that you consider using a consistent hashing algorithm so that when a server is added/removed, Only a handful of keys will be re-loaded to different servers.

Hashing algorithm: Here is the load balancing based on the request URI, you can use Nginx variable, so you can implement a complex algorithm.
Upstream Backend {

Hash $uri;

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2;

}

Consistent hash algorithm: Consistent_key dynamically specified.
Upstream Nginx_local_server {

Hash $consistent _key consistent;

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2;

}

The following location specifies a consistent hash key, where the parameter cat (class) is given priority, and if not, load-balanced based on the request URI.
Location/{

Set $consistent _key $arg _cat;

if ($consistent _key = "") {

Set $consistent _key $request _uri;

}

}

We actually set up a consistent hash key through LUA.

Set_by_lua_file $consistent _key "Lua_balancing.lua";
Lua_balancing.lua code.

Local Consistent_key = Args.cat

If not consistent_key or Consistent_key = = "Then

Consistent_key = Ngx_var.request_uri

End



Local value = Balancing_cache:get (Consistent_key)

If not value then

Success,err = Balancing_cache:set (Consistent_key, 1, 60)

Else

Newval,err = BALANCING_CACHE:INCR (Consistent_key, 1)

End

If one of the categorical requests is too large, the upstream server may not be able to handle so many requests, at which point the consistent hash key can be followed by incrementing the count to implement a polling-like algorithm.
If newval >

Consistent_key = Consistent_key. ‘_‘ .. newval

End

Least_conn: Load balance the request to the least active connected upstream server. If fewer servers are configured, the weight-based polling algorithm is used instead.

The Nginx commercial version also provides Least_time, which is load balancing based on the minimum average response time.



2.3 Failed retry

There are two main configurations: Upstream server and Proxy_pass.

Upstream Backend {

Server 192.168.61.1:9080 max_fails=2 fail_timeout=10s weight=1;

Server 192.168.61.1:9090 max_fails=2 fail_timeout=10s weight=1;

}

By configuring the upstream server's max_fails and fail_timeout to specify each upstream server, when max_fails requests fail within fail_timeout time, the upstream server is considered unavailable/not viable and will then be removed from the upstream server, Fail_ After timeout time, the server is added again to the list of surviving upstream servers for retry.

location/test {

Proxy_connect_timeout 5s;

Proxy_read_timeout 5s;

Proxy_send_timeout 5s;



Proxy_next_upstreamerror timeout;

Proxy_next_upstream_timeout 10s;

Proxy_next_upstream_tries 2;



Proxy_pass Http://backend;

Add_header upstream_addr $upstream _addr;

}

}

The Proxy_next_upstream is then configured to retry the next upstream server when a configured error is encountered.

For detailed configuration, refer to the Nginx section in the agent layer timeout and retry mechanism.



2.4 Health Check

Nginx Health Check on the upstream server by default is a lazy policy, Nginx commercial version provides health_check for active health check. Of course, the Nginx_upstream_check_module (https://github.com/yaoweibin/nginx_upstream_check_module) module can also be integrated for active health checks.
Nginx_upstream_check_module supports the TCP heartbeat and HTTP heartbeat for health checks.


2.4.1 TCP Heartbeat Check

Upstream Backend {

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2;

Check interval=3000 rise=1 fall=3 timeout=2000 type=tcp;

}

This is configured to use TCP for heartbeat detection.

Interval: Detection interval, CONFIGURED here every 3s detection.
Fall: The upstream server is identified as not surviving after the number of failed detections.
Rise: After the detection succeeds, the upstream server is identified as surviving and can process the request.
Timeout: Detects the request time-out configuration.


2.4.2 http Heartbeat Check

Upstream Backend {
Server 192.168.61.1:9080 weight=1;
Server 192.168.61.1:9090 weight=2;
Check interval=3000 rise=1 fall=3 timeout=2000 type=http;
Check_http_send "Head/status http/1.0\r\n\r\n";
Check_http_expect_alive http_2xx http_3xx;

}

The HTTP heartbeat check has the following two additional configurations required.
Check_http_send: The HTTP request content that was sent when the check was made.
Check_http_expect_alive: When the upstream server returns a matching response status code, the upstream server is considered to be alive.
It is important to note that the check interval should not be too short, or it may be because the heartbeat check pack is too many to cause the upstream server to hang up, and to set a reasonable time-out.
This article uses openresty/1.11.2.1 (corresponding nginx-1.11.2), before installing Nginx need to hit Nginx_upstream_check_module patch (Check_1.9.2+.patch), To the Nginx directory, execute the following shell:
Patch-p0 </usr/servers/nginx_upstream_check_module-master/check_1.9.2+.patch.
If the patch is not installed, then the Nginx_upstream_check_module module is not working, it is recommended to use the Wireshark grab to see if it works.
2.5 Other configurations
2.5.1 Domain name upstream server

Upstream Backend {

Server c0.3.cn;

Server c1.3.cn;

}

Nginx Community Edition, is in the Nginx parsing configuration file phase to resolve the domain name to an IP address and recorded on the upstream, when the corresponding IP address of the two domain name changes, the upstream will not be updated. The Nginx Business Edition supports dynamic updates only.
However, Proxy_pass http://c0.3.cn is supported for dynamic domain name resolution.

2.5.2 backing up the upstream server

Upstream Backend {

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2 backup;

}

The 9090-port upstream server is configured to prepare the upstream server, and when all primary upstream servers are not alive, the request is forwarded to the standby upstream server.
If the compression of the upstream server for pressure measurement, to remove some of the upstream server for the pressure test, but in order to be safe to configure some of the upstream server, when the pressure measurement of the upstream server is hung off, traffic can be forwarded to the upstream server, so that the user request processing.

2.5.3 upstream server not available

Upstream Backend {

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2 down;

}

The 9090-port upstream server is configured to be permanently unavailable and temporarily removed from the machine when a test or machine fails.
2.6 Long connections

Configure the long connection between Nginx and the upstream server, the long connection between the client and Nginx can refer to the corresponding part of the location "timeout and retry".
Configure the number of long connections with the keepalive directive.
Upstream Backend {

Server 192.168.61.1:9080 weight=1;

Server 192.168.61.1:9090 weight=2 backup;

KeepAlive 100;

}
This directive configures the maximum number of idle connections that each worker process can cache with the upstream server. When this amount is exceeded, the least recently used connection will be closed. The keepalive directive does not limit the worker process's total connection to the upstream server.
If you want to establish a long connection with the upstream server, you must not forget the following configuration.

Location/{

#支持keep-alive

Proxy_http_version 1.1;

Proxy_set_header Connection "";

Proxy_pass Http://backend;

}

If it is http/1.0, you need to configure the send "Connection:keep-alive" request header.
Upstream server do not forget to turn on long connection support.
Next, let's take a look at how Nginx implements KeepAlive (Ngx_http_upstream_keepalive _module) to get some of the code at the time of the connection.
Ngx_http_upstream_get_keepalive_peer (ngx_peer_connection_t*pc, void *data) {

1. Start by asking which server (IP and port) the load balancer uses

RC =kp->original_get_peer (PC, Kp->data);
Cache =&kp->conf->cache;

2. Poll "Idle connection Pool"

for (q =ngx_queue_head (cache);

q!= Ngx_queue_sentinel (cache);

Q =ngx_queue_next (q))

{

Item = Ngx_queue_data (q,ngx_http_upstream_keepalive_cache_t, queue);

C =item->connection;

2.1. Use this connection if the connection IP and port of the free connection pool cache are the same as the IP and port to which the load is balanced

if (ngx_memn2cmp (U_char *) &item->sockaddr, (U_char *) pc->sockaddr,

Item->socklen,pc->socklen)

= = 0)

{

2.2. Remove this connection from "Free Connection pool" and press the "Release Connection pool" stack top

Ngx_queue_remove (q);

Ngx_queue_insert_head (&kp->conf->free, q);



Goto found;

}

}

3. If the free connection pool does not have a long connection available, a short connection is created

return NGX_OK;


Part of the code when the connection is released.

Ngx_http_upstream_free_keepalive_peer (ngx_peer_connection_t*pc, void *data, ngx_uint_t state) {

c = pc->connection;//The current connection to be freed

1. If the release connection pool is not ready to be released, you need to make a space from the free connection pool to use for the new connection (this can happen if the number of connections created exceeds the connection pool size)

if (Ngx_queue_empty (&kp->conf->free)) {

Q =ngx_queue_last (&kp->conf->cache);

Ngx_queue_remove (q);

item= Ngx_queue_data (q, ngx_http_upstream_keepalive_cache_t, queue);

Ngx_http_upstream_keepalive_close (item->connection);

} else {//2. Release a connection from "Release Connection Pool"

Q =ngx_queue_head (&kp->conf->free);

Ngx_queue_remove (q);

item= Ngx_queue_data (q, ngx_http_upstream_keepalive_cache_t, queue);

}

3. Pushes the current connection to the top of the free connection pool stack for next use

Ngx_queue_insert_head (&kp->conf->cache, q);

Item->connection = C;

The total number of connections is the total number of long connections for "free connection pool" + "Free Connection pool". First, the long connection configuration does not limit the total number of connections that the worker process can open (over-the-time as a short connection). In addition, the connection pool must be set according to the actual scene reasonable.

1. The idle connection pool is too small to connect enough to keep building connections.
2. The idle connection pool is too large to have too many idle connections to time out before it is used.
In addition, it is recommended to open long connections only to tabloid text.

2.7 HTTP Reverse Proxy example

In addition to load balancing, reverse proxies provide caching to reduce the pressure on the upstream server.
1. Global configuration (proxy cache)

Proxy_buffering on;

Proxy_buffer_size 4k;

Proxy_buffers 4k;

Proxy_busy_buffers_size 64k;

Proxy_temp_file_write_size 256k;

Proxy_cache_lock on;

Proxy_cache_lock_timeout 200ms;

Proxy_temp_path/tmpfs/proxy_temp;

Proxy_cache_path/tmpfs/proxy_cache Levels=1:2keys_zone =cache:512m Inact Ive=5m max_size=8g;

Proxy_connect_timeout 3s;

Proxy_read_timeout 5s;

Proxy_send_timeout 5s;

With the proxy buffer on, the cached content will be stored in the TMPFS (memory file system) to improve performance and set the time-out.
2. Location configuration
Location ~ ^/backend/(. *) $ {

#设置一致性哈希负载均衡key

Set_by_lua_file $consistent _key "/export/app/c.3.cn/lua/lua_ balancing_backend.properties";

#失败重试配置

Proxy_next_upstream Error timeout http_500 http_502 http_504;

Proxy_next_upstream_timeout 2s;

Proxy_next_upstream_tries 2;



#请求上游服务器使用GET方法 (no matter what method the request is)

Proxy_method GET;

#不给上游服务器传递请求体

Proxy_pass_request_body off;

#不给上游服务器传递请求头

Proxy_pass_request_headers off;

#设置上游服务器的哪些响应头不发送给客户端

Proxy_hide_header Vary;

#支持keep-alive

Proxy_http_version 1.1;

Proxy_set_header Connection "";

#给上游服务器传递Referer, cookies and host (on-demand delivery)

Proxy_set_header Referer $http _referer;

Proxy_set_header Cookie $http _cookie;

Proxy_set_header Host web.c.3.local;

Proxy_pass Http://backend/$1$is_args$args;

}

We have opened the Proxy_pass_request_body and Proxy_pass_request_headers, prohibit the delivery of the request header and content body to the upstream server, so that the upstream server is not subject to the request header attack, nor need to parse; You can use Proxy_set_header to deliver on demand.
We can also turn on gzip support with the following configuration to reduce the packet size of network transmissions.

gzip on;

Gzip_min_length 1k;

Gzip_buffers 16k;

Gzip_http_version 1.0;

Gzip_proxied any;

Gzip_comp_level 2;

Gzip_types text/plainapplication/x-javascript text/css Application/xml;

Gzip_vary on;

For content-based responses, it is recommended to turn on gzip compression, and the Gzip_comp_level compression level depends on the actual pressure measurement (bandwidth and throughput options).

Note: The content of the article is from the online data collation, for reference only.

Nginx load balancing and reverse proxy-"the core technology of billion-level traffic website Architecture"