Analysis of "High performance server" Nginx

Introduction

Nginx is a popular high-performance server that officially claims to support 50,000 concurrent connections under stress tests and consumes very little memory. In contrast to other expensive hardware load balancing solutions, Nginx is open source free and can greatly reduce costs. This article will analyze its internal structure from a few aspects.

2. Characteristics
4. Process Model
   1. Surprise group effect
   2. Load Balancing
6. Core modules
   1. Module classification
   2. Event-driven module mechanism
   3. Reverse proxy Module
8. Configuration file

Features of Nginx

Nginx is a high-performance Web server developed by Russian engineers, in order to achieve efficient nginx all in C language, because the bottom of the different operating systems are encapsulated, so nginx implementation of platform-independent. As we all know, nginx performance is high, and high performance is inseparable from its architecture. Nginx handles connection requests in a multi-worker process, and binds processes and specific CPUs with CPU and process affinity, eliminating the overhead of process context switching while implementing concurrency. At the same time, all IO operations, whether network IO or disk Io,nginx, are unified using the Linux kernel's epoll mechanism to implement asynchrony with callback functions. This allows nginx to easily achieve the level of concurrency, compared to other major web servers, such as Apache thousand other concurrency has made great progress. In addition to the above two characteristics, nginx interior is visible in a large number of detail optimization is the cornerstone of nginx efficient.

?

Process Model

Nginx after booting, in the UNIX system will be daemon in the background, the background process contains a master process and multiple worker processes. We can also manually turn off the background mode, let Nginx run in the foreground, and through the configuration to let Nginx cancel the master process, so that nginx can be run in a single process mode.

Nginx after booting, there will be a master process and multiple worker processes. The master process is primarily used to manage worker processes, including receiving signals from the outside, sending signals to the worker processes, monitoring the health status of the worker process, and automatically restarting the new worker process when the worker process exits (in exceptional cases). The basic network events are handled in the worker process. Multiple worker processes are equivalent, they compete for requests from clients equally, and each process is independent of each other. A request can only be processed in a worker process, and a worker process cannot handle requests from other processes. The number of worker processes can be set, generally we will set the number of CPU cores consistent with the machine, as we have said before, the recommended number of workers is the number of CPU cores, it is easy to understand here, more worker number, will only cause the process to compete for CPU resources, This can lead to unnecessary context switching. Furthermore, Nginx provides a binding option for CPU affinity in order to make better use of multicore features, and we can bind a process to a kernel so that it does not invalidate the cache because of the process switching. Like this small optimization in nginx is very common, for example, Nginx in the 4-byte string comparison, the 4 characters will be converted to an int, and then compared to reduce the number of CPU instructions and so on. Nginx process model, can be the origin of the expression:

Figure 3-1

After Nginx starts, the master process receives signals from the outside world, and then does different things according to the signal. So we have to control nginx, just need to send a signal through kill to the master process. For example Kill-hup Pid, is to tell Nginx, need to restart Nginx. What does the master process do after it receives the kill signal? First, after the master process receives the signal, it reloads the configuration file, starts the new worker process, and sends a signal to all old worker processes that they can retire honorably. After the new worker starts, it begins to receive the new request, and the old worker receives a signal from master, ceases to receive the new request, and exits after processing of all outstanding requests in the current process is complete.

How does the worker process handle the request? As we mentioned earlier, the worker process is equal, and every process, the opportunity to process the request is the same. When we provide a 80 port HTTP service, a connection request comes in, each process has the possibility to handle this connection, how to do it? First, each worker process is forked from the master process, and in the master process, the socket (LISTENFD) that needs to be listen is first established, and then the multiple worker processes are forked. The LISTENFD of all worker processes becomes readable when a new connection arrives, and to ensure that only one process processes the connection, all worker processes seize Accept_mutex before registering the LISTENFD read event, and the process that grabs the mutex registers the LISTENFD read event. The accept accepts the connection in the Read event. When a worker process begins to read the request, parse the request, process the request, generate the data, and then return it to the client after the connection is accept, this is the complete request. As we can see, a request is handled entirely by the worker process and is handled only in a worker process.

What are the benefits of Nginx using this process model? Of course, the benefits will certainly be many. First, for each worker process, the independent process does not need to be locked, so it saves the overhead of the lock, and is much more convenient when programming and problem finding. Second, the use of independent processes, can make no impact on each other, after a process exits, the other process is still working, the service will not be interrupted.

Surprise group effect

What is the surprise group effect, the Nginx process model is concerned. Nginx uses a multi-worker process to process connections, a request can only be owned by a single process, so in addition to the first to get the request and Tomcat and other general server, there is no dedicated work unit for processing the connection. Each process is a separate unit of work. If there is no connection request to the server at this time, all worker processes are in hibernation, but once a request arrives, all dormant worker processes are awakened, but other processes are returned to hibernation in addition to the process in which the connection was established. Such overhead can be avoided, and in some cases, the process is frequently awakened by large areas, and hibernation consumes a lot of resources. Nginx practice is that at some point there is only one worker process listening port, the specific practice is to set a lock. When the process is idle to attempt to listen to the port, it tries to acquire the lock first, and if it does not get to the lock then the process continues to work. If the lock is acquired, the listening port is responsible for processing the new connection request. Here's a question: When to release the lock? Wait until the process finishes processing all the events? This can lead to events that are too long, so how can you reduce the time? Nginx does this by setting up two queues, one for new connection events established after acquiring the lock, and another for ordinary events. When a process finishes processing a new connection event, it releases the lock, reducing the lock's occupancy event.

Load Balancing

How does nginx implement load balancing between multiple worker processes? Nginx uses a simple algorithm, which is roughly described as follows: Each worker process will have a ngx_accept_disabled shaping variable at initialization time. The initial value of this variable is 7/8 of the size of each process's own connection pool, and the symbol is negative. Each process establishes a connection and adds 1 to this value. Until this value is positive, the current process is not processing a new connection, and if there is a lock, release the lock. At this point the chances of other processes getting locks become larger. Each time the connection is completed, the value is reduced by 1 until he returns to the initial value, and the process begins to attempt to acquire the lock. This strategy is relatively simple, to a certain extent, to achieve the balance of responsibility.

Nginx's Core module

Nginx will be organized into a chain of functional modules, when the request arrives, the request in turn through the chain of some or all of the modules, processing. Each module implements a specific function. For example, the implementation of the request decompression module, the implementation of SSI's module, the realization of communication with the upstream server module, to achieve communication with the FASTCGI service module.

There are two modules that are special, they live in the middle of nginx core and each function module. These two modules are the HTTP module and the Mail module. These 2 modules implement an additional layer of abstraction over the Nginx core, handling events related to the HTTP protocol and the E-mail Protocol (SMTP/POP3/IMAP), and ensuring that these events can be called in the correct order for other functional modules. Currently the HTTP protocol is implemented in the HTTP module, but it is possible to be stripped into a separate module in the future to extend the Nginx support Spdy protocol.

Classification of modules

Nginx module According to its function can basically be divided into the following types:

1. Event Module: Constructs a framework for operating system-independent event handling and provides the handling of specific events. including Ngx_events_module, Ngx_event_core_module and Ngx_epoll_module. Nginx specifically uses what event processing module, which depends on the specific operating system and compilation options.
2. Phase Handler: This type of module is also directly referred to as the Handler module. Mainly responsible for processing client requests and generating the content to be responded to, such as the Ngx_http_static_module module, responsible for the client's static page request processing and prepare the corresponding disk file for the response content output.
3. Output filter: Also known as the filter module, is mainly responsible for the output of the content of processing, can be modified. For example, you can implement work that adds a predefined footbar to all HTML pages in the output, or replaces the URL of an exported picture.
4. The Upstream:upstream module implements the function of the reverse proxy, forwards the real request to the back-end server, and reads the response from the back-end server and sends it back to the client. The upstream module is a special kind of handler, except that the response content is not actually generated by itself, but is read from the backend server.

Load-balancer: Load balancer module, implement a specific algorithm, in a large number of back-end servers, select a server to come out as a forwarding server for a request.

Event Module Core Epoll

Nginx How to do hundreds of thousands of of concurrent connections, the answer is Epoll event-driven mechanism. Suppose there are 1 million users with a process that keeps a TCP connection, although the number of connections is huge, but only a small percentage of connections are active at some point. So, the process only needs to deal with the small portions of these 1 million connections enough. To achieve this goal, how to do is the most efficient. The process to find a small active part in 1 million connections at some point, is to give all the connections to the operating system, so that the operating system is responsible for finding it? This is done by the Select or poll event-driven approach before the Linux kernel 2.4 release. The disadvantage of this is that if each collection of events passes all the connections to the operating system, which can result in a large replication of the user-state-to-kernel memory, and the discovery process is a huge waste of resources, select and poll take this approach, causing them to handle only thousands of concurrent connections.

Figure 4-1

Let's take a look at Epoll's approach, epoll in memory to apply for a red-black tree for storing all the events. A doubly linked list is also requested to hold active events. All events added to the red-black tree will have a callback relationship with the device (primarily the NIC) driver, and when an event occurs, the callback function puts the event into a doubly linked list. Compared to select and poll just notifies the user state that an event has occurred, but exactly which events occurred to him regardless, must be by the user to traverse the judgment of the practice, epoll because there is a two-way list of active events, can greatly improve efficiency. So when something happens, just copy the list into the user's memory. And the use of red-black tree storage events facilitates the discovery and deletion of events. In Nginx, Epoll is not only used for monitoring the network IO device, but also for listening for file Io, which improves the efficiency of nginx file reading and writing. Is the performance comparison of the Apache server with the Select IO Model and the Nginx server using the Epoll model. It can be seen that in the concurrency of more than 2500 Apache server efficiency began to decline significantly, and nginx server efficiency decreased smoothly.

Figure 4-2

Handling Expired Events

There is an event expiration problem in the Epoll model, assuming that call Epoll_wait returns 3 events at a time, when the process processes the first event, the third event may expire, such as when the network connection is actively closed by the client, and the connection to the third event is returned to the connection pool. So how to flag this event has expired, one way is to set the connection descriptor-to indicate that the connection corresponding to the event has expired. But in some extreme situations this can happen, such as when the process finishes processing the first event and starts processing the second event, which happens to be a connection to the client, then the connection from the connection pool is likely to be the connection that was just returned from the third event. At this point the second event establishes a connection, and the descriptor is no longer 1, and the process does not recognize an event that has expired when it processes a third event. Nginx's approach is to use a separate flag bit processing, each time the connection is removed from the connection pool when the flag is reversed. So before each processing to see the current event of the flag bit and before the consistency, inconsistent is the expiration event.

Reverse proxy Module

The data forwarding function provides Nginx with horizontal processing capability spanning single machine, so that nginx can not only provide a single function limit for the end node, but it has the function of splitting, encapsulating and consolidating the network application level. At the same time, the level and loose coupling provided by Nginx configuration system make the system expansibility more high. A typical reverse proxy:

Figure 4-3

When the client initiates a request service to an IP address, Nginx is responsible for establishing the connection as a high-load portal Server, because the server has a limited storage size, so the specific request content is not stored locally, but the content is stored in the upstream Web server. Nginx communicates with third-party servers in two fully asynchronous ways, upstream and subrequest. Both of these methods ensure that there is no blocking at the time of communication, so that the efficiency of the server is maximized. Surequest from his own and upstream no difference, his bottom-level implementation is based on upstream. The difference between the two is that the purpose of the design is different, the main function of upstream is to forward the request to the upstream server, known as the transmission. Nginx gets the content and does not go through too much processing to pass the content to the client. Subrequst refers to the establishment of the connection, Nginx needs and upstream server to make multiple upstream communication, the content obtained is mostly not the complete content, but some data. Nginx will splice the data and pass it back to the client.

Load Balancing algorithm

The algorithm for load balancing supported by Nginx Upstream module:

1. The default is polling, each request is assigned to a different back-end server in chronological order, and can be automatically rejected if the backend server goes down.
2. Weight Poll Equalization: You can specify the polling probability, the weight (Weight) and the access ratio are proportional to the performance of the backend server.
3. Each request is allocated according to the Hash result of the access Ip, so that each visitor has fixed access to a back-end server, and in some applications, all requests from the same client are assigned to the same server, such as the local database where the server stores the client registration, shopping and other service request information. Assigning a client's child requests to the same server for processing is critical.
4. The response time of the back-end server is allocated to the request, and the response time is short of priority allocation.
5. Assign requests by the Hash result of the access URL so that each URL is directed to the same back-end server, which is more efficient when the backend server is cached. In the nginx.conf configuration file, define a group with the Upstream directive

(Take 4 servers for example) load-balanced backend server pool:

Upstream Servername {

Server 192.168.1.10:80 weight=1 max_fails=3 fail_timeout=60s;

Server 192.168.1.11:80 weight=1 max_fails=3 fail_timeout=60s;

Server 192.168.1.12:80 weight=1 max_fails=3 fail_timeout=60s;

Server 192.168.1.13:80 weight=1 max_fails=3 fail_timeout=60s;

}

Where Servername is the server group name; Weight: Sets the server's weight, the default value is 1, the greater the weight value, the greater the probability that the server will be accessed; Max_fails and Fail_timeout: The two are related if a server is in Fail_ Timeout time has occurred in the Max_fails connection failure, then Nginx will think that the server has been down, so that in fail_timeout time no longer to query it.

?

Configuration file

Nginx configuration system consists of a master configuration file and some other auxiliary configuration files. These configuration files are plain text files, all in the Conf directory under the Nginx installation directory. The configuration information in nginx.conf, according to its logical meaning, classifies them, that is, it is divided into multiple scopes, or is called the configuration Directive context. The different scopes contain one or more configuration items. Several instruction contexts supported by the current Nginx:

1. Main:nginx some parameters that are not related to specific business functions (such as HTTP services or email service proxies) at runtime, such as the number of worker processes, the identity of the run, and so on.
2. http: Some configuration parameters related to providing HTTP services. For example: whether to use keepalive, whether to use gzip compression, and so on.
3. Several virtual hosts are supported on the Server:http service. Each virtual host has a corresponding server configuration item, and the configuration item contains the virtual host-related configuration. When you provide a proxy for the mail service, you can also establish several servers. Each server is differentiated by the address of the listener.
4. Location:???? A series of configuration items that correspond to some specific URLs in the HTTP service.
5. Mail: When implementing an email-related SMTP/IMAP/POP3 agent, some of the shared configuration items (because multiple proxies may be implemented, work on multiple listening addresses).

The context of the instruction, which may be contained in the condition that appears. For example, the HTTP context and the mail context must always appear in the main context. In one context, it may contain a different type of context multiple times. For example, if the HTTP service supports multiple virtual hosts, multiple server contexts appear in the HTTP context.

Summarize

Nginx is a more complex server, this article simply introduces some of the important components and their operating principles, for other internal components, such as the log module, mail agent module, etc. is not involved. In this paper, the typical function of nginx and its implementation mechanism are analyzed, which is of reference significance for the server selection in the construction of large-scale system.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Analysis of "High performance server" Nginx