Nginx achieves a high-performance and scalable design model

The overall architecture of NGINX is characterized by a group of processes working collaboratively:

Master process: executes privileged operations, such as reading configuration files, binding sockets, and creating/notifying sub-processes.
Worker process: receives and processes connection requests, reads and writes data to disks, and communicates with upstream servers. When NGINX is active, only the working process is busy.
Cache loader process: loads the high-speed cache of the disk to the memory. This process runs at startup and then exits.
Cache Manager Process: sorts the data cached on the disk to ensure that the data does not cross the border. This process runs intermittently.

The key to NGINX's high performance and scalability depends on two basic designs:

Limit the number of working processes as much as possible to reduce the overhead caused by context switching. By default, the recommended configuration is to allow each CPU core to correspond to a working process to efficiently use hardware resources.
The worker process uses a single thread and processes multiple concurrent connections in a non-blocking manner.

Each working process of NGINX processes multiple connection requests through the state machine. This state machine is implemented as a non-blocking working mode:

Each worker process processes several sockets, including listening sockets or connecting sockets.
When the listener socket receives a new request, a new connection socket is opened to process communication with the client.
When an event arrives at the connection socket, the working process quickly completes the response and processes the newly received event from any other socket.
Garret said NGINX chose such a design to fundamentally distinguish it from other Web servers. Generally, the Web server assigns each connection to an independent thread, which makes it easy to process multiple connections, because each connection can be considered as a linear sequence containing multiple steps, context switching overhead will be incurred. In fact, the working thread is in the blocking status most of the time, waiting for the client or other upstream servers. When the number of concurrent connections/threads trying to perform operations such as I/O exceeds a certain threshold, or the memory consumption is exhausted, the context switching cost is apparent.

On the other hand, NGINX is designed to prevent working processes from blocking network traffic, unless there is no work to do. In addition, each new connection consumes only a small amount of resources, including only one file descriptor and a small amount of worker process memory.

In general, after the system is optimized, every working process of NGINX can process hundreds of thousands of concurrent HTTP connections.



In-depth NGINX: how do we design its performance and scalability?

The reason why NGINX is so superior in performance is the design behind it. Many web servers and application servers use a simple threaded or process-based architecture. NGINX uses a complex event-driven architecture) this architecture can support thousands of concurrent connections on modern hardware.

Inside NGINX infographic involves the illustration from high-level process architecture mining to NGINX single process processing multi-connection. This article explains these details.

Set scenario-NGINX process model

Setting the Scene? The NGINX Process Model

Ps: // watermark "height =" 297 "width =" 500 "/>

To better understand the design, you need to understand how NGINX works. NGINX has a master process (which performs privileged operations, such as reading configurations and binding ports) and a series of worker processes and helper processes ).

In this quad-core server, the NGINX main process creates four working processes and two cache auxiliary processes (cache helper processes) to manage the disk content cache (on-disk content cache ).

Why is architecture important?

Why Is Architecture Important?

The fundamental foundation of any Unix application is a thread or process. (From the perspective of the Linux operating system, threads and processes are basically the same, the main difference is their degree of shared memory .) A process or thread is a set of independent instruction sets that can be scheduled by the operating system and run on the CPU kernel. Most complex applications run multiple threads or processes in parallel for two reasons:

● More computer kernels can be used at the same time.

● Threads and processes make parallel operations easy (for example, processing multiple connections at the same time ).

Both processes and threads consume resources. They all use memory and other OS resources, causing frequent kernel switching (called context switch ). Most modern servers can process hundreds of small, active (active) threads or processes at the same time, but once the memory is exhausted or the high I/O load leads to a large number of context switches, the performance of the server will seriously decline.

For network applications, a thread or process is usually assigned to each connection. This architecture is easy to implement, but when applications need to process thousands of concurrent connections, the scalability of this architecture will become a problem.

How does NGINX work?

How Does NGINX Work?

NGINX uses a predictable (predictable) process model to schedule available hardware resources:

1. The master process executes privileged operations, such as reading configurations and binding ports, and is also responsible for creating sub-processes (three types below ).

2. The cache loader process runs at startup, loads the disk-based cache into the memory, and then exits. The scheduling is very cautious, so the resource demand is very low.

3. the cache manager process runs cyclically and reduces the disk cache (prunes entries from the disk caches) so that it remains within the configured range.

4. worker processes is the process that executes all the actual tasks: processes network connections, reads and writes content to the disk, and communicates with upstream servers.

In most cases, NGINX recommends that one worker process be run for each CPU core to make the most effective use of hardware resources. You can set the following command in the configuration:

Worker_processes auto

When the NGINX server is running, only the working process is busy. Each worker process processes multiple connections in a non-blocking manner to reduce the overhead of context switching.

Each worker process is a single thread and runs independently. It captures and processes new connections. The cache data, session persistence data, and other shared resources are shared between processes through memory sharing.

NGINX internal working process

Inside the NGINX Worker Process

Every NGINX working process is initialized by the NGINX configuration, and a group of listening sockets (listen sockets) is set by the main process ).

The NGINX working process listens to events on the socket (accept_mutex and kernel socket sharding) to determine when to start working. Events are initiated by new connections. These connections will be allocated to the state machine-the HTTP state machine is the most common, but NGINX is also a stream (native TCP) and a large number of mail protocols (SMTP, IMAP, and POP3) the state machine is implemented.

A state machine is essentially a group of commands that tell NGINX how to process requests. Most web servers with the same features as NGINX also use a similar state machine-but the implementation is different.

Scheduling state machine

Scheduling the State Machine

Imagine a state machine as a chess rule. Each HTTP transaction is a chess game. One side of the board is the web server-a master player who can quickly make decisions. The other side is the remote client-the web browser that accesses the site or application in a relatively slow network.

However, the rules may be complex. For example, a web server may need to communicate with each other (acting as an upstream application) or the authentication server. Third-party modules of web servers can also expand competition rules.

Blocking state machine

A Blocking State Machine

Recall our previous description of processes and threads: it is a set of independent instruction sets that can be scheduled by the operating system and run on the CPU kernel. Most web servers and web applications use a connection/process or a connection/thread model to perform this chess game. Each process or thread contains a command to finally play the game. In this process, the process is run by the server, and most of its time is spent on blocking (blocked), waiting for the client to complete its next action.

1. web server process listens to new connections (new competitions initiated by the client) on the listening socket ).

2. After a new game starts, the process starts to work. After each game is completed, it is blocked and waits for the client to take the next step.

3. Once the competition ends, the web server process will check whether the customer wants to start a new competition (this is equivalent to a surviving connection ). If the connection is closed (the client leaves or times out), the web server process returns to the listening status and waits for a new game.

Remember the important thing: every active HTTP connection (each chess game) requires a dedicated process or thread (a master player ). This architecture is very easy to expand third-party modules ("New Rules "). However, there is a huge imbalance: a lightweight HTTP connection represented by a file descriptor and a small amount of memory, will be mapped to a separate process or thread-they are very heavyweight operating system objects. This is convenient for programming, but it causes a huge waste.

NGINX is a real master

NGINX is a True Grandmaster

Maybe you have heard of the wheel performance competition. In the competition, a chess master wants to deal with dozens of opponents at the same time.

Kiril Georgiev played 360 games against 284 players in Sofia, Bulgaria's capital, and eventually won victories, 70 draws and six losses.

This is how the NGINX working process plays "chess. Every working process is a master (remember: Normally, each working process occupies a CPU core) and can play against hundreds of players (in fact thousands) At the same time ).

1. The working process waits for the event on the listening socket and connection socket.

2. When an event occurs on a socket, the worker process processes the event.

● Listening for events on sockets means that the client starts a new game. The worker creates a new connection socket.

● Events connected to the socket mean that the client has moved the pawns. The working process will respond quickly.

A working process never stops on the network. It is always waiting for its "competitor" (client) to respond. When it has moved the pawns in this game, it will immediately handle the next game or greet new opponents.

Why is it faster than a congested multi-process architecture?

Why Is This Faster than a Blocking, Multi-Process Architecture?

The scale of NGINX can well support tens of thousands of connections on each worker process. Each new connection creates another file descriptor and consumes a small amount of extra memory in the working process. Each connection consumes very little. NGINX processes can maintain a fixed CPU usage. When there is no work, context switching is also less.

In a blocking, one connection/one process mode, each connection requires a lot of additional resources and overhead, and context switching (from one process to another) is very frequent.

For more information, see the article on NGINX architecture written by Andrew Alexeev, vice president of NGINX development and co-founder.

Through proper system optimization, NGINX can process 100,000 concurrent HTTP connections for each worker process on a large scale, and no information is lost during traffic peaks (starting from the new game ).

Configuration update and NGINX upgrade

Updating Configuration and Upgrading NGINX

The NGINX process architecture that only contains a few working processes makes the configuration and updates of binary files very efficient.

Updating NGINX configuration is a simple, lightweight, and reliable operation. Run nginx? The s reload command checks the configuration on the disk and sends a SIGHUP signal to the main process.

When the main process receives the SIGHUP signal, it will do two things:

1. Reload the configuration and fork a new working process. These new working processes immediately begin to accept connections and process traffic (using new configurations ).

2. Send a signal to notify the old working process to exit quietly. These old processes will no longer accept new connections. As long as the HTTP requests they process end, they close the connection cleanly. Once all connections are closed, the worker process exits.

This process will cause a small peak of CPU usage and memory usage, but this small peak is negligible compared to loading resources from an active connection. You can reload the configuration multiple times in one second. In rare cases, generation after generation of worker processes may encounter problems when the connection is closed, but even if the problem occurs, they will be immediately solved.

The binary upgrade process of NGINX is even more amazing-you can quickly upgrade NGINX itself without any loss of connection, downtime, or service interruption on the server.

The binary upgrade process is similar to the configuration update process. The new NGINX master process is in parallel with the original master process, and they share the listening socket. Both processes are active and their respective working processes process their respective traffic (traffic ). Then, you can notify the old master process and its working process to exit perfectly.

In Controlling NGINX, the entire process is described in more detail.

Conclusion

Conclusion

The internal chart of NGINX gives a high overview of how NGINX works, but behind this simple explanation is more than ten years of innovation and optimization. These innovations and optimizations enable NGINX to deliver excellent performance on a variety of hardware, as well as the security and reliability required by modern web applications.