Starting from nginx development to mastering nginx Platform

Preliminary Research on nginx architecture (100%)

As we all know, nginx has high performance, and its high performance is inseparable from its architecture. So what exactly is nginx? In this section, let's first get to know the nginx framework.

After nginx is started, it runs in the background in the form of daemon on the UNIX system. The background process contains one master process and multiple worker processes. We can also manually turn off the background mode so that nginx can run on the foreground, and enable nginx to cancel the master process through configuration, so that nginx can run in a single process. Obviously, we certainly won't do this in the production environment, so the background mode is usually used for debugging. In the following sections, we will explain in detail how to debug nginx. Therefore, we can see that nginx works in the form of multi-process. Of course, nginx also supports multithreading, but our mainstream approach is also the multi-process approach, it is also the default nginx method. Nginx has many advantages in adopting the multi-process method, so I will mainly explain the multi-process mode of nginx.

As mentioned earlier, after nginx is started, there will be one master process and multiple worker processes. The master process is mainly used to manage worker processes, including receiving external signals, sending signals to worker processes, and monitoring the running status of worker processes, when the Worker Process exits (abnormal), it automatically restarts the new worker process. The basic network events are handled in the worker process. Multiple worker processes are peer-to-peer. They compete for requests from clients, and each process is independent of each other. One request can only be processed in one worker process. One worker process cannot process requests from other processes. The number of worker processes can be set. Generally, the number of CPU cores is the same as that of the machine. The reason is that the nginx process model and event processing model are inseparable. Nginx process model, which can be represented by the following sources:

After nginx is started, what should we do if we want to operate nginx? From the above, we can see that the master manages the worker process, so we only need to communicate with the master process. The master process will receive signals from outside, and then perform different tasks based on the signals. Therefore, to control nginx, you only need to send a signal to the master process through kill. For example, kill-hup pid tells nginx to restart nginx with ease. We usually use this signal to restart nginx or re-load the configuration, because it is easy to restart, therefore, the service is uninterrupted. How does the master process do it after receiving the HUP signal? First, the master process will re-load the configuration file after receiving the signal, then start the new worker process, and send a signal to all old worker processes, telling them that they can retire with honor. After the new worker starts, it starts to receive new requests, while the old worker no longer receives new requests after receiving a signal from the master, and exit after all unprocessed requests in the current process are processed. Of course, sending signals directly to the master process is an old operation method. nginx introduced a series of command line parameters after version 0.8 to facilitate our management. For example,./nginx-s reload is to restart nginx,./nginx-s stop is to stop nginx running. How can we do this? For reload, we can see that when executing a command, we start a new nginx process, and after the new nginx process is parsed to the reload parameter, we know that our goal is to control nginx to reload the configuration file. It will send a signal to the master process, and then the next action will be the same as sending a signal directly to the master process.

Now, we know what is done inside nginx when we are operating nginx. How does the Worker Process requests? As we have mentioned above, worker processes are equal, and each process has the same opportunity to process requests. When we provide an HTTP service with port 80 and a connection request comes over, every process may process this connection. How can this problem be solved? First, each worker process comes from the master process fork. In the master process, first establish the socket (listenfd) that requires listen, and then fork multiple worker processes. Listenfd of all worker processes becomes readable when new connections are established. To ensure that only one process processes process the connection, all worker processes obtain the accept_mutex before registering the listenfd read event, the process that grabbed the mutex lock registers the listenfd read event and calls accept to accept the connection in the read event. After a worker process connects to accept, it starts to read the request, parse the request, process the request, generate the data, and then return it to the client. The connection is closed, such a complete request is like this. We can see that a request is completely handled by the worker process and only processed in one worker process.

So what are the advantages of nginx adopting this process model? Of course, there will certainly be many benefits. First, for each worker process, there is no need to lock an independent process, saving the lock overhead and facilitating programming and problem searching. Second, the use of independent processes can ensure that they do not affect each other. After a process exits, other processes are still working and the service will not be interrupted. The master process will soon start a new worker process. Of course, the abnormal exit of the Worker Process must be due to a bug in the program. The abnormal exit will cause all requests on the current worker to fail, but will not affect all requests, thus reducing the risk. Of course, there are many benefits that you can learn from.

The above describes a lot of nginx process models. Next, let's take a look at how nginx handles events.

Someone may ask, nginx uses multiple worker methods to process requests. Each worker has only one master thread, so the number of concurrent jobs that can be processed is limited, how many workers can process and how many concurrency can be processed? Why is high concurrency? That is, nginx uses asynchronous and non-blocking methods to process requests. That is to say, nginx can process thousands of requests simultaneously. Think about the common working methods of Apache (APACHE also has an asynchronous non-blocking version, but it is not frequently used because it conflicts with some built-in modules). Each request occupies a single worker thread, when the number of concurrent requests reaches several thousand, thousands of threads are processing requests at the same time. This is a great challenge for the operating system. The memory occupied by threads is very large, and the CPU overhead caused by thread context switching is very high, so the natural performance will not go up, these overhead are completely meaningless.

Why does nginx support asynchronous non-blocking or asynchronous non-blocking? Let's first go back to the origin to see the complete process of a request. First, you need to establish a connection for the request, then receive the data, and then send the data after receiving the data. At the bottom of the system, it is a read/write event. When a read/write event is not ready, it cannot be operated. If you do not use a non-blocking method, You have to block the call, if the event is not ready, you have to wait. When the event is ready, continue. Blocking calls will enter the kernel waiting, and the CPU will be used by others. It is obviously not suitable for single-threaded worker. When there are more network events, everyone will be waiting, when the CPU is idle, no one will use it. The CPU utilization will naturally fail, let alone high concurrency. Well, you can add the number of processes. What is the difference between this and the Apache thread model? Be sure not to add unnecessary context switches. Therefore, in nginx, blocking system calls are the most taboo. Don't block it, It's not blocking. Non-blocking means that the event is not ready, and the eagain will be returned immediately, telling you that the event is not ready yet. What are you panic about? Come back later. Okay. After a while, check the event again until the event is ready. During this period, you can do other things first, and then check whether the event is ready. Although it is not blocked, you have to check the event status from time to time. You can do more, but the overhead is not small. Therefore, there will be an asynchronous non-blocking event processing mechanism, specifically to system calls such as select/poll/epoll/kqueue. They provide a mechanism for you to monitor multiple events at the same time, and call them is blocked, but you can set the timeout time. If an event is ready within the timeout time,. This mechanism solves the two problems above. Taking epoll as an example (in the following example, we use epoll as an example to represent this type of function), when the event is not ready, put it in epoll. When the event is ready, we will read and write it. When the Read and Write returns eagain, we will add it to epoll again. In this way, as long as an event is ready, we will handle it. Only when all events are not ready will we wait in epoll. In this way, we can process a large number of concurrent requests concurrently. Of course, the concurrent requests here refer to the unfinished requests with only one thread, therefore, there is only one request that can be processed at the same time, but the requests are constantly switched. The switchover is also because the asynchronous events are not ready and the requests are automatically made. There is no price for switching here. You can understand it as loop processing of multiple prepared events. In fact, this is the case. Compared with multithreading, this event processing method has great advantages. No threads need to be created, and each request occupies a small amount of memory. There is no context switch, and event processing is very lightweight. A large number of concurrent operations will not cause unnecessary resource waste (Context switching ). More concurrency only occupies more memory. I have tested the number of connections. On a 24 GB memory machine, the number of concurrent requests processed has exceeded 2 million. Currently, most network servers use this method, which is also the main reason for nginx performance efficiency.

As we have said before, it is recommended to set the number of workers to the number of CPU cores. It is easy to understand here that more workers will only lead to processes competing for CPU resources, this leads to unnecessary context switching. In addition, to better utilize the multi-core features, nginx provides CPU affinity binding options. We can bind a process to a certain core, in this way, the cache will not become invalid due to process switching. Such a small optimization is very common in nginx, and it also shows the painstaking efforts of nginx authors. For example, when nginx compares four-byte strings, it converts the four characters into an int type and then compares them to reduce the number of CPU commands.

Now, I know why nginx chose this process model and event model. For a basic web server, there are usually three types of events: Network events, signals, and timers. As mentioned above, network events can be effectively solved through asynchronous non-blocking. How to process signals and timers?

First, the signal processing. For nginx, some specific signals represent specific meanings. The signal will interrupt the current operation of the program, and continue to execute after the status changes. If it is a system call, the system call may fail and must be re-imported. You can learn some professional books about signal processing. For nginx, if nginx is waiting for the event (epoll_wait), if the program receives a signal, after the signal processing function completes, epoll_wait will return an error, then the program can go to epoll_wait for calling again.

In addition, let's take a look at the timer. Since epoll_wait and other functions can set a timeout time when calling, nginx uses this timeout time to implement the timer. The timer events in nginx are placed in a red/black tree that maintains the timer. Each time before entering epoll_wait, the minimum time of all timer events is obtained from the red/black tree, enter epoll_wait After calculating the epoll_wait timeout. Therefore, epoll_wait times out when no event is generated or the signal is interrupted. That is to say, the timer event arrives. In this case, nginx checks all timeout events, sets their status to timeout, and then processes network events. From this we can see that when we write nginx code, when processing the network event callback function, the first thing we usually do is to judge the timeout and then process the network event.

We can use a piece of pseudo code to summarize the nginx event processing model:

Starting from nginx development to mastering nginx Platform