Understanding of Web servers

soon to graduate, but also to start looking for a job, the university wrote so many code, but not a good summary of the commonly used concept is a pity amount, through this blog record my most commonly used knowledge of the good.

When it comes to Web servers, there are a lot of articles are introduced very well, before I saw a very good, to my help is very big, but now can not find the original text, see the blog is reproduced in the park, I am here also remember that good, here very grateful to the author's analysis, benefited.

So before you say a Web server, talk about threads, processes, and the number of concurrent connections.

1. Processes and Threads

A process is a program with a certain independent function, about one run activity on a data set, and a process is an independent unit of the system for resource allocation and scheduling. From a logical point of view, the meaning of multithreading is that in an application (process), there are multiple execution parts that can be executed concurrently. But the operating system does not implement multiple threads as separate applications, but as a process to dispatch and manage as well as resource allocations. This is the important difference between a process and a thread, where the main difference between a process and a thread is that the process has a separate address space, and after a process crashes, it does not affect other processes in protected mode, and the thread is just a different execution path in a process. Thread has its own stack and local variables, but there is no separate address space between the threads, a thread dead is equal to the entire process dead, so the multi-process program is more robust than multithreaded programs, but in the process of switching, the cost of large resources, efficiency is worse. But for some concurrent operations that require simultaneous and shared variables, only threads can be used, and processes cannot be used. Let's talk about the number of concurrent connections.

2. Number of concurrent connections

(1). What is the maximum number of concurrent connections?

The maximum number of concurrent connections is the maximum number of sessions that the server can handle at the same time.

(2). What is a session?

we open a Web site is a client browser with the server side of a session, and we browse the Web page is based on the HTTP protocol.

(3). How does the HTTP protocol work?

HTTP supports two ways to make connections: non-persistent and persistent (HTTP1.1 The default connection mode is persistent).

(4). The following 7 steps will be completed between the browser and the Web server

• Establish a TCP connection

• Web browser sends request command to Web server

• Web browser sends request header information

• Web server Answer

• Web server sends answer header information

• Web server sends data to browser

• Web server shuts down TCP connection

In general, once the Web server sends the request data to the browser, it closes the TCP connection, but the browser typically joins this line of code connection:keep-alive, and the TCP connection remains open after it is sent, The browser can continue to send requests through the same connection. Maintaining the connection purpose saves the time required to establish a new connection for each request and also saves network bandwidth.

3. How to calculate the number of concurrent connections

When the user downloads the file on the server, it is a connection and the user file is downloaded and the connection disappears. Sometimes users with the multi-threaded way of thunder download, this one user opened 5 threads, even if it is 5 connections.

Users open your page, even if the page does not make any request to the server, then the user open one side of the next 15 minutes will be counted an online.

If the user continues to open other pages of the same site, the number of people online will be 15 minutes after the user's last click (Making the request), and in this 15 minute, regardless of how the user clicks (including the new window), the person is online.

When the user opens the page and closes the browser normally, the user's online number will be cleared immediately.

Second, the way the Web server provides services

a Web server needs to support this multi-tasking approach in some way because it wants to serve multiple customers at the same time. In general, there are three ways to choose, Multi-process mode , multi- threaded mode and asynchronous mode . Among them, the multi-process mode of the server to a customer to use a process to provide services, because in the operating system, the generation of a process requires process memory replication, such as additional overhead, so that when the customer more performance will be reduced. To overcome the additional overhead of this build process, you can use multithreading or asynchronous methods. In multithreaded mode, services are provided using multiple threads in a process, and performance increases due to the small overhead of the thread. In fact, it does not require any extra overhead or asynchronously, it communicates with each customer in a non-blocking way, and the server uses a process to poll the line.

Although the asynchronous approach is most efficient, it has its own drawbacks. Because of the asynchronous way, scheduling between multiple tasks is done by the server program itself, and once a problem occurs in one place, the entire server is in trouble. Therefore, to add functionality to such a server, on the one hand to comply with the server's own specific task scheduling, on the other hand to ensure that there is no error in the code, which limits the functionality of the server, so that the asynchronous Web server is the most efficient, but simple function, such as Nginx server .

Because multithreading uses threads for task scheduling, server development is simple and facilitates multi-person collaboration due to standards compliance. However, multiple threads are in the same process and can access the same memory space, so there is a thread-wide impact, and the requested memory must be secure for application and release. For a server system, because it takes days, months or even years of continuous operation, a little bit of error will gradually accumulate and eventually affect the normal operation of the server, it is difficult to write a high-stability multi-threaded server program. However, it is not impossible to do so when. The Apache worker module is a good way to support multithreading.

The advantage of a multi-process approach is stability, because when a process exits, the operating system recycles its resources so that it does not leave any garbage behind. Even if there are errors in the program, because the process is isolated from each other, the error does not accumulate, but is cleared as the process exits. Apache's Prefork module is a module that supports multiple processes.

Three, multi-process, multi-threading, Asynchronous Pattern comparison

Web servers generally provide services in three ways, multi-process , multi-threaded, and asynchronous. the most efficient is the asynchronous way, the most stable is the multi-process mode, the use of less resources is multi-threaded way.

1. Multi-process

in this architectural approach, the Web server generates multiple processes that process multiple user requests in parallel, and processes can be generated on demand or in advance. Some Web server applications respond by generating a separate process for each user request, but when the number of concurrent requests reaches tens of thousands, multiple concurrently running processes consume a large amount of system resources. (that is, each process can respond to only one request or multiple requests per process)

Advantages:

• The biggest advantage is stability, and one process error does not affect other processes. For example, the server connects 100 simultaneous requests to 100 processes, one of which fails, kills only one process, and 99 processes continue to respond to user requests. Each process responds to a request

Disadvantages:

• High process volume, too many process switches, resulting in inefficient use of CPU resources, each process's address space is independent, a lot of spatial duplication of data, so memory use efficiency is low, process switching due to kernel completion, CPU resource consumption.

2. Multithreading

in a multithreaded manner, each thread responds to a request, and because of the sharing of the process's data between threads, the threads are less expensive and performance increases.

Advantages:

• The process data is shared between threads, each thread responds to a request, thread switching is unavoidable (switching magnitude is lighter), the thread of the same process can share many resources of the process, the need for memory is much lower than the process, read can be shared, write can not be shared

Disadvantages:

• Thread jitter When threads switch quickly, multithreading can cause server instability

3. Async mode

a process or thread responds to multiple requests, does not require any additional overhead, has the highest performance, and consumes the least resources. But there are problems. One, however, a process or thread error can lead to downtime for the entire server.

Iv. WEB Service Request process

      

• 1. First, our client sends a request to the Web server, The request is to the NIC first. 2. the network card will request to the kernel space kernel processing, is actually unpacking, found that the request is 80 port. 3. The kernel sends the request to the Web server in the user space, and the Web server accepts the index.html page to request discovery client requests. 4. the Web server makes a system call to send the request to the kernel. 5. the kernel discovers that a page is requested, the driver of the disk is called, and the disk is connected. 6. the kernel obtains a paging file by driving the calling disk. 7. The kernel saves the resulting paging file in its own cache area and notifies the Web process or thread to fetch the corresponding paging file. 8. the Web server copies the paging files in the kernel cache to the process cache area through system calls. 9. the Web server obtains the paging file in response to the user and sends the paging file to the kernel again through system calls. 10. the encapsulation of the kernel process paging file and sends it through the network card. when the message arrives at the network card, it responds to the client via the Web

Simply put: The user requests--to the user space--the system calls--the kernel space--and the kernel-to-disk read page resource--returns to user space--response to the user. The above simple explanation of the client to the Web service request process, in this process, there are two I/O process, one is the client requested network I/O, and the other is the Web server request page disk I/O. Let's talk about the I/O model for Linux.

V. Linux I/O model

1.I/O Model Classification

Description: We all know The Web server's process responds to a user request, but cannot directly manipulate the I/O device, and it must request kernel through a system call to assist with the I/O action, such as:

For data entry, it takes time to wait for (wait) data to enter buffer, while copying data from buffer to process takes time
Depending on the wait mode, I/O actions can be divided into five modes.

• 1. Block i/o,2. non-blocking i/o,3. I/O multiplexing (SELECT and poll), 4. signal (Event) driver I/O (SIGIO), 5. asynchronous I/O (Aio_ series functions for posix.1)

Terminology related to 2.I/O models

(1). Blocking and non-blocking:

Blocking and non-blocking refers to whether an operation is performed until the end of the operation, or return immediately. For example, you go to the station to meet friends, this is an operation. There are two ways to do this. The first kind, you this person special honest, old already arrived at the station to wait for the car to arrive the friend so far. The second, you arrive at the station, ask the train of duty to come no, "not yet", you go out to stroll around, may come back to ask again. The first is blocking, and the second is non-blocking. I think blocking and non-blocking is about doing things, it's about people who do things.

(2). Synchronous and Asynchronous:

synchronization and Asynchrony are another concept, which is a property of the event itself. For example, the boss let you to carry a pile of stone, and only let you do it alone, you have to go, the final result is to move the end, or you hit the foot, only the move is finished you know. This is the synchronous event. If the boss also give you a younger brother, you can let the younger brother to move, the move finished Sue you. This becomes asynchronous. In fact, asynchronous can be divided into two types: with notification and without notification. The one in front of you belongs to the one with the notice. Some younger brother work may not be enough initiative, do not actively inform you, you need to pay attention to the state from time to time. This is asynchronous without notification. for synchronized events, you can only do this in a blocking manner. For asynchronous events, blocking and non-blocking are all possible. There are two ways of non-blocking: actively querying and passively receiving messages. Passive does not mean that it is not good, here it is precisely the more efficient, because in the active query most of the query is doing no work. Both are available for asynchronous events with notifications. And for without notice, you can only use active query.

(3). / o

back to I/O, whether I or O, access to peripherals (disks) can be divided into two phases: request and execution. The request is to look at the status information of the peripheral (such as whether it is ready), and execution is the real I/O operation. Prior to Linux 2.6, only "request" was an asynchronous event, and after 2.6, AIO was introduced to "execute" asynchronously. Although Linux/unix is used as a server, this is a lot behind windows, and the IOC (AIO on Windows) is on the Win2000.

(4). Summary

The "Execution" phase of the top four I/O models on Linux is synchronous, and only the last one achieves true full asynchrony. The first type of blocking is the most primitive method and the most exhausting way. Of course tired and not tired to see who. The application is dealing with the kernel. This is the most tiring way for an application, but it is the easiest way for the kernel to do this. Also take pick this thing as an example, you are the application, the attendant is the kernel, if you go to have been waiting, the attendant on the convenient. Of course now the computer design, including the operating system, more and more for the end user to consider, in order to satisfy the user, the kernel slowly assume more and more work, the evolution of the IO model is the same. non-blocking I/O, I/O multiplexing, and signal-driven I/O are all non-blocking, of course, for the "request" phase. Non-blocking is the active querying of peripheral state. The select,poll in I/O multiplexing is also active query, the difference is that select and poll can simultaneously query the state of multiple FD (file handle), and select has the limit of FD number. Epoll is based on the callback function. The signal-driven I/O is based on the signal message. These two should be in the "passive Receive message" category. Finally, the emergence of the great AIO, the kernel has done everything, the application of the upper layer to achieve full-asynchronous, the best performance, of course, the highest degree of complexity.

Six, Linux I/O model specific description

first, let's take a look. simple matrices for basic Linux I/O models , we can see models with synchronous blocking I/O (blocking I/O), synchronous non-blocking I/O (non-blocking I/O), asynchronous blocking I/O (I/O multiplexing), asynchronous nonblocking I/O (two types, signal-driven I/O, and asynchronous I/O). All right, now, let's talk about it in detail.

1. Blocking I/O

Description: The application invokes an IO function that causes the application to block and wait for the data to be ready. If the data is not ready, wait until the data is ready, copy from the kernel to the user space, and the IO function returns a success indicator. This doesn't have to be explained, blocking sockets. Is the diagram of the procedure it invokes: (Note that the general network I/O is blocking I/O, the client makes the request, the Web server process responds, and the request is in a waiting state until the process does not return the page)

2. Non-blocking I/O

We set up a set of interfaces to non-blocking is to tell the kernel, when the requested I/O operation cannot be completed, do not sleep the process, but return an error. This way our I/O operations function will constantly test whether the data is ready, and if not, continue testing until the data is ready. During this ongoing testing process, CPU time is heavily consumed, and the I/O model is not used by all general Web servers. Specific processes such as:

3.I/O multiplexing (SELECT and poll)

The I/O multiplexing model uses the Select or poll function or the Epoll function (which is supported by the Linux2.6 kernel), which also blocks the process, but unlike blocking I/O, these two functions can block multiple I/O operations at the same time. I/O functions can be detected at the same time for multiple read operations, multiple write operations, and I/O operation functions are not actually invoked until there is data readable or writable. Specific processes such as:

4. Signal-driven I/O (SIGIO)

First, we allow the socket interface to drive the signal-driven I/O and install a signal processing function, and the process continues to run without blocking. When the data is ready, the process receives a sigio signal that can be called by the I/O operation function in the signal processing function to process the data. Specific processes such as:

As can be seen, the more backward, the less the blockage, the theoretical efficiency is also optimal . Of the five I/O models, the first three are synchronous I/O, the latter of which are asynchronous I/O.

Synchronous I/O:

• 1. Block i/o,2. non-blocking i/o,3. I/O multiplexing (SELECT and poll)

asynchronous I/O:

• 1. Signal-driven I/O (SIGIO) (semi-asynchronous), 2. asynchronous I/O (posix.1 Aio_ series functions) (True async)

Differences between asynchronous I/O and signal-driven I/O:

• In the signal-driven I/O mode, the kernel can copy the notification to our application when sending a Sigio message. in asynchronous I/O mode, the kernel notifies our application only after all operations have been completed by the kernel operation.

Vii. specific implementation of Linux I/O model

1. There are several main ways to achieve this:

• Select,poll,epoll,kqueue,/dev/poll,iocp

where IOCP is implemented by Windows, select, poll, Epoll are implemented by Linux, Kqueue is implemented by FreeBSD, and/dev/poll is implemented by Sun's Solaris. Select, poll corresponds to the 3rd type (I/O multiplexing) model, IOCP corresponds to the 5th (asynchronous I/O) model, then Epoll, Kqueue,/dev/poll? In fact, the same model as SELECT, just a bit more advanced, can be seen as having a 4th (signal-driven I/O) model of some features, such as the callback mechanism.

2. Why Epoll, Kqueue,/dev/poll than select Advanced?

the answer is that they are not polled. Because they replaced it with callback. Think about it, when the socket is more, each time select () through the traversal fd_setsize a socket to complete the dispatch, regardless of which socket is active, all traverse over again. This can waste a lot of CPU time. If you can register a callback function with the socket, and when they are active, the related operation is done automatically, which avoids polling, which is what Epoll, Kqueue,/dev/poll do. This may not be easy to understand, then I say a realistic example, if you are in college, living in the dormitory building has a lot of rooms, your friends will come to you. Select version of the tube aunt will take your friends to the room to find, until you find you. and epoll version of the house tube aunt will first write down each classmate's room number, when your friend comes, just tell your friend you live in which room can, do not have to personally take your friend filled the building to find someone. If there are 10,000 people, you have to find their own living in this building students, select version and Epoll version of the aunt, who is more efficient, self-evident. Similarly, in high concurrent servers, polling I/O is one of the most time-consuming operations, and the performance of Select, Epoll, and/dev/poll is also very clear.

3.Windows or *nix (IOCP or Kqueue, Epoll,/dev/poll)?

Admittedly, Windows IOCP is very good and there are few systems that support asynchronous I/O , but because of the limitations of the system itself, large servers are still under UNIX. And as mentioned above, Kqueue, Epoll,/dev/poll and IOCP are more than a layer of blocking from the kernel copy data to the application layer, and thus cannot be counted as asynchronous I/O classes. However, this layer of small blockage insignificant, kqueue, Epoll,/dev/poll have done very good.

4. Summarize some of the highlights

only IOCP (Windows implementation) is asynchronous I/O, and other mechanisms are more or less blocked. Select (Linux implementation) is inefficient because it requires polling every time. But inefficient is also relative, depending on the situation, can also be improved by good design Epoll (Linux implementation), Kqueue (FreeBSD implementation),/dev/poll (Solaris Implementation) is the Reacor mode, IOCP is the Proactor mode. Apache 2.2.9 previously supported only select models, 2.2.9 supported Epoll models,Nginx supported Epoll models,Java NIO packages are select models

• (end) by JONNYF

Understanding of Web servers