High-performance network programming (1)-accept Establish a connection? (Reproduced, author: Tao Hui)

When writing a server, many programmers are accustomed to using high-level components, middleware (such as OO (object-oriented) layer encapsulated open source components), compared to the efficiency of the server, they are more concerned about the efficiency of the development of the program, the pursuit of faster completion of the project function point, I hope the application code does not care about communication details. They prefer to implement an interface, implement the various predefined patterns of the component, and set component parameters to achieve the goal in the OO world. Learning complex communication frameworks and bottom-level details is absolutely inefficient in the eyes of programmers accustomed to using OO language. The above practices are understandable, but there are certain limitations, this article describes the network programming head "high-performance", it refers to the programmer designed to write a server needs to deal with a large throughput, and simple network applications have a qualitative difference. Because: 1, high throughput, easy to trigger to some design boundary conditions, 2, contingency small probability event, will become a certainty event at high throughput. 3, Io is slow, high throughput usually means high concurrency, as there are tens, 100,000, millions of TCP active connections at a time. Therefore, to do high-performance network programming can not only be satisfied with the learning of open source components, middleware is how to help me achieve the desired function, for enterprise-class products, need to know more knowledge.

Mastering high-performance network programming, involving the network, operating system protocol stack, processes and threads, common network components and other knowledge points, need to have rich experience in project development, can weigh the efficiency of server operation and project development efficiency. I have been talking about my personal understanding of high-performance network programming.

Above this picture, from top to bottom has the following characteristics:

• Focus, gradually shifting from a specific business to a generic technology

• The use of the scene, from the professional domain to the general domain transfer

• Increased flexibility requirements

• Increased performance requirements

• Mastery of details, principles, and higher requirements

• High demand for handling of unusual situations

• Increased stability and fewer bug rates

In the application layer of network programming, if the server throughput is large, you should be moderately aware of the above layers of concern.

As the red text shows, I think there are 3 points of interest in writing high-performance servers:

1. If programming based on general-purpose components, the focus is on how the component encapsulates the socket programming details. For applications that do not perceive sockets Layer, these components often provide network services to the application-layer code through various callback mechanisms, often in the sense that the component uses a lot of threads (nginx, etc.) for the application layer to provide greater development efficiency, and of course, the use of threads can often reduce the complexity of the code. However, the concurrency resolution mechanism introduced by multithreading still needs to focus on, especially the use of locks. In addition, the use of multithreading means that the application layer of code complexity is thrown to the operating system, high throughput, you need to focus on multithreading to the operating system core performance loss.

Based on generic component programming, for high-performance applications, the following characteristics of the component need to be understood clearly: how to use IO multiplexing or asynchronous IO? How to achieve concurrency? How does the threading model be organized? How to deal with the abnormal conditions caused by high throughput?

2, the General Assembly is only in the encapsulation socket, the operating system by providing sockets to provide network communication capabilities for the process. Therefore, do not understand socket programming, often the performance of the component is not a theoretical understanding. Learning Sockets Layer programming is necessary, perhaps rarely write themselves from scratch, but the operating system API to provide a durable, once learned, useful for life, while in the project architecture design, the choice of which network components are very accurate.

Learn socket programming, focus on: What are the programming methods of sockets? How does blocking sockets block the current code snippet? How does a method on a non-blocking socket not block the current code snippet? How is the IO multiplexing mechanism combined with sockets? How is asynchronous IO implemented? What are the various anomalies of the network protocol, and how the various anomalies of the operating system are passed to the application program through sockets?

3, the complexity of the network will affect the throughput of the server, and, in high-throughput scenarios, a variety of critical conditions can cause the application is not normal, especially if there are bugs in the component or poorly considered or not configured correctly. Understanding the network grouping can locate these problems, can correctly configure the system, components, can correctly understand the bottleneck of the system.

The main focus here is: TCP, UDP, IP protocol features? How do Linux and other operating systems handle these protocols? Analyze each network grouping using tcpdump and other grab kits.

Generally grasp the above 3 points, you can aplomb the implementation of high-performance network server.

The following is a detailed discussion of how to achieve high performance network programming.

As is known to all, Io is the slowest part of the computer, first without looking at disk IO, for network programming, is naturally for network IO. Network protocol has a great impact on network IO, the current, TCP/IP protocol is undoubtedly the mainstream protocol, this article mainly takes the TCP protocol as an example to illustrate the network IO.

The application server in network IO tends to focus on the following several functions that consist of network IO: A) Establish A TCP connection with the client. B) Read the client's request stream. C) send a response stream to the client. D) Close the TCP connection. E) Initiate a TCP connection to another server.

To master these 5 functions, you need not only familiarize yourself with the use of some APIs, but also understand how the underlying network can interact with the upper API. At the same time, there is a need to weigh the development efficiency, processes, threads and combinations of these APIs for different scenarios. These network IO, in turn, are mentioned below.

1. Establish a TCP connection with the client

Before talking about this function, let's look at the relationship between network, protocol and application server:

In the following:

To simplify programming in different scenarios, the TCP/IP protocol family divides the application layer, the TCP Transport layer, the IP network layer, the link layer, and so on, each layer only focuses on a small number of functions.

For example, the IP layer focuses only on how each network packet reaches the destination host, regardless of the destination host's handling.

The most basic function of the transport layer is to focus on the end-to-end, that is, the package issued by a process on a host, and how to reach a process on the destination host. Of course, in order to be reliable, the TCP layer also needs to solve 3 big problems: Packet loss (lost network packets in transit), repetition (multiple identical network groupings caused by protocol layer exceptions), delay (long after network grouping arrives at destination).

The link layer is concerned only with the transmission of network packets in Ethernet or other two-layer networks.

Back to the application layer, it is often necessary to invoke an API similar to accept to establish a TCP connection. The process of establishing a connection everyone knows – three handshakes, how does it interact with the accept? The following is a less precise but understandable diagram:

Friends who study the meaning of the backlog are easy to understand. These two queues are implemented by the kernel, and when the server binds and listens on a port, the SYN queue and the accept queue for that port are established. The client initiates a TCP connection to the server using connect, and when the 1.1-step client's SYN packet arrives at the server, the kernel puts this information in the SYN queue (i.e. the unfinished handshake queue) and returns a Syn+ack packet to the client. After a period of time, when the client again sends an ACK network packet for the server SYN packet in the 2.1 step, the kernel pulls the connection out of the SYN queue and puts the connection in the Accept queue (that is, the completed handshake queue). When the server calls accept in the 3rd step, it actually pulls out the successful connection socket directly from the accept queue.

Now we can discuss the application layer components: Why some application server processes, the use of 1 threads alone, only call the Accept method to establish a connection, such as Tomcat, some application server process, but with 1 threads do all things, including accept to get new connections.

The reason: First, the SYN queue and the Accept queue are not infinite lengths, and their length limits are related to the backlog parameters that are passed when the call to listen listens on an address port. Now that the queue length is a value, will the queue be full? Of course, if the 1th step executes faster than the 2nd step, the SYN queue will continue to grow until the queue is full, and if the 2nd step executes much faster than the 3rd step, the Accept queue will also reach the upper limit. The 1th and 2 steps are not application-controllable, but the 3rd step is the behavior of the application, assuming that the code segment that calls accept to acquire new connections in the process is not executed for long periods of time, such as obtaining locks, io blocking, and so on.

So, when the two queues are full, what happens when the new request arrives?

If the SYN queue is full, the request is discarded, that is, the new SYN network packet is discarded, and if the accept queue is full, it will not cause the connection to be discarded, and the connection will not be removed from the SYN queue, which will aggravate the growth of the SYN queues. Therefore, for the application server, if the accept queue has already established TCP connection, but did not take it out in time, so that once the two queues are full, it will make the client can no longer establish a new connection, causing serious problems.

Therefore, such as Tomcat, such as the server will use a separate thread, only do accept to get the connection, to prevent the timely access to accept the connection.

Then, why, such as Nginx and other servers, in a thread to do the accept, but also do other IO operations?

This brings up the concept of blocking and non-blocking. The application can set the socket on listen to non-blocking mode (default is blocking mode), which causes the Accept method to behave differently. For blocking sockets, accept behavior such as:

As you can see in this image, the first phase of the blocking socket using accept, which is a phase that waits for the accept queue to be empty, is time-consuming and may take a long time for the client to initiate a TCP request to itself.

For non-blocking sockets, accept has two types of returns, such as:

Accept on a non-blocking socket, there is no stage waiting for the accept queue to be empty, it either returns success and gets the established connection or fails to return.

Therefore, in an enterprise-level server process, if a thread is using accept to obtain a new connection, and continue to read on this connection, and write the stream, then the corresponding socket for this connection is usually set to non-blocking. For example, the call to accept does not occupy the CPU time slice of the owning thread for long, so that the thread can do other work in time.

High-performance network programming (1)-accept Establish a connection? (reprinted, Tao Hui)