[Reprint] solve the problem of "sticking packets" for TCP network transmission

Solve the problem of "sticking packets" for TCP network transmission

Author: Yang Xiaoping Wang shengkai

Source: http://www.ciw.com.cn/

Currently, TCP/IP communication protocol and its standard socket application programming interface (API) are widely used in network transmission applications ). The TCP/IP transport layer has two parallel protocols: TCP and UDP. TCP (transport control protocol) is connection-oriented and provides high-reliability services. User Data protocol (UDP) is connectionless and provides efficient services. In practical engineering applications, the selection of reliability and efficiency depends on the application environment and requirements. In general, the network transmission of common data adopts high-efficiency udp, and the network transmission of important data adopts high-reliability TCP.

During application development, I found that TCP-based applications sometimes stick packets (that is, several packets of data sent by the sender are stuck to a packet when receiving data from the receiver ). In response to this situation, we conducted special research and experiments. This article focuses on the TCP network packet sticking problem, and puts forward countermeasures and methods to solve the problem based on the experimental results, for the reference of relevant engineering and technical personnel.

1. Introduction to the TCP protocol

TCP is a connection-oriented transport layer protocol. Although TCP is not a set of protocols set by iso, it has become a de facto network standard due to its successful application in the business and industry, it is widely used for communication between various network hosts.

As a connection-oriented transport layer protocol, TCP aims to provide users with reliable end-to-end connections to ensure orderly transmission of information. In addition to providing basic data transmission functions, it also uses a series of measures such as data number, verification and calculation, and data validation to ensure reliability. It numbers each transmitted data byte and requests the recipient to return the confirmation message (ack ). If the sender fails to receive the data confirmation within the specified time, the sender retransmits the data. The data number enables the receiver to handle data out of order and duplication issues. The data error is solved by adding a checksum in each transmitted data segment. After receiving the data, the receiver checks the checksum. If the checksum is incorrect, the data segment with the error code is discarded and the sender is required to re-transmit the data segment. Traffic control is also an important measure to ensure reliability. If there is no traffic control, a large amount of data may be lost due to the overflow of the receiving buffer, resulting in many retransmission, resulting in a vicious circle of network congestion. TCP uses a Variable Window for traffic control. The receiver controls the volume of data sent by the sender.

TCP provides users with high-reliability network transmission services, but the reliability assurance measures also affect the transmission efficiency. Therefore, in practical engineering applications, only the transmission of key data uses TCP, while the transmission of common data generally uses high-efficiency udp.

Ii. Analysis and Countermeasures of sticking packets

A tcp packet sticking refers to the packet data sent by the sender to the receiver when it is received. From the receiving buffer, the header of the next packet is followed by the end of the previous packet data.

There are many reasons for the phenomenon of sticking packets, which may be caused by both the sender and the receiver. The adhesive packet caused by the sender is caused by the TCP protocol itself. To improve transmission efficiency, the sender often needs to collect enough data before sending a packet of data. If few data is sent several times in a row, TCP will merge the data into a packet based on the optimization algorithm and send the data once. In this way, the receiver receives the data in the sticky packet. The user process of the receiver fails to receive data in time, leading to the packet sticking phenomenon. This is because the receiver puts the received data in the system receiving buffer, and the user process obtains data from the buffer. If the data of the previous packet has not been taken away by the user process when the next packet arrives, then, when the next packet of data is placed in the system's receiving buffer zone, it is received after the previous packet of data, and the user process obtains data from the system's receiving buffer zone according to the preset buffer size, in this way, multiple packets of data are obtained at a time (as shown in Figure 1 ).

 
Figure 1

 
Figure 2

 
Figure 3

There are two types of packet sticking. One is that all the packets together are complete data packets (figure 1 and figure 2 ), the other case is that the bundled package contains an incomplete package (as shown in Figure 3). Here we assume that the length of the buffer received by the user is m bytes.

Not all sticking packets need to be processed. If the transmitted data is continuous stream data without structure (such as file transmission), you do not have to separate the sticking packets (for short, subcontracting ). However, in actual engineering applications, the transmitted data is usually structured data, and subcontracting is required.

The subcontracting algorithm is relatively simple when dealing with the packet Sticking Problem of the fixed-length structure data. When dealing with the packet Sticking Problem of the variable-length structure data, the subcontracting algorithm is more complicated. In particular, as shown in figure 3, the data content of a package is divided into two consecutive receiving packages, which is difficult to process. Avoid sticking packets in actual engineering applications.

To avoid sticking packets, you can take the following measures. First, you can use programming settings to avoid the packet sticking problem caused by the sender. TCP provides an operation command to force data to be transmitted immediately. After the TCP software receives the operation command, this section of data is immediately sent out, without waiting for the sending buffer to be full; second, for the sticky packet caused by the receiver, you can optimize the program design, streamline the workload of the receiving process, and increase the priority of the receiving process so that it can receive data in a timely manner, so as to avoid the phenomenon of sticking packets. The third is controlled by the recipient, A packet of data is manually controlled and received multiple times based on the structure field, and then merged to avoid sticking to the packet.

The three measures mentioned above all have their shortcomings. Although the first programming setting method can avoid the packet sticking caused by the sender, It disables the optimization algorithm, reduces the network sending efficiency, and affects the application performance. This method is generally not recommended. The second method can only reduce the possibility of sticking packets, but it cannot completely avoid sticking packets. When the sending frequency is high, or the network burst may cause a packet to arrive at the receiver quickly for a certain period of time, the receiver may still be too late to receive the package, resulting in sticking to the package. Although the third method avoids sticking packets, the application is less efficient and is not suitable for real-time applications.

A comprehensive countermeasure is: the receiver creates a preprocessing thread, processes the received data packets, and separates the encapsulated packets. We have tested this method to prove it is efficient and feasible.

3.Programming and implementation

　　1. Implementation Framework

The experimental network communication program uses the socket api programming of TCP/IP protocol. Socket is oriented to the client/server model. TCP implementation framework 4.

Figure 4

　　2. Lab hardware environment:

Server: pentium 350 Microcomputer

Client: pentium 166 Microcomputer

Network Platform: LAN connected by a 10g shared hub

　　3. Lab software environment:

Operating System: windows 98

Programming Language: visual c ++ 5.0

　　4. Main threads

Multi-thread programming is adopted. There are two threads on the server side: the data sending thread and the statistics sending display thread. The client has three threads: The receiving data thread, the receiving pre-processing sticky package thread, and the receiving statistics display thread. Among them, the priority of the sending and receiving threads is set to thread_priority_time_critical (highest priority), the priority of the preprocessing thread is thread_priority_above_normal (higher than the normal priority), and the display thread priority is thread_priority_normal ).

The data structure of the data sent by the experiment is as follows:

Figure 5

　　5. Subcontracting Algorithm

Based on three different packet sticking phenomena, the subcontracting algorithm adopts corresponding solutions. The basic idea is to first forcibly convert the received data stream to be processed (the length is set to m) into a predefined structure data form, and take out the structure data length field, that is, n in figure 5, then, the length of the first packet is calculated based on n.

1) if n <m, it indicates that the data stream contains multiple packets of data. n Bytes are intercepted from its header and saved to the temporary buffer. The rest of the data continues to be processed cyclically until the end of the process.

2) if n = m, it indicates that the data stream content is exactly a complete structured data and can be directly stored in the temporary buffer.

3) if n> m, it indicates that the content of the data stream is not enough to form a complete structure data, and it must be retained for merging with the next package of data before processing.

If you are interested in the specific content of the subcontract algorithm and software implementation, contact the author.

Iv. Analysis of experiment results

　　The experiment results are as follows:

1. in the above experimental environment, when the sum of the Data lengths of several packets continuously sent by the sender is less than B, the packet sticking problem often occurs, after being processed by the preprocessing thread, the receiver can properly unbind the bundled package. If the Program sets "sending without delay" :( setsockopt (socket_name, ipproto_tcp, tcp_nodelay, (char *) & on, sizeof on), where on = 1 ), there is no package sticking.

2. When the sent data is 1 kb ~ If the sending interval is less than 10 ms for a 2 KB data with an indefinite length, the packet will occasionally become stuck. After processing by the preprocessing thread, the receiver can properly unbind the packets together.

3. To determine the time for processing the adhesive packet, the sender sends 1000 kb, 1.9kb, 1.2kb, 1.6kb, and 1.0kb data cyclically, totaling packets. To create a sticky packet, the receiving thread waits for 10 ms each time before receiving the packet. The receiving buffer is set to 526 B. As a result, the receiving thread receives 175 packets of data, of which are packets in length. The pre-processing thread can obtain 1000 correct packets of data, and the total time for processing stick packets is less than 1 ms.

The experimental results show that the TCP packet sticking phenomenon exists, but it can be solved through the preprocessing of the receiver, and the processing time is very short (the total processing time of 1000 packets of data in the experiment is less than 1 ms ), the normal operation of applications is almost unaffected.

Author:

I recently encountered the TCP packet sticking problem in a project. I think this article is very good and I will share it with you.