Transmission design of network TCP data (Sticky packet processing)

1. The film is quoted by others: http://www.cnblogs.com/alon/archive/2009/04/16/1437599.html

Troubleshoot TCP network Transport "Sticky pack" issues

Troubleshoot TCP network Transport "Sticky pack" issues

Yangxiaoping Wang Sheng Open

Original source: http://www.ciw.com.cn/

Currently in the network transmission application, the TCP/IP communication protocol and its standard socket application Development Programming Interface (API) are widely adopted. The TCP/IP transport layer has two parallel protocols: TCP and UDP. TCP (Transport Control Protocol, transmission Protocol) is connection-oriented and provides high reliability services. UDP (User Datagram Protocol, Subscriber Datagram Protocol) is non-connected and provides efficient service. In practical engineering applications, the choice of reliability and efficiency depends on the environment and requirements of the application. In general, the network transmission of ordinary data uses high-efficiency UDP, and the network transmission of important data uses TCP with higher reliability.

In the application development process, I found that applications based on TCP network transmission sometimes appear sticky packet phenomenon (that is, the sender sends a number of packets of data to the receiving party to stick to a packet). In view of this situation, we have carried on the special research and the experiment. In this paper, the problem of TCP network sticky packet is analyzed emphatically, and the countermeasures and methods for solving this problem are put forward, which are used for reference by engineering technicians.

I. Introduction to the TCP protocol

TCP is a connection-oriented transport layer protocol, although TCP does not belong to the ISO set of protocols, but because of its successful application in the business and industry, it has become a de facto network standard, widely used in various network host communication.

As a connection-oriented transport layer protocol, the goal of TCP is to provide users with reliable end-to-end connections to ensure the orderly and accurate transmission of information. In addition to providing basic data transmission functions, it also adopts a series of measures such as data number, checksum calculation and data confirmation to ensure reliability. It numbers each byte of data that is transmitted and requests the receiver to return a confirmation message (ACK). If the sender does not receive the data confirmation within the specified time, it will retransmit the data. The data number allows the receiver to handle the disorder and repetition of the data. The data error problem is solved by adding a checksum to each transmitted data segment, and the receiver checks the checksum after receiving the data, and if the checksum is incorrect, the data segment of the error code is discarded and the sender is required to retransmit it. Flow control is also an important measure to ensure the reliability, if no flow control, may be due to receive buffer overflow and loss of large amounts of data, resulting in a lot of retransmission, causing network congestion vicious circle. TCP uses a variable window for flow control, and the receiver controls the amount of data sent by the sender.

TCP provides users with high-reliability network transmission services, but reliability assurance measures also affect the transmission efficiency. Therefore, in the actual engineering application, only the transmission of the key data is TCP, and the transmission of common data generally uses the high efficiency UDP.

Analysis on the problem of sticky bag and its countermeasures

A TCP sticky packet is a packet of packets sent by the sender to the receiver when it is received, viewed from the receive buffer, followed by the head of the packet data immediately preceding the end of the packet.

There are many reasons for the sticky-packet phenomenon, which may be caused by the sender or by the receiver. The sticky packets caused by the sender are caused by the TCP protocol itself, and TCP is often needed to collect enough data to send a packet of data to improve transmission efficiency. If the data sent several times in a row is very small, usually TCP will be based on the optimization algorithm to synthesize the data packets sent out once, so that the receiver received the sticky packet data. The sticky packet caused by the receiver is due to the fact that the receiver user process does not receive the data in time, resulting in sticky packets. This is because the receiver first put the received data in the system receive buffer, the user process from the buffer to fetch data, if the next packet of data arrives before a packet of data has not been taken away by the user process, the next packet of data into the system receive buffer when the previous packet of data is received, The user process takes the data from the system receive buffer based on the pre-set buffer size, so that it takes more than one packet of data at a time (as shown in Figure 1).

Figure 1

Figure 2


Figure 3

There are two kinds of sticky packets, one of which is glued together is the complete packet (Figure 1, Figure 2), the other case is stuck together with the package has an incomplete package (Figure 3), this assumes that the user receive buffer length of M bytes.

Not all sticky-packet phenomena need to be processed, if the transmitted data is a continuous stream of data without structure (such as file transfer), you do not have to separate the packets of adhesion (short-subcontracting). But in the actual engineering application, the transmitted data is usually the structure data, then needs to do the subcontract processing.

When dealing with the sticky-packet problem of fixed-length structure data, the sub-packet algorithm is simple, and the sub-packet algorithm is more complicated when it deals with the sticky-packet problem of uncertain long structure data. Especially in the case of 3, as a packet of data content is divided into two successive receiving packets, the processing is more difficult. In practical engineering application, the phenomenon of sticking should be avoided as far as possible.

In order to avoid sticking, the following measures can be taken. One is caused by the sender of the sticky packet phenomenon, the user can be programmed to avoid, TCP provides a mandatory data transfer immediately after the operation of the instructions PUSH,TCP software received the operation instruction, the data immediately sent out, without waiting for the transmission buffer full, and the receiver caused by the sticky packet, By optimizing the program design, reducing the workload of receiving process, improve the priority of receiving process and so on, so that it can receive data in a timely manner, so as to avoid the phenomenon of sticky packet; the third is controlled by the receiver, a packet of data by the structure of the field, the human control sub-multiple reception, and then merge, through this means to avoid

The above mentioned three kinds of measures, all have their shortcomings. Although the first method of programming can avoid the sticky packets caused by the sender, it shuts down the optimization algorithm, reduces the network sending efficiency, affects the performance of the application, and is generally not recommended for use. The second method can only reduce the likelihood of sticky packets, but does not completely avoid the sticky package, when the sending frequency is high, or because the network burst may make a time period packet arrives at the receiver faster, the receiver may still be too late to receive, resulting in sticky packets. The third approach avoids sticky packets, but the application is less efficient and unsuitable for real-time applications.

One of the most comprehensive countermeasures is that the receiver creates a preprocessing thread, and the received packets are preprocessed to separate the stuck packets. We have experimented with this method and proved to be efficient and feasible.

Third , programming and implementation

　　1. Implementation Framework

The Experiment Network communication program uses the TCP/IP protocol Socket API programming implementation. The socket is for the client/server model. The TCP implementation is shown in Framework 4.

Figure 4

　　2. Experimental hardware environment:

Server: Pentium 350 microcomputer

Client: Pentium 166 microcomputer

Network platform: A LAN connected by a 10-Gigabit shared hub

　　3. Lab Software Environment:

Operating system: Windows 98

Programming language: Visual C + + 5.0

　　4. Main thread

Programming in a multi-threaded way, the server has a total of two threads: Send data thread, send statistics display thread. There are three threads in the client: receiving a data thread, receiving a preprocessing sticky packet thread, receiving a statistics display thread. Where the send and receive thread priority is set to thread_priority_time_critical (highest priority), the preprocessing thread priority is thread_priority_above_normal (higher than the normal priority), The display thread priority is thread_priority_normal (normal priority).

The data structure that the experiment sends is shown in 5:

Figure 5

　　5. Sub-packet algorithm

According to the three kinds of different sticky-packet phenomena, the sub-package algorithm adopts the corresponding solution respectively. The basic idea is to first convert the received data stream (length set to m) to a predetermined structure data form, and remove the structure data length field, which is the N in Figure 5, and then the first packet data length according to n calculation.

1) If n<m, indicates that the data stream contains multiple packets of data, from its head intercept n bytes into the temporary buffer, the rest of the data continues to loop processing, until the end.

2) If n=m, it indicates that the data stream content is exactly a complete structure data, directly into the temporary buffer.

3) If the n>m, it indicates that the data stream content is not enough to form a complete structure of data, to be left with the next packet of data to be processed.

Interested in the specific content of the subcontracting algorithm and the software implementation, can contact the author.

Iv. Analysis of experimental results

　　The experimental results are as follows:

1. In the above experimental environment, when the sender of several packets of packet length of the sum of less than 1500b, often appear sticky packet phenomenon, the receiver after preprocessing thread processing can correctly solve the sticky package. If send no delay is set in the program: (SetSockOpt (Socket_name,ipproto_tcp,tcp_nodelay, (char *) &on,sizeof on), where on=1), there is no sticky-packet phenomenon.

2. When sending data to the 1kb~2kb of each packet, if the sending interval is less than 10ms, occasionally sticky packets will appear, and the receiving party can correctly unpack the packets that stick together after the preprocessing thread processing.

3. In order to determine the time of processing the adhesive packet, the sending party sequentially sends the length 1.5kb, 1.9kb, 1.2kb, 1.6kb, 1.0kb data, a total of 1000 packets. In order to make the sticky packet phenomenon, the receiving thread waits for 10ms before each receive, the receive buffer is set to 5000b, and the receiving party receives 526 packets of data, of which 175 packets are in length 5000b. The preprocessing thread processing can get 1000 packets of correct data, the total time of the adhesive packet processing is less than 1ms.

The experimental results show that the TCP sticky packet phenomenon exists, but it can be solved by the preprocessing of the receiver, and the processing time is very short (the 1000 packet data processing time is less than 1ms in the experiment), which hardly affects the normal operation of the application.

Transmission design of network TCP data (Sticky packet processing)