[go] about socket sticky packet problem

these two days to see Csdn have some questions about the socket sticky packet, socket buffer settings, found that they are not very clear, so check the information to understand the record:

One or two simple concepts long connections with short connections:
1. Long connection

The client and the server first establish a communication connection, after the connection is established, and then the message is sent and received.

2. Short connection

The client side communicates with the server every time the message is sent and received, and disconnects immediately after the transaction is completed. This approach is often used in a point-to-multipoint
Communications, such as multiple client connections to a server.

When do I need to consider the problem of sticky bag?

1: If you use TCP to send data each time, you establish a connection with each other, and then after the two sides send a piece of data, then close the connection, so that there is no sticky packet problem (because there is only one package structure, similar to the HTTP protocol). Close connections are mostly sent by both parties to the close connection (refer to TCP shutdown protocol). Such as: a need to send a string to B, then A and B to establish a connection, and then send both sides of the default good protocol word such as "Hello give me sth abour yourself", and then B receives the message, then the buffer data is received, and then close the connection, so the sticky packet problem is not considered, Because everyone knows to send a certain character.
2: If send data unstructured, such as file transfer, so that the sender just send, the receiver only receive storage OK, do not consider sticky bag
3: If the two sides establish a connection, it is necessary to send different structure data after the connection, such as after the connection, there are several kinds of structure:
1) "Hello give me sth abour yourself"
2) "Don ' t give me sth abour yourself"
That way, if the sender sends this two packets in succession, the receiving party may be "Hello give me sth abour yourselfdon ' t give me sth abour yourself" so the receiver is silly, what is it going to do? Do not know, because the protocol does not stipulate such a strange string, so to deal with it sub-package, how to divide also need to organize a better package structure, it is generally possible to add a packet of data length, such as the header to ensure the reception.

three sticky packet causes: occurs in the stream loss, UDP does not appear sticky packet, because it has a message boundary (refer to Windows network programming)
1 The sending side needs to wait for the buffer to be full before sending out, resulting in sticky packets
2 The receiver does not receive the buffer packet in time, resulting in multiple packets receiving

Workaround:
In order to avoid sticking, the following measures can be taken. One is caused by the sender of the sticky packet phenomenon, the user can be programmed to avoid, TCP provides a mandatory data transfer immediately after the operation of the instructions PUSH,TCP software received the operation instruction, the data immediately sent out, without waiting for the transmission buffer full, and the receiver caused by the sticky packet, By optimizing the program design, reducing the workload of receiving process, improve the priority of receiving process and so on, so that it can receive data in a timely manner, so as to avoid the phenomenon of sticky packet; the third is controlled by the receiver, a packet of data by the structure of the field, the human control sub-multiple reception, and then merge, through this means to avoid

The above mentioned three kinds of measures, all have their shortcomings. Although the first method of programming can avoid the sticky packets caused by the sender, it shuts down the optimization algorithm, reduces the network sending efficiency, affects the performance of the application, and is generally not recommended for use. The second method can only reduce the likelihood of sticky packets, but does not completely avoid the sticky package, when the sending frequency is high, or because the network burst may make a time period packet arrives at the receiver faster, the receiver may still be too late to receive, resulting in sticky packets. The third approach avoids sticky packets, but the application is less efficient and unsuitable for real-time applications.

Related articles interception:

A package does not have a fixed length, Ethernet limit of 46-1500 bytes, 1500 is the MTU of the Ethernet, over this amount, TCP will set an offset for IP datagrams for the Shard transfer, now generally allows the application layer to set the 8k (NTFS system) buffer, 8k of data from the underlying shards, And the application seems to be just one send. The experience value of Windows buffers is that 4k,socket itself is divided into two types, stream (TCP) and datagram (UDP), and your problem is different for both of these uses

Sample. It's even about programming with blocking or non-blocking sockets.

1, communication length, this is your own decision, no system forcing you to send how big package, the actual should be based on the needs and network conditions to decide. For TCP, this length can be a bit larger, but you know, the socket internal default send and receive buffer size is about 8K, you can use setsockopt to change. But for UDP, don't be too big, usually 1024 to 10K. Note that you no matter how big the package, the IP layer and the link layer will send your packet to be fragmented, the general LAN is about 1500, WAN is only dozens of bytes. The packet after the Shard will reach the receiver by different routes, and for UDP, if one of the shards is lost, then the receiver's IP layer will discard the entire sending packet, which forms a packet loss. Obviously, if a UDP packet like Blake Large, it is fragmented, the probability of the link layer lost shards like Blake Large, you this UDP packets, like Blake easy to lose, but too small and affect efficiency. It is best to configure this value to adjust to the optimal state according to the different environment.

The Send () function returns the actual length of the send, and in the case of the network, it will never return (send failed) errors, up to 0. For TCP You can write a loop of bytes to send. When the Send function returns SOCKET_ERROR, it marks an error. But for UDP, you do not write the loop to send, otherwise it will bring great trouble to your reception. So UDP needs to use setsockopt to change the size of the socket internal buffer to accommodate your contract. To be clear, TCP as a stream, the package will not be the whole packet arrives, but a steady flow of, that the receiver must pack. While UDP is a message or datagram, it must be the entire packet to the receiving party.

2, about the reception, the general contract has the package boundary, the first is you the length of the package to let the receiver know, so there is a Baotou information, for TCP, the receiver first receive this Baotou information, and then collect the packet data. It is also possible to collect the entire package at a time, and you can verify whether the results are received or not. This completes the package process as well. UDP, then you can only receive the entire packet. If the receive buffer you provide is too small, TCP will return the actual received length, the remaining can be received, and UDP is different, the remaining data is discarded and returned wsaemsgsize error. Note TCP, if you provide buffer like Blake Large, then you may receive multiple packages, you have to separate them, and when the buffer is too small, and at a time to receive the data inside the socket, then the socket receive event (OnReceive), may not be triggered again, Pay close attention to this when using event mode for reception. These features represent the difference between a stream and a packet.

Related Reference articles:
Troubleshoot TCP network Transport "Sticky pack" issues

Currently in the network transmission application, the TCP/IP communication protocol and its standard socket application Development Programming Interface (API) are widely adopted. The TCP/IP transport layer has two parallel protocols: TCP and UDP. TCP (Transport Control Protocol, transmission Protocol) is connection-oriented and provides high reliability services. UDP (User Datagram Protocol, Subscriber Datagram Protocol) is non-connected and provides efficient service. In practical engineering applications, the choice of reliability and efficiency depends on the environment and requirements of the application. In general, the network transmission of ordinary data uses high-efficiency UDP, and the network transmission of important data uses TCP with higher reliability.

In the application development process, I found that applications based on TCP network transmission sometimes appear sticky packet phenomenon (that is, the sender sends a number of packets of data to the receiving party to stick to a packet). In view of this situation, we have carried on the special research and the experiment. In this paper, the problem of TCP network sticky packet is analyzed emphatically, and the countermeasures and methods for solving this problem are put forward, which are used for reference by engineering technicians.

I. Introduction to the TCP protocol

TCP is a connection-oriented transport layer protocol, although TCP does not belong to the ISO set of protocols, but because of its successful application in the business and industry, it has become a de facto network standard, widely used in various network host communication.

As a connection-oriented transport layer protocol, the goal of TCP is to provide users with reliable end-to-end connections to ensure the orderly and accurate transmission of information. In addition to providing basic data transmission functions, it also adopts a series of measures such as data number, checksum calculation and data confirmation to ensure reliability. It numbers each byte of data that is transmitted and requests the receiver to return a confirmation message (ACK). If the sender does not receive the data confirmation within the specified time, it will retransmit the data. The data number allows the receiver to handle the disorder and repetition of the data. The data error problem is solved by adding a checksum to each transmitted data segment, and the receiver checks the checksum after receiving the data, and if the checksum is incorrect, the data segment of the error code is discarded and the sender is required to retransmit it. Flow control is also an important measure to ensure the reliability, if no flow control, may be due to receive buffer overflow and loss of large amounts of data, resulting in a lot of retransmission, causing network congestion vicious circle. TCP uses a variable window for flow control, and the receiver controls the amount of data sent by the sender.

TCP provides users with high-reliability network transmission services, but reliability assurance measures also affect the transmission efficiency. Therefore, in the actual engineering application, only the transmission of the key data is TCP, and the transmission of common data generally uses the high efficiency UDP.

Analysis on the problem of sticky bag and its countermeasures

A TCP sticky packet is a packet of packets sent by the sender to the receiver when it is received, viewed from the receive buffer, followed by the head of the packet data immediately preceding the end of the packet.

There are many reasons for the sticky-packet phenomenon, which may be caused by the sender or by the receiver. The sticky packets caused by the sender are caused by the TCP protocol itself, and TCP is often needed to collect enough data to send a packet of data to improve transmission efficiency. If the data sent several times in a row is very small, usually TCP will be based on the optimization algorithm to synthesize the data packets sent out once, so that the receiver received the sticky packet data. The sticky packet caused by the receiver is due to the fact that the receiver user process does not receive the data in time, resulting in sticky packets. This is because the receiver first put the received data in the system receive buffer, the user process from the buffer to fetch data, if the next packet of data arrives before a packet of data has not been taken away by the user process, the next packet of data into the system receive buffer when the previous packet of data is received, The user process takes the data from the system receive buffer based on the pre-set buffer size, so that it takes more than one packet of data at a time (as shown in Figure 1).

Figure 1

Figure 2


Figure 3

There are two kinds of sticky packets, one of which is glued together is the complete packet (Figure 1, Figure 2), the other case is stuck together with the package has an incomplete package (Figure 3), this assumes that the user receive buffer length of M bytes.

Not all sticky-packet phenomena need to be processed, if the transmitted data is a continuous stream of data without structure (such as file transfer), you do not have to separate the packets of adhesion (short-subcontracting). But in the actual engineering application, the transmitted data is usually the structure data, then needs to do the subcontract processing.

When dealing with the sticky-packet problem of fixed-length structure data, the sub-packet algorithm is simple, and the sub-packet algorithm is more complicated when it deals with the sticky-packet problem of uncertain long structure data. Especially in the case of 3, as a packet of data content is divided into two successive receiving packets, the processing is more difficult. In practical engineering application, the phenomenon of sticking should be avoided as far as possible.

In order to avoid sticking, the following measures can be taken. One is caused by the sender of the sticky packet phenomenon, the user can be programmed to avoid, TCP provides a mandatory data transfer immediately after the operation of the instructions PUSH,TCP software received the operation instruction, the data immediately sent out, without waiting for the transmission buffer full, and the receiver caused by the sticky packet, By optimizing the program design, reducing the workload of receiving process, improve the priority of receiving process and so on, so that it can receive data in a timely manner, so as to avoid the phenomenon of sticky packet; the third is controlled by the receiver, a packet of data by the structure of the field, the human control sub-multiple reception, and then merge, through this means to avoid

The above mentioned three kinds of measures, all have their shortcomings. Although the first method of programming can avoid the sticky packets caused by the sender, it shuts down the optimization algorithm, reduces the network sending efficiency, affects the performance of the application, and is generally not recommended for use. The second method can only reduce the likelihood of sticky packets, but does not completely avoid the sticky package, when the sending frequency is high, or because the network burst may make a time period packet arrives at the receiver faster, the receiver may still be too late to receive, resulting in sticky packets. The third approach avoids sticky packets, but the application is less efficient and unsuitable for real-time applications.

One of the most comprehensive countermeasures is that the receiver creates a preprocessing thread, and the received packets are preprocessed to separate the stuck packets. We have experimented with this method and proved to be efficient and feasible.

Third , programming and implementation

　　1. Implementation Framework

The Experiment Network communication program uses the TCP/IP protocol Socket API programming implementation. The socket is for the client/server model. The TCP implementation is shown in Framework 4.

Figure 4

　　2. Experimental hardware environment:

Server: Pentium 350 microcomputer

Client: Pentium 166 microcomputer

Network platform: A LAN connected by a 10-Gigabit shared hub

　　3. Lab Software Environment:

Operating system: Windows 98

Programming language: Visual C + + 5.0

　　4. Main thread

Programming in a multi-threaded way, the server has a total of two threads: Send data thread, send statistics display thread. There are three threads in the client: receiving a data thread, receiving a preprocessing sticky packet thread, receiving a statistics display thread. Where the send and receive thread priority is set to thread_priority_time_critical (highest priority), the preprocessing thread priority is thread_priority_above_normal (higher than the normal priority), The display thread priority is thread_priority_normal (normal priority).

The data structure that the experiment sends is shown in 5:

Figure 5

　　5. Sub-packet algorithm

According to the three kinds of different sticky-packet phenomena, the sub-package algorithm adopts the corresponding solution respectively. The basic idea is to first convert the received data stream (length set to m) to a predetermined structure data form, and remove the structure data length field, which is the N in Figure 5, and then the first packet data length according to n calculation.

1) If n<m, indicates that the data stream contains multiple packets of data, from its head intercept n bytes into the temporary buffer, the rest of the data continues to loop processing, until the end.

2) If n=m, it indicates that the data stream content is exactly a complete structure data, directly into the temporary buffer.

3) If the n>m, it indicates that the data stream content is not enough to form a complete structure of data, to be left with the next packet of data to be processed.

Interested in the specific content of the subcontracting algorithm and the software implementation, can contact the author.

Iv. Analysis of experimental results

　　The experimental results are as follows:

1. In the above experimental environment, when the sender of several packets of packet length of the sum of less than 1500b, often appear sticky packet phenomenon, the receiver after preprocessing thread processing can correctly solve the sticky package. If send no delay is set in the program: (SetSockOpt (Socket_name,ipproto_tcp,tcp_nodelay, (char *) &on,sizeof on), where on=1), there is no sticky-packet phenomenon.

2. When sending data to the 1kb~2kb of each packet, if the sending interval is less than 10ms, occasionally sticky packets will appear, and the receiving party can correctly unpack the packets that stick together after the preprocessing thread processing.

3. In order to determine the time of processing the adhesive packet, the sending party sequentially sends the length 1.5kb, 1.9kb, 1.2kb, 1.6kb, 1.0kb data, a total of 1000 packets. In order to make the sticky packet phenomenon, the receiving thread waits for 10ms before each receive, the receive buffer is set to 5000b, and the receiving party receives 526 packets of data, of which 175 packets are in length 5000b. The preprocessing thread processing can get 1000 packets of correct data, the total time of the adhesive packet processing is less than 1ms.

The experimental results show that the TCP sticky packet phenomenon exists, but it can be solved by the preprocessing of the receiver, and the processing time is very short (the 1000 packet data processing time is less than 1ms in the experiment), which hardly affects the normal operation of the application.

[go] about socket sticky packet problem