"Reprint" Socket of the half-pack, sticky bag and sub-package problems

http://zhaohuiopensource.iteye.com/blog/1541270

First look at two concepts:
Short connections:
Connection, transfer data, close connection
HTTP is stateless, and the browser and server make a connection every time an HTTP operation is made, but the end of the task interrupts the connection.
It is also possible to say that a short connection is when the socket connection is sent and the data is disconnected immediately after it is received.

Long connections:
Connect, transfer data--keep connected, transfer data ... , close the connection.
A long connection is when a socket connection is established, regardless of whether it is used or not, but is less secure.

The reason for the sticky and half-packet phenomenon is that there is only the concept of flow in TCP, without the concept of package.

Half Pack
The recipient does not receive a complete package, only accept the part, this situation is mainly due to TCP to improve transmission efficiency, the allocation of a packet is large enough, resulting in the receiving party can not be accepted at the end. (both long and short connections appear).

Sticky bags and sub-packages
A packet of packets sent by the sender to the receiver when it is received, and from the receiving buffer, the header of the packet data follows the end of the previous packet of data. There are many reasons for the sticky-packet phenomenon, which may be caused by the sender or by the receiver. The sticky packets caused by the sender are caused by the TCP protocol itself, and TCP is often needed to collect enough data to send a packet of data to improve transmission efficiency. If the data sent several times in a row is very small, usually TCP will be based on the optimization algorithm to synthesize the data packets sent out once, so that the receiver received the sticky packet data. The sticky packet caused by the receiver is due to the fact that the receiver user process does not receive the data in time, resulting in sticky packets. This is because the receiver first put the received data in the system receive buffer, the user process from the buffer to fetch data, if the next packet of data arrives before a packet of data has not been taken away by the user process, the next packet of data into the system receive buffer when the previous packet of data is received, The user process takes data from the system receive buffer based on the pre-set buffer size, so that it takes more than one packet of data at a time. Subcontracting refers to the sub-processing of our receivers when there is a sticky packet. (will appear in long connections)

when do I need to consider a half-pack case?
From the note we learned that the default socket internal and outgoing buffer size is about 8K, but in practice we often need to consider the efficiency problem, reconfigure this value, to achieve the best state of the system.
A practical example: use Mina as the server side, using a cache size of 10k, where a short connection is used, all without regard to the problem of sticky packets.
Problem Description: In the case of a large concurrency ratio, an acceptance will occur and no full data can be obtained.
Processing method:
1. Through the header + packet length + package of the Protocol form, when the server side gets to the specified length of the package only to obtain the complete.
2. Specify the end identity of the package so that when we get to the specified identity, the description package gets complete.


when do I need to consider the case of sticky bags?
1. In the case of short connection, do not consider the case of sticky bag
2. If the sending data is unstructured, such as file transfer, so that the sender only send, the receiver just receive storage is OK, do not consider sticky bag
3. If the two parties establish a connection, it is necessary to send different structural data within a period of time after connection
Processing method:
The receiver creates a preprocessing thread that is preprocessed by the received packets, separating the stuck packets
Note: There are two types of sticky bags, one of which is a package that is glued together is a complete packet, and the other case is that the package is stuck together with an incomplete package

Note:
A package does not have a fixed length, Ethernet limit of 46-1500 bytes, 1500 is the MTU of the Ethernet, over this amount, TCP will set an offset for IP datagrams for the Shard transfer, now generally allows the application layer to set the 8k (NTFS system) buffer, 8k of data from the underlying shards, And the application seems to be just one send. The experience value of Windows buffers is that the 4k,socket itself is divided into two types, stream (TCP) and datagram (UDP), and your problem is different for both types of use. It's even about programming with blocking or non-blocking sockets.
1, communication length, this is your own decision, no system forcing you to send how big package, the actual should be based on the needs and network conditions to decide. For TCP, this length can be a bit larger, but you know, the socket internal default send and receive buffer size is about 8K, you can use setsockopt to change. But for UDP, don't be too big, usually 1024 to 10K. Note that you no matter how big the package, the IP layer and the link layer will send your packet to be fragmented, the general LAN is about 1500, WAN is only dozens of bytes. The packet after the Shard will reach the receiver by different routes, and for UDP, if one of the shards is lost, then the receiver's IP layer will discard the entire sending packet, which forms a packet loss. Obviously, if a UDP packet like Blake Large, it is fragmented, the probability of the link layer lost shards like Blake Large, you this UDP packets, like Blake easy to lose, but too small and affect efficiency. It is best to configure this value to adjust to the optimal state according to the different environment.
The Send () function returns the actual length of the send, and in the case of the network, it will never return (send failed) errors, up to 0. For TCP You can write a loop of bytes to send. When the Send function returns SOCKET_ERROR, it marks an error. But for UDP, you do not write the loop to send, otherwise it will bring great trouble to your reception. So UDP needs to use setsockopt to change the size of the socket internal buffer to accommodate your contract. To be clear, TCP as a stream, the package will not be the whole packet arrives, but a steady flow of, that the receiver must pack. While UDP is a message or datagram, it must be the entire packet to the receiving party.
2, about the reception, the general contract has the package boundary, the first is you the length of the package to let the receiver know, so there is a Baotou information, for TCP, the receiver first receive this Baotou information, and then collect the packet data. It is also possible to collect the entire package at a time, and you can verify whether the results are received or not. This completes the package process as well. UDP, then you can only receive the entire packet. If the receive buffer you provide is too small, TCP will return the actual received length, the remaining can be received, and UDP is different, the remaining data is discarded and returned wsaemsgsize error. Note TCP, if you provide buffer like Blake Large, then you may receive multiple packages, you have to separate them, and when the buffer is too small, and at a time to receive the data inside the socket, then the socket receive event (OnReceive), may not be triggered again, Pay close attention to this when using event mode for reception. These features represent the difference between a stream and a packet.


Reference: http://176170847.iteye.com/blog/819446

"Reprint" Socket of the half-pack, sticky bag and sub-package problems