Research on the send function of TCP

The TCP protocol itself is reliable and does not mean that the application can send data over TCP. No matter whether it is blocked or not, the size of the send does not represent the amount of data from the peer Recv.

In blocking mode, the send function copies the data sent by application requests to the sending cache for sending, and returns the data after confirmation. however, the existence of the sending cache shows that if the size of the sending cache is greater than that of the request, the send function returns immediately and sends data to the network. Otherwise, send sends the part of data that cannot be contained in the cache to the network, and waits for confirmation from the peer end before returning the data. (the receiving end only needs to receive the data in the cache and confirms it, does not have to wait for the application to call Recv );

In non-blocking mode, the send function only copies data to the cache area of the protocol stack. If the available space in the cache area is insufficient, you can do your best to copy the data, the size of the successful copy is returned. If the available space in the cache area is 0,-1 is returned, and errno is set to eagain.

In Linux, use sysctl-A | grep net. ipv4.tcp _ WMEM to view the default sending cache size:
Net. ipv4.tcp _ WMEM = 4096 16384 81920
There are three values. The first value is the minimum number of bytes allocated in the socket sending cache, and the second value is the default value (this value will be net. core. wmem_default overwrite). The cache can be increased to this value when the system load is not heavy. The third value is the maximum number of bytes in the sending cache space (this value will be net. core. wmem_max overwrite ).
According to the actual test, if the net. net. the value of ipv4.tcp _ WMEM is run according to the changed value. Otherwise, the protocol stack is usually based on net. core. wmem_default and net. core. the value of wmem_max to allocate memory.

The application should change the sending cache size in the program according to the application features:

Socklen_t sendbuflen = 0;
Socklen_t Len = sizeof (sendbuflen );
Getsockopt (clientsocket, sol_socket, so_sndbuf, (void *) & sendbuflen, & Len );
Printf ("default, sendbuf: % d \ n", sendbuflen );

Sendbuflen = 10240;
Setsockopt (clientsocket, sol_socket, so_sndbuf, (void *) & sendbuflen, Len );
Getsockopt (clientsocket, sol_socket, so_sndbuf, (void *) & sendbuflen, & Len );
Printf ("now, sendbuf: % d \ n", sendbuflen );

It should be noted that, although the sending cache is set to 10 K, the protocol stack actually doubles it to 20 K.
----------------- Instance analysis ---------------

In practical applications, if the sending end is not blocked, the sending application may seem to have sent 10 KB of data due to network congestion or slow processing at the receiving end, but only 2 k is sent to the peer cache, and 8 K is in the local cache (not sent or not confirmed by the receiver ). at this time, the data received by the receiving application is 2 K. if the receiving application calls the Recv function to obtain 1 k of data for processing, one of the following occurs at this moment:

A. The sending application considers that 10 K of data has been sent and socket has been disabled:
As the active listener of TCP, the sending host will be in the semi-closed status of fin_wait1 (waiting for the other's ACK), and the 8 K data in the sending cache will not be cleared, it will still be sent to the peer end. if the receiving application is still in Recv, it will receive the remaining 8 K data (the receiving end will receive the remaining 8 K data before the fin_wait1 status of the sending end times out .), then we get a message that the Peer socket is closed (Recv returns 0 ). in this case, close it.

B. The sending application calls send again to send 8 K data:
If the size of the sent cache is 20 KB, the available size of the sent cache is 20-8 = 12 kb, which is greater than the size of 8 KB. Therefore, after the send function copies the data, and returns 8192 immediately;

If the size of the sent cache is 12 K, the available size of the sent cache is 12-8 = 4 K, and send () returns 4096, after the application finds that the returned value is smaller than the size value sent by the request, it can be considered that the cache zone is full and must be blocked (or wait for the next socket writable signal through select ), if the application ignores the request and immediately calls send again, the value of-1 is returned, Which is errno = eagain in Linux.

C. After the application processes 1 k of data, the socket is disabled:
The receiving host is active closed, and the connection will be in the semi-closed status of fin_wait1 (waiting for the other's ACK ). then, the sending application will receive the socket readable signal (usually the Select call returns the socket readable), but the Recv function returns 0 upon reading, in this case, the close function should be called to close the socket (send to the other party ACK );

If the sending application does not process this readable signal, but sends it, there are two cases to consider. If sending is called after the sender receives the RST mark, send returns-1, and errno is set to econnreset, indicating that the peer network is disconnected. However, it is also said that the process will receive the sigpipe signal, and the default response of this signal is to exit the process, if this signal is ignored, send returns-1, errno is epipe (unconfirmed), and send works as usual before the sender receives the RST mark;

The above is a non-blocking send case. If send is a blocking call and is in a blocking situation (for example, sending a large Buf at a time, beyond the sending cache), the peer socket is closed, send returns the number of successfully sent bytes. If you call send again, it will be the same as above.

D. switch or router network disconnection:
After processing 1 k of data received by the receiving application, it will continue to read the remaining 1 K data from the cache, and then it will become non-readable, in this case, the application needs to handle the timeout. the general practice is to set the maximum waiting time for a select statement. If no data is readable after this time, the socket is considered unavailable.

The sending application will continuously send the remaining data to the network, but it will never be confirmed. Therefore, the available space in the cache area remains 0. In this case, the application needs to process the data.

If the application cannot handle the timeout, you can also use the TCP protocol itself. For details, refer to the following in sysctl:
Net. ipv4.tcp _ keepalive_intvl
Net. ipv4.tcp _ keepalive_probes
Net. ipv4.tcp _ keepalive_time