Several TCP/IP options in high-performance network programming in Linux

Http://blog.chinaunix.net/uid-20357359-id-1963666.html

Recently, I tested programs on the new platform. Some issues that have not been noticed before have become performance bottlenecks. Some problems can be solved by setting some TCP/IP options, of course, the fundamental solution is to refactor the code and redesign the Server framework. First, list several TCP/IP options:

Option
MAN 7 socket:
So_reuseaddr
So_recvbuf/so_sndbuf
So_keepalive
So_linger

MAN 7 TCP:
Tcp_cork
Tcp_nodelay
Tcp_defer_accept
Tcp_keepcnt/tcp_keepidle/tcp_keepintvl

So_reuseaddr
In server programs, the so_reuseaddr socket option is usually set before BIND () is called.
So_reuseaddr can be used in the following four cases.
(From volume 1 of UNIX network programming, unpv1)
1. When socket1 with the same local address and port is in the time_wait state, and socket2 of the program you start occupies the address and port, your program will use this option.
2. so_reuseaddr allows multiple instances (multiple processes) of the same server to be started on the same port ). However, the IP addresses bound to each instance cannot be the same. This situation can be tested on machines with multiple NICs or with IP alias technology.
3. so_reuseaddr allows a single process to bind the same port to multiple sockets, but each socket is bound with a different IP address. This is very similar to 2. for the difference, see unpv1.
4. so_reuseaddr allows repeated binding of identical addresses and ports. But this is only used for UDP multicast and not for TCP.

Tcp_nodelay/tcp_chork
Tcp_nodelay and tcp_cork basically control the "Nagle" of the package. The meaning of Nagle here is that the Nagle algorithm is used to assemble a smaller package into a larger frame. Both tcp_nodelay and tcp_cork disable the Nagle algorithm, but their behavior is different.
Tcp_nodelay does not use the Nagle algorithm. It does not splice a small packet into a large package before sending it. directly sending the small packet will make the user experience very good.
When transmitting a large amount of data, you can set tcp_cork to improve the efficiency of TCP transmission. As the name suggests, cork means "plug-in", which will try to send the maximum data volume each time. When tcp_cork is set, it will block 200 ms. After the blocking time, the data will be automatically transmitted.

For more information, see references 5.

So_linger
Linger, as the name implies, is the delay. Here, it is the delay of the connection-oriented socket close operation. By default, close returns immediately, but when there is still some data in the sending buffer, the system will try to send the data to the peer. So_linger can change the close action. Control so_linger through the following structure:
Struct linger
{
Int l_onoff;/* 0 = OFF, nonzero = on */
Int l_linger;/* Linger time, POSIX specifies units as seconds */
};

Different values of the members in the struct can be used in the following situations:
1. l_onoff is set to 0, and the option is disabled. The l_linger value is ignored, which is the default case above. Close is returned immediately.
2. When l_onoff is set to non-0 and l_linger is set to 0, close () is not blocked and executed immediately. The data in the buffer zone is discarded by the socket and an RST packet is sent to the peer end. This method is called "forced" or "invalid.
3. If l_onoff is set to non-0 and l_linger is set to non-0, close () calls the blocking process until all data has been sent or timed out. This type of closure is called an "elegant" closure.

Note:
This option needs to be used with caution, especially if it is forced to close, it will lose the last part of data sent from the server to the client. UNP:
The time_wait state is our friend and is there to help us (I. e., to let the old duplicate segments expire in the network ).

Tcp_defer_accept
Defer accept, which literally delays accept. In fact, a connection is created only after the first data is received. For non-interactive servers such as HTTP, this makes sense and can be used to defend against null connection attacks (only establish connections, but do not send any data ).
The usage is as follows:

Val = 5; Setsockopt (srv_socket-> FD, sol_tcp, tcp_defer_accept, & Val, sizeof (VAL ));

The unit of Val is second. Note that if this function is enabled, the kernel does not receive data within Val seconds and will not wake up the process, but directly discard the connection. If the server sets the tcp_defer_accept option and the server receives a CONNECT request, after three handshakes, the new socket status is still syn_recv instead of established, and the operating system does not accept.
Because after the tcp_defer_accept option is set, the status does not reach established after the three-way handshake, but syn_recv. At this time, if the client has not sent a "data" packet, the server will re-transmit the SYN/ACK packet. ipv4.tcp _ synack_retries parameter control. After the number of retransmissions is reached, the timeout value set in setsockopt will be performed again. Therefore, the syn_recv survival time is greater than the set value.

For more information about syn_recv status, see document 7.

So_keepalive/tcp_keepcnt/tcp_keepidle/tcp_keepintvl
If one party has closed or terminated the connection abnormally, but the other party does not know, we call this TCP connection semi-open. TCP uses keepalive to detect semi-open connections.
In highly concurrent network servers, the socket is often missed, and a large number of close_wait connections occur. At this time, you can set the keepalive option to solve this problem. Of course, there are other ways to solve this problem. For details, refer to reference 8.

The usage is as follows:

// Setting for keepalive Int keepalive = 1; Setsockopt (incomingsock, sol_socket, so_keepalive, (void *) (& keepalive), (socklen_t) sizeof (keepalive )); Int keepalive_time = 30; Setsockopt (incomingsock, ipproto_tcp, tcp_keepidle, (void *) (& keepalive_time), (socklen_t) sizeof (keepalive_time )); Int keepalive_intvl = 3; Setsockopt (incomingsock, ipproto_tcp, tcp_keepintvl, (void *) (& keepalive_intvl), (socklen_t) sizeof (keepalive_intvl )); Int keepalive_probes = 3; Setsockopt (incomingsock, ipproto_tcp, tcp_keepcnt, (void *) (& keepalive_probes), (socklen_t) sizeof (keepalive_probes ));

Set the so_keepalive option to enable keepalive, and then set the start time, interval, and number of keepalive parameters through tcp_keepidle, tcp_keepintvl, and tcp_keepcnt.
Of course, you can also set kernel parameters such as/proc/sys/NET/IPv4/tcp_keepalive_time, tcp_keepalive_intvl, and tcp_keepalive_probes. However, this will affect all sockets, therefore, we recommend that you use setsockopt.

Refer:
1. MAN 7 socket
2. MAN 7 TCP
3. UNIX network programming section 7.5 "generic socket options"
4. http://www.linuxsir.org/bbs/showthread.php? T = 55738
5. http://blog.chinaunix.net/u/12592/showart_1723934.html
6. http://blog.csdn.net/fullsail/archive/2009/08/09/4429102.aspx
7. http://www.kernelchina.org /? Q = node/110
8. http://blog.chinaunix.net/u/12592/showart.php? Id = 2059174