"Reprint" TCP time_wait detailed

Time_wait status

TCP ensures that all data is delivered correctly and in all possible cases.

When a socket connection is closed, the socket that is actively closed on one end enters the time_wait state, while the passive shutdown side is transferred to the closed state.

When a socket is closed, it is done by the four handshake process that sends the information between the two ends, and when close () is called, it means that there is no data to send at the end. It seems that after the handshake is complete, the socket should be closed closed state. But there are two problems,
First: We do not have any mechanism to guarantee that the last ACK will be served properly.
Second: There may still be residual packets (wandering duplicates, or old duplicate packets) on the network, and we must be able to handle them properly.

Assuming that the last ACK is lost, the server will resend the last fin it sends, so the client must maintain a status message so that the ACK can be re-sent, and if this state is not maintained, the client will respond to a RST after receiving the fin, which is considered an error after receiving the RST from the server side. If the TCP protocol can complete the necessary operation to terminate the traffic of the two sides, it is necessary to transmit the four sections of the four handshake completely correctly, and there must be no loss. This is why the socket is still in the TIME_WAIT state after it is closed, as he waits for the ACK to be re-sent.
If both sides of the current connected communication have called close (), assuming both sides have reached the closed state, and there is no time_wait state, the following situation occurs. Now that a new connection is established, the IP address and port used are exactly the same as the previous one, and the subsequent connection is called an avatar of the original connection. It is also assumed that there are datagrams in the original connection that are in the network, so that the new connection receives a datagram that is likely to be a previously connected datagram. To prevent this, TCP does not allow a connection to be established from a socket in the TIME_WAIT state. A socket in the TIME_WAIT state waits twice times the MSL time (the reason is twice times the MSL is because the MSL is a datagram in the network one-way to identify the lost time, a datagram may be sent in the diagram or its response to become a residual datagram, Confirming that a datagram and its response are dropped by twice times the MSL will change to the closed state. This means that a successful connection will inevitably result in the loss of the remaining datagrams in the previous network.
Due to the problem caused by the time_wait state, we can prevent the socket from entering the TIME_WAIT state by setting the SO_LINGER flag, which can replace the normal TCP four handshake termination mode by sending the RST. But this is not a very good idea, time_wait is often beneficial to us.

----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------

Q: What does so_reuseaddr mean when writing a Tcp/sock_stream service program?

A: This socket option notifies the kernel that if the port is busy but the TCP state is in time_wait, it can be reused
Port. If the port is busy and the TCP state is in a different state, the port is reused with an error message.
Indicates "address is already in use." If your service program stops and wants to restart immediately, the new socket still
The SO_REUSEADDR option is useful when using the same port. Must be aware that at this time any non-period
Data arrival can lead to confusion in the service process, but it is only possible, in fact, not
possible.

A socket consists of a related five-tuple, protocol, local address, local port, remote address, remote
Mouth. SO_REUSEADDR only means that local local addresses, local ports, and the entire related five-tuple can be reused
is still the only certainty. Therefore, after restarting the service program may receive non-expected data. Must be careful to make
Use the SO_REUSEADDR option.

Q: In client/server programming (Tcp/sock_stream), how to understand the TCP automaton time_wait
State?

A:w Richard Stevens <1999 died at the age of 49 >

Let me explain the status of Time_wait in <>
The explanations in section 2.6 are clear.

MSL (maximum segment lifetime) indicates the longest lifetime of TCP packets on the Internet, with each specific TCP implementation
Must select a certain MSL value. The RFC 1122 recommendation is 2 minutes, but the BSD legacy implementation takes 30 seconds.

TIME_WAIT state maximum hold Time is 2 * MSL, which is 1-4 minutes.

The IP header has a TTL with a maximum value of 255. Although the unit of TTL is not a second (fundamental and time independent), we still need to
Suppose that a TCP message with a TTL of 255 cannot survive on the internet for more than MSL.

TCP packets may be forced to buffer delay, select non-optimal path and so on during transmission, the result
The sender TCP mechanism begins a timeout retransmission. The previous TCP message could be called a "roaming TCP duplicate Message", and the latter
TCP messages can be referred to as "time-out retransmission of TCP duplicate messages", as a reliable connection-oriented protocol, the TCP implementation must
Handle this duplicate message correctly, as both may eventually arrive.

A typical TCP connection termination can be described in the following diagram:

Client Server
FIN M
Close (active off)-----------------> (passive off)

ACK m+1
<---------------------------------

FIN N
<-------------------------------CLOSED

ACK n+1
------------------------------->
Time_wait

Why do I need time_wait status?

Assuming that the final ACK is lost, the server will be re-issued fin,client must maintain TCP state information so that it can be re-sent
The final ACK, otherwise the RST is sent, resulting in the server thinking that an error occurred. TCP implementations must reliably terminate the connection
Two directions (full duplex off), the client must enter the TIME_WAIT state because the client may face
The situation where the final ACK is to be re-issued.

In addition, consider a situation in which the TCP implementation may face a two of the same related five tuples. If the previous connection
is in the TIME_WAIT state, allowing another connection with the same related five-tuple to appear, possibly handling
TCP messages, two connections interfere with each other. Use the SO_REUSEADDR option to consider this scenario.

Why does the time_wait state need to remain 2MSL for such a long time?

If the TIME_WAIT state hold time is not long enough (for example, less than 2MSL), the first connection terminates normally.
A second connection with the same related five-tuple appears, and the first concatenated duplicate message arrives, interfering with the second
A connection. The TCP implementation must prevent a duplicate packet of a connection from appearing after the connection is terminated, so let time_wait
The status hold time is long enough (2MSL) to connect the TCP packets in the corresponding direction either completely or
Discarded. When a second connection is established, it is not confused.

A: Small Four

Kernel parameters corresponding to time_wait state hold time under Solaris 7

# ndd-get/dev/tcp Tcp_time_wait_interval
240000
# ndd-set/dev/tcp Tcp_time_wait_interval 1000

The default setting is 240000ms, which is 4 minutes. If you modify this value with NDD, the minimum can be set to 1000ms,
That means 1 seconds. Obviously the kernel is limited and needs to be kernel Hacking.

# echo "Tcp_param_arr/w 0t0" | Adb-kw/dev/ksyms/dev/mem
Physmem 3b72
Tcp_param_arr:0x3e8 = 0x0
# ndd-set/dev/tcp Tcp_time_wait_interval 0

I don't know what disastrous consequences this is, see <> 's statement.

What are the disastrous consequences of a q:time_wait state hold time of 0? In the general application of reality, it seems
is the server unstable point, not necessarily have any disastrous consequences?

D: [email protected]

Linux Kernel source/usr/src/linux/include/net/tcp.h

#define Tcp_timewait_len (60*hz)/* How long to wait to successfully
* Close the socket, about seconds */

It is better not to change to 0, change to 1. Port allocations are allocated from the last assigned port number +1, so the general
There won't be any problems. The port allocation algorithm is tcp_v4_get_port in tcp_ipv4.c.

----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------
Time_wait status to HTTP effect

According to the TCP protocol, the party that initiates the shutdown will go into the time_wait state and Continue 2*MSL (Max Segment Lifetime) with a default of 240 seconds, which briefly explains why this state is needed in this post.

It is worth saying that for TCP-based HTTP protocol, the server side of the TCP connection is closed, so that the server side will enter the TIME_WAIT state, it can be imagined that for the large traffic of the Web server, there will be a large number of time_wait state, If the server receives 1000 requests in a second, the backlog of 240*1000=240,000 Time_wait records will be maintained, which can be a burden to the server. Of course, the modern operating system will use a fast lookup algorithm to manage these time_wait, so for the new TCP connection request, determine whether hit in a time_wait not too much time, but there are so many States to maintain is always bad.

HTTP protocol version 1.1 stipulates that the default behavior is keep-alive, that is, the reuse of TCP connections to transmit multiple request/response, one of the main reasons is to find this problem. There is also a way to slow down the time_wait pressure is to reduce the system's 2*MSL time, because 240 seconds of time is really a bit long, for Windows, modify the registry, in Hkey_local_machine\ system\ Currentcontrolset\services\ Tcpip\Parameters adds a DWORD type value TcpTimedWaitDelay, which is generally considered not to be less than 60, otherwise it may be troublesome.

For large services, one server is uncertain and a lb (Load Balancer) is required to allocate traffic to several back-end servers, which can be problematic if the LB is working in a NAT manner. If all the source addresses of the IP packets from lb to backend server are the same (the internal address of LB), then the TCP connection to the backend server is restricted because frequent TCP connections are established and shut down, leaving time_ on the server Wait state, and the remote address for these states is lb, the source port of LB is more than 60,000 (2^16=65536,1~1023 is a reserved port, and some other ports are not used by default). Once the port on each lb enters the server's time_wait blacklist, there is a 240-second no longer used to establish a connection to the server, so lb and server can support up to 300 connections. Without LB, this would not be the problem, because the server sees the remote address as a vast and expansive collection on the Internet, which is sufficient for each address,60000 multiple ports.

At first I felt that the use of LB would greatly limit the number of TCP connections, but the experiment shows that there is no such thing, lb behind a Windows Server 2003 per second processing requests reached 600, is the TIME_WAIT state does not play a role? After observing with net monitor and netstat, the connection between the server and LB's xxxx Port entered Time_wait state, then a SYN packet of the XXXX port of LB, the server received processing, but was dropped off as imagined. Book, from the pile of books to find covered in the dust of the university era of the "UNIX Network programming, Volume 1, Second edition:networking apis:sockets and XTI", in the middle of a sentence, for bsd- Derived implementation, as long as the SYN sequence number is larger than the last time the maximum sequence number is closed, then the TIME_WAIT state to accept the SYN, it is difficult to Windows also count bsd-derived? With this clue and keyword (BSD), find this post, in NT4.0, or bsd-derived, but Windows Server 2003 is already NT5.2, perhaps a little different.

Do a test, using the socket API to compile a client side, each bind to a local port such as 2345, a duplicate TCP connection to a server to send Keep-alive=false HTTP requests, The implementation of Windows keeps sequence number growing, so although the server maintains a time_wait state for the client's 2345 port connection, it is always able to accept new requests and not deny them. What if SYN's sequence number gets smaller? Also with the socket API, but this time with the raw IP, send a small sequence number of the SYN packet in the past, Net monitor inside See, this SYN was received by the server, such as mud ox like the sea, a little reaction, was dropped.

According to the book, bsd-derived and Windows Server 2003 have security implications, but at least there will be no time_wait blocking TCP requests, of course, the client should cooperate to ensure the sequence of different TCP connections Number must rise not to fall.

Article Links:
The TIME_WAIT state in the socket http://www.cnblogs.com/alon/archive/2009/04/01/1427625.html
TCP time_wait Status QA http://www.cnblogs.com/jason-jiang/archive/2006/11/03/549337.html
Time_wait status to HTTP effect http://morganchengmo.spaces.live.com/blog/cns!9950CE918939932E!1721.entry

Article turned from: http://blog.chinaunix.net/uid-20384806-id-1954363.html

"Reprint" TCP time_wait detailed