TCP/IP Connection status

1. Establish Connection Agreement (three handshake)
(1) The client sends a TCP message with a SYN flag to the server. This is the message 1 in the three-time handshake process.
(2) server-side response to the client, this is the 2nd message in the three handshake, the message with both an ACK flag and a SYN flag. So it represents the response to the client's SYN message, and it also flags the SYN to the client and asks the client if it is ready for data communication.
(3) The customer must again respond to the service segment an ACK message, which is the message segment 3.
2. Connection termination protocol (four handshake)
Because TCP connections are full-duplex, each direction must be closed separately. The principle is that when a party completes its data sending task, it can send a fin to terminate the connection in this direction. Receiving a fin only means there is no data flow in this direction, and a TCP connection can still send data after receiving a fin. The first party to close will perform an active shutdown, while the other side performs a passive shutdown.
(1) The TCP client sends a fin to shut down the client-to-server data transfer (message segment 4).
(2) The server receives this fin, it sends back an ACK, confirms that the serial number is the received sequence number plus 1 (message Segment 5). As with Syn, a fin will occupy a sequence number.
(3) The server shuts down the client connection and sends a FIN to the client (message segment 6).
(4) The customer segment sends back ACK message confirmation, and the confirmation serial number is set to receive the serial number plus 1 (message segment 7).
CLOSED: This is nothing to say, it means the initial state.
LISTEN: This is also very easy to understand a state, indicating that a server side of a socket in the brain to kill the gold Beast listening State, can accept the connection.
SYN_RCVD: This status means that the SYN message is received, under normal circumstances, this state is the server side of the socket in the establishment of a TCP connection during the three handshake session process of an intermediate state, very short, basically with netstat you are very difficult to see this state, Unless you specifically write a client-side test program, deliberately not send the last ACK message during the three TCP handshake. Therefore, when the ACK message is received from the client, it goes into the established state.
Syn_sent: This state is echoed with the syn_rcvd thinking back, when the client socket performs a connect connection, it sends the SYN message first, so it then enters the syn_sent state and waits for the server to send the 2nd message in the three-time handshake. The Syn_sent status indicates that the client has sent a SYN message.
Established: This is easy to understand, indicating that the connection has been established.
Fin_wait_1: This state should be well explained, in fact, the real meaning of fin_wait_1 and fin_wait_2 state is to wait for each other's fin message. The difference between the two states is: The fin_wait_1 state is actually when the socket in the established state, it would like to actively close the connection, send a FIN message to the other side, when the socket is entered into the fin_wait_1 state. And when the other party responds to the ACK message, then into the fin_wait_2 state, of course, under the actual normal circumstances, regardless of the circumstances of each other, should immediately respond to the ACK message, so fin_wait_1 state is generally more difficult to see, and Fin_wait_ 2 states can also sometimes be seen with netstat.
Fin_wait_2: Above has explained in detail this state, actually fin_wait_2 the socket in the state, indicates the half connection, also namely has the party request close connection, but also tells the other side, I temporarily also some data need to transmit to you, later closes the connection again.
Time_wait: Said to receive the other side of the fin message, and sent out an ACK message, just wait for 2MSL to return to the closed usable state. If the fin_wait_1 state, received the other side with the FIN flag and the ACK flag message, you can directly into the time_wait state, without having to go through the fin_wait_2 state.
CLOSING: This kind of state is special, the actual situation should be very rare, belong to a relatively rare exception state. Normally, when you send a fin message, it is supposed to receive (or receive) the other's ACK message before receiving the other's fin message. But closing status indicates that you send fin message, and did not receive the other's ACK message, but also received the other side of the fin message. Under what circumstances will this happen? In fact, it is not difficult to come to a conclusion: that is, if the two sides close a socket at the same time, then there is a situation where both sides send the fin message, there will be a closing state, indicating that both sides are shutting down the socket connection.
Close_wait: The meaning of this state is actually expressed in waiting to be closed. How do you understand it? When the other side close a socket to send fin message to yourself, you will undoubtedly respond to an ACK message to each other, then enter into the close_wait state. Next, the real thing you really need to consider is whether you still have the data sent to the other person, if not, then you can close the socket, send fin messages to each other, that is, close the connection. So what you need to accomplish in the close_wait state is waiting for you to close the connection.
Last_ack: This state is still relatively easy to understand, it is the passive close side after sending fin messages, and finally wait for each other's ACK message. When an ACK message is received, it is also possible to enter the closed available state.
Finally there are 2 questions answered, my own analysis of the conclusion (not necessarily guaranteed 100% correct)
1, why to establish a connection agreement is three handshake, and close the connection is four handshake it?
This is because the socket in the listen state of the server is sent in a message after it receives a request for the connection of the SYN message, and it can put the ACK and SYN (ACK response, and SYN synchronous). However, when the connection is closed, when receiving the other's fin message notification, it simply means that no data is sent to you, but not all of your data are sent to the other side, so you may not immediately close the socket, that is, you may also need to send some data to the other side, Send the FIN message to the other side to show that you agree that you can now close the connection, so it is here that the ACK message and fin messages are sent separately in most cases.
2, why time_wait state also need to wait 2MSL to return to closed status?
This is because: although both sides agree to close the connection, and the handshake of the 4 messages are also coordinated and sent, can be directly back to the closed state (like from the Syn_send state to establish state); but because we have to assume that the network is unreliable, You cannot guarantee that the last ACK message you send will be received by the other party, so the socket in the Last_ack state may be re-sending the fin message because the timeout does not receive an ACK message, so this time_wait state is used to resend the possible missing ACK message.

TCP is a connection-oriented protocol, so it is necessary to first establish a connection before sending data to both parties. This is completely different from the protocol mentioned earlier. All the protocols mentioned above are just sending data, most of them do not care about sending the data is sent to, UDP is especially obvious, from the programming point of view, the UDP programming is also much simpler----UDP do not consider data sharding.

In the book with Telnet login to explain the TCP protocol connection establishment and abort process, you can see that the establishment of a TCP connection can be simply called three handshake , and the termination of the connection can be called four times handshake .

1. Establishment of the connection

When the connection is established, the client first requests to the server to open a port (with a SYN segment equal to 1 TCP packets), and then the server sends back an ACK message to notify the client request message received, the client received a confirmation message after the confirmation message confirmed that just the server-side sent confirmation message (around the mouth) , at this point, the establishment of the connection is complete. This is called a three-time handshake. If you plan on getting both sides ready, you must send three messages, and only three messages are required.

As you can imagine, if you add a TCP time-out retransmission mechanism, then TCP can ensure that a packet is sent to the destination.

2. End Connection

TCP has a special concept called Half-close, which means that TCP connections are full-duplex (can be sent and received simultaneously), so when you close the connection, you must close the transmission and send the two-direction connection. The client gives the server a fin of 1 TCP message, then the server returns a confirmation ACK message to the client, and sends a FIN message, when the client replies to the ACK message (four handshake), the connection is over.

3. Maximum message length

At the time of establishing the connection, the two sides of the communication must confirm each other's maximum message length (MSS) in order to communicate. This SYN length is generally the MTU minus the fixed IP header and the TCP Head ministerial degree. For an Ethernet, it can generally reach 1460 bytes. Of course, for non-native IP, this MSS may be only 536 bytes, and if the intermediate transmission network MSS better small, this value will become smaller.

Status migration diagram for 4.TCP

The book P182 page gives a status graph of TCP, which is a more complex state migration diagram, because it contains two parts---server State Migration and client State migration, if from a certain point of view this diagram, it is clear that the server and the client is not absolute, The data is sent to the client, accepting the data is the server.

4.1. State migration diagram for client applications

The state of the client can be represented by the following process:

Closed->syn_sent->established->fin_wait_1->fin_wait_2->time_wait->closed

The above process is in the normal situation of the program should have a process, from the diagram in the book can be seen, when the connection is established, when the client receives the ACK of the SYN message, the client opens the data to connect interactively. The end-of-connection is usually the client's active end, and after the client finishes the application, it needs to go through the state of Fin_wait_1,fin_wait_2, which is the four-time handshake of the end connection mentioned earlier.

4.2. State migration diagram of the server

The status of the server can be represented by the following process:

Closed->listen->syn received->established->close_wait->last_ack->closed

When the connection is established, the server side enters the data interaction state after the third handshake, and the close connection is after the second handshake of the connection is closed (note that it is not the fourth time). After closing, it waits for the client to give the final ACK packet in order to enter the initial state.

4.3. Other State migrations

The diagram in the book also has some other state migrations, which are summarized for both the server and the client:

1. Listen->syn_sent, for this explanation is very simple, the server sometimes to open the connection.
2. Syn_sent->syn received, the server and the client in the Syn_sent state if received SYN datagram, you need to send the SYN ACK datagram and adjust its state to the SYN received state, ready to enter the established
3. Syn_sent->closed, in the case of a send timeout, is returned to the CLOSED state.
4. Syn_ receives->listen, if received by the RST packet, it returns to the LISTEN state.
5. Syn_ received->fin_wait_1, this migration is said, can not go to established state, and can jump directly to the fin_wait_1 state and wait to close.

4.4.2MSL Wait Status

The picture given in the book, there is a time_wait wait state, this state is called the 2MSL state, that is, after the time_wait2 sent the last ACK datagram, to enter the TIME_WAIT state, This state is prepared to prevent the last handshake from being delivered to the other party (note that this is not a four-time handshake, which is the fourth time that the handshake is insured). This state is to a large extent guaranteed that both sides can end normally, but the problem also comes.

Due to the 2MSL status of the socket (socket is the meaning of IP and port pairs, socket), so that the application in 2MSL time is unable to use the same socket again, for the client program is good, but for the service program, such as httpd, it always use the same port to service, In 2MSL time, the boot httpd will be error (socket is used). To avoid this error, the server gives the concept of a quiet time, which means that in 2MSL time, although the server can be restarted, but the server will still wait for 2MSL time in the past before the next connection.

4.5.fin_wait_2 status

This is known as the semi-closed state, which is the state after the client and server two handshake when the connection is closed. In this state, the application also has the ability to accept data, but it has been unable to send data, but it is also possible that the client has been in a fin_wait_2 state, and the server has been in the Wait_close state, and until the application layer decides to close the state.

5.RST, simultaneously open and close simultaneously

The RST is another way to close the connection, and the application should be able to determine the authenticity of the RST package, that is, whether the exception was aborted. While opening and closing at the same time are two special TCP states, the probability of occurrence is very small.

6.TCP Server Design

The previous talk about UDP server design, you can find that the UDP server completely does not need the so-called concurrency mechanism, it just set up a data input queue can be. But unlike TCP, TCP servers need to establish a separate process (or lightweight, thread) for each connection to ensure the independence of the conversation. Therefore, the TCP server is concurrent. And TCP also needs to be equipped with an incoming connection request queue (also not required by the UDP server) to establish a conversation process for each connection request, which is why the various TCP servers have a maximum number of connections. Depending on the IP and port number of the source host, the server can easily distinguish between different sessions for data distribution.

Execute the following command:

#netstat-N | awk '/^tcp/{++state[$NF]} END {for (key in) print key. " \ t ". State[key]} '

You will get a result similar to the following, the exact number will be different:

Fin_wait_1 286

Fin_wait_2 960

Syn_sent 3

Last_ack 32

CLOSING 1

CLOSED 36

SYN_RCVD 144

Time_wait 2520

Established 352 #差不多等于连接的并发数

This command categorizes the current system's network connection status.

This statement was seen at the feast, and it was said to have been obtained from Wang Lu, the technical director of the Sina Interactive Community division, very well.

The description of the return parameter is as follows:

SYN_RECV indicates the number of requests waiting to be processed;

Established indicates the normal data transmission status;

Time_wait indicates the number of requests that have finished processing and waiting for the timeout to expire.

------------------------------------------------------------------
Let's take a look at awk:
/^tcp/
Filter out the beginning of the TCP record, blocking UDP, socket and other unrelated records.
State[]
Equivalent to defining an array named state
Nf
Indicates the number of recorded fields, as shown above, NF equals 6
$NF
Represents the value of a field, such as the record shown above, $NF that is $6, which represents the value of the 6th field, which is time_wait
state[$NF]
The number of connections representing the values of the array elements, such as the record shown above, which is the state[time_wait] state
++state[$NF]
To add a number to a record, as shown above, is to add a state[time_wait] state to the number of connections
END
Represents the command to be executed in the final phase
For (key in state)
Iterating through an array
Print key, "\ T", State[key]
Print the array of keys and values, in the middle with a \ t tab split, beautify a bit.

Close_wait state Causes and treatment:

In the case of a passive shutdown connection, the connection is in the close_wait state when the fin has been received but has not yet sent its own fin.
Generally speaking, the duration of the close_wait state should be very short, just as the SYN_RCVD state. However, in some special cases, there is a situation where the connection is close_wait for a long time.
There is a large number of close_wait phenomenon, the main reason is that in some cases the other side closed the socket link, but we are busy with reading or writing, did not close the connection.

Close_wait will last up to 2 hours, and if the system has too many close_wait states, it may cause "Too Many open files"

Close_wait first needs to rule out the application, and secondly, it can shorten the close_wait time by modifying the kernel:

net.ipv4.tcp_keepalive_time = 1800= 3

TIME_WAIT state Causes and treatment:

This situation is more common, some crawler servers or Web servers (if the network management is not in the installation of kernel parameter optimization) often encounter this problem, how this problem is generated?

From the above can be seen, time_wait is active to close the side of the connection to maintain the state, for the crawler server He is the "client", after completing a crawl task, he will initiate the active shutdown connection, thus enter the state of Time_wait, Then, after maintaining this state 2MSL (max segment lifetime) time, completely shut down the recycle resource. Why do you do this? Clearly has been actively shut down the connection why should we keep the resources for some time? This is the design of TCP/IP, mainly for the following two aspects of consideration:

1. Prevent the package in the last connection, get lost and re-appear, affect the new connection (after 2MSL, all the duplicates in the last connection will disappear)
2. Reliable shutdown of TCP connections. The last ACK (FIN) sent at the active shutdown is likely to be lost, when the passive side will resend fin, and if the active side is in the CLOSED state, it will respond to RST rather than ACK. So the active side should be in the TIME_WAIT state, but not CLOSED. In addition, this design time_wait will periodically recycle resources, and will not occupy a lot of resources, unless a short period of time to accept a large number of requests or be attacked.

It is worth saying that for TCP-based HTTP protocol, the server side of the TCP connection is closed, so that the server side will enter the TIME_WAIT state, it can be imagined that for the large traffic of the Web server, there will be a large number of time_wait state, If the server receives 1000 requests in a second, the backlog of 240*1000=240,000 Time_wait records will be maintained, which can be a burden to the server. Of course, the modern operating system will use a fast lookup algorithm to manage these time_wait, so for the new TCP connection request, determine whether hit in a time_wait not too much time, but there are so many States to maintain is always bad.
HTTP protocol version 1.1 stipulates that the default behavior is keep-alive, that is, the reuse of TCP connections to transmit multiple request/response, one of the main reasons is to find this problem.

That is to say that the HTTP interaction with the picture above is not the same, close the connection is not the client, but the server, so the Web server will also appear a lot of time_wait situation. Now, how to solve this problem. The solution is simply to allow the server to quickly reclaim and reuse those time_wait resources. Here is a look at our network management changes to the/etc/sysctl.conf file:

#for a new connection, how many SYN connection requests the kernel will send to decide to discard, should not be greater than 255, the default value is 5, corresponding to 180 seconds or so timenet.ipv4.tcp_syn_retries=2#net.ipv4.tcp_synack_retries=2#indicates the frequency at which TCP sends keepalive messages when KeepAlive is employed. The default is 2 hours, instead of 300 seconds.net.ipv4.tcp_keepalive_time=1200net.ipv4.tcp_orphan_retries=3#indicates that if the socket is closed by a local requirement, this parameter determines how long it remains in the Fin-wait-2 stateNet.ipv4.tcp_fin_timeout=30#represents the length of the SYN queue, which defaults to 1024, and a larger queue length of 8192, which can accommodate more network connections waiting to be connected. Net.ipv4.tcp_max_syn_backlog = 4096#indicates that SYN Cookies are turned on. When a SYN wait queue overflow occurs, cookies are enabled to protect against a small number of SYN attacks, which defaults to 0, which means closeNet.ipv4.tcp_syncookies = 1#means to turn on reuse. Allows time-wait sockets to be re-used for new TCP connections, which defaults to 0, which means shutdownNet.ipv4.tcp_tw_reuse = 1#represents a fast recycle of the time-wait sockets on a TCP connection, which defaults to 0, which indicates a shutdownNet.ipv4.tcp_tw_recycle = 1##减少超时前的探测次数Net.ipv4.tcp_keepalive_probes=5##优化网络设备接收队列net.core.netdev_max_backlog=3000

TCP/IP Connection status