TCP/IP connection status, tcpip connection

TCP/IP connection status, tcpip connection

1. Establish a connection protocol (three-way handshake)
(1) the client sends a TCP packet with SYN flag to the server. This is packet 1 during the three-way handshake.
(2) The server responds to the client. This is the first packet in the three-way handshake. This packet carries both the ACK mark and SYN sign. Therefore, it indicates the response to the client SYN Packet. It also marks the SYN message to the client and asks whether the client is ready for data communication.
(3) The customer must re-respond to an ACK packet in the service segment, which is packet segment 3.
2. Connection termination protocol (four handshakes)
Because the TCP connection is full-duplex, each direction must be closed separately. This principle is that when one party completes its data sending task, it can send a FIN to terminate the connection in this direction. Receiving a FIN only means that there is no data flow between the two parties. a tcp connection can still send data after receiving a FIN. First, the party that closes the service will take the initiative to close the service, and the other party will passively close the service.
(1) The TCP client sends a FIN to disable data transmission from the client to the server (packet segment 4 ).
(2) When the server receives the FIN, it sends back an ACK and confirms that the serial number is 1 (packet segment 5 ). Like SYN, a FIN occupies a sequence number.
(3) The server closes the client connection and sends a FIN to the client (packet segment 6 ).
(4) The customer segment sends back the ACK message for confirmation, and sets the confirmation sequence number to receive the serial number plus 1 (packet segment 7 ).
CLOSED: this is nothing to say, indicating the initial state.
LISTEN: this is also a very easy-to-understand state, indicating that a SOCKET on the server is in the listening state of the Guardian brain and can accept connections.
SYN_RCVD: this status indicates that the SYN packet is received. Normally, this status is an intermediate status of the server SOCKET during the three-way handshake session when a TCP connection is established. It is very short, basically, you can hardly see this status with netstat unless you write a client test program and deliberately disable the last ACK packet from the three TCP handshakes. Therefore, when an ACK packet is received from the client, it enters the ESTABLISHED status.
SYN_SENT: this status echo SYN_RCVD. When the client SOCKET executes the CONNECT connection, it first sends the SYN packet, and then it enters the SYN_SENT status, and wait for the server to send 2nd messages in the three-way handshake. SYN_SENT status indicates that the client has sent SYN packets.
ESTABLISHED: This is easy to understand, indicating that the connection has been ESTABLISHED.
FIN_WAIT_1: The FIN_WAIT_1 and FIN_WAIT_2 statuses both indicate waiting for the FIN packets of the other party. The difference between the two States is that the FIN_WAIT_1 state is actually when the SOCKET is in the ESTABLISHED State, it wants to actively close the connection and send a FIN packet to the other party, the SOCKET enters the FIN_WAIT_1 state. When the other Party responds to the ACK packet, it enters the FIN_WAIT_2 status. Of course, under normal circumstances, the other party should immediately respond to the ACK packet, regardless of the situation, therefore, the FIN_WAIT_1 status is generally difficult to see, while the FIN_WAIT_2 status is often seen using netstat.
FIN_WAIT_2: The above has explained in detail this status. In fact, the SOCKET in the FIN_WAIT_2 status indicates a semi-connection, that is, either party requires a close connection, but also tells the other party, I have some data to send to you, and close the connection later.
TIME_WAIT: indicates that the FIN packet of the other party is received, and the ACK packet is sent concurrently, so that 2MSL can return to the CLOSED available status. If FIN_WAIT_1 receives a message with both the FIN mark and ACK mark, it can directly enter the TIME_WAIT status without passing through the FIN_WAIT_2 status.
CLOSING: this is a special exception that is rare in actual situations. Normally, when you send a FIN packet, you should first (or simultaneously) receive the ACK packet from the other party and then receive the FIN packet from the other party. However, the CLOSING status indicates that after you send the FIN packet, you have not received the ACK packet from the other party, but have also received the FIN packet from the other party. Under what circumstances will this happen? In fact, it is not difficult to come to the conclusion that if both parties close a SOCKET at the same time, both parties may send FIN packets at the same time, that is, the CLOSING status may occur, both parties are closing the SOCKET connection.
CLOSE_WAIT: the meaning of this state is actually waiting to be closed. How can this problem be solved? When the other party closes a SOCKET and sends a FIN packet to itself, your system will undoubtedly respond to an ACK packet to the other party, and then enters the CLOSE_WAIT status. Next, in fact, what you really need to consider is to check if you still have data to send to the other party. If not, you can close the SOCKET and send the FIN packet to the other party, that is, close the connection. So what you need to do in CLOSE_WAIT is to wait for you to close the connection.
LAST_ACK: this status is easy to understand. It passively closes a side and waits for the other side's ACK packet after sending the FIN message. After receiving the ACK message, you can enter the CLOSED available status.
Finally, I have two answers. I have analyzed the results (not necessarily 100% correct)
1. Why is the three-way handshake while the four-way handshake when the connection is closed?
This is because the SOCKET in the LISTEN status of the server can respond to ACK and SYN after receiving the SYN Packet connection request) it is sent in a message. However, when the connection is closed, when the other party receives the FIN Message notification, it only indicates that the other party has no data to send to you; but not all your data may have been sent to the other party, therefore, you may not close the SOCKET immediately, that is, you may need to send some data to the other party, and then send the FIN message to the other party to indicate that you agree to close the connection now, therefore, the ACK messages and FIN messages are sent separately in most cases.
2. Why does the TIME_WAIT status still need to be 2 MSL before it can be returned to the CLOSED status?
This is because, although both parties agree to close the connection, and the four packets of the handshake are also coordinated and sent, it can be directly returned to the CLOSED state (just like from SYN_SEND to ESTABLISH State), but because we must assume that the network is unreliable, you cannot guarantee that the last ACK message you sent will be received by the other party. Therefore, the SOCKET in the LAST_ACK status of the other party may fail to receive the ACK message due to timeout, but resend the FIN message, therefore, the TIME_WAIT status is used to resend potentially lost ACK packets.

 

 

TCP is a connection-oriented protocol. Therefore, a connection must be established before both parties send data. This is totally different from the preceding protocol. All the Protocols mentioned above only send data. Most of them do not care whether the sent data is sent or not, especially UDP. From a programming perspective, UDP programming is also much simpler-UDP does not need to consider data sharding.

In this document, telnet login and exit are used to explain the process of establishing and suspending a TCP connection. We can see that the establishment of a TCP connection can be simply calledThree-way handshakeAnd the disconnection can be calledFour handshakes.

1. Establish a connection

When establishing a connection, the client first requests to the server to open a port (using a TCP packet with SYN segment equal to 1), and then the server sends back an ACK packet to notify the client to receive the request message, after receiving the confirmation message, the client sends a confirmation message again to confirm the confirmation message (bypass) sent by the server. At this point, the connection is established. This is called a three-way handshake. If you want to prepare both parties, you must send three packets, and only three packets are required.

We can imagine that if TCP's timeout retransmission mechanism is added, TCP can completely ensure that a data packet is sent to the destination.

2. End the connection.

TCP has a special concept calledHalf-closeThis concept means that the TCP connection is a full-duplex (both sending and receiving can be done at the same time) connection. Therefore, when closing the connection, you must close the connection between the transmission and sending directions. The client sends a TCP packet whose FIN is 1 to the server, and then the server returns an ACK packet to the client, and sends a FIN packet. When the client replies the ACK packet (four handshakes ), the connection is over.

3. Maximum packet length

When establishing a connection, both parties must confirm the maximum message length (MSS) of each other to facilitate communication. Generally, the SYN length is MTU minus the fixed IP header and TCP Header Length. For an Ethernet, it can generally reach 1460 bytes. Of course, for non-local IP addresses, the MSS may only have 536 bytes, and the value will be smaller if the MSS of the intermediate transmission network is smaller.

4. TCP status migration Diagram

The P182 page of the book provides the TCP status chart, which looks complicated because it contains two parts: Server Status migration and client status migration, from a certain point of view, this figure will be much clearer. The servers and clients here are not absolute. The clients that send data are the clients, and the servers that receive data are the servers.

4. Status migration diagram of client applications

The client status can be expressed in the following process:

CLOSED-> SYN_SENT-> ESTABLISHED-> FIN_WAIT_1-> FIN_WAIT_2-> TIME_WAIT-> CLOSED

The above process is a proper process in the normal circumstances of the program. From the figure in the book, we can see that when a connection is established, when the client receives the ACK of the SYN packet, the client opens an interactive data connection. The client ends the connection actively. After the client ends the application, it needs to go through the FIN_WAIT_1, FIN_WAIT_2, and other statuses. The migration of these statuses is the four handshakes mentioned above to end the connection.

4. Server Status migration Diagram

The server status can be expressed in the following process:

CLOSED-> LISTEN-> SYN received-> ESTABLISHED-> CLOSE_WAIT-> LAST_ACK-> CLOSED

When a connection is established, the server enters the data interaction status only after the third handshake, while closing the connection is after the second handshake (note not the fourth ). After the feature is disabled, you must wait for the client to provide the final ACK package before entering the initial state.

4. Migration in other statuses

The diagram in the book also contains some other State migration, which summarizes the two aspects of server and client as follows:

 

4.4.2MSL waiting status

In the figure given in the book, there is a TIME_WAIT wait state, which is also called the 2MSL state. It means that after TIME_WAIT2 sends the last ACK datagram, it will enter the TIME_WAIT state, this status prevents the datagram of the last handshake from being transmitted to the other party and prepared (note that this is not the four handshakes, but the fourth handshake is the insurance status ). This State ensures that both parties can end normally, but the problem also arises.

Because of the 2MSL status of the plug-in port (the plug-in Port indicates the IP address and port pair, socket), the application cannot use the same plug-in again in 2MSL time, which is better for the customer program, however, for a service program, such as httpd, it always needs to use the same port for service. In 2MSL time, an error occurs when httpd is started (the plug is used ). To avoid this error, the server provides a concept of Calm time. This means that although the server can be restarted within 2MSL, the server still needsCalmWait 2MSL time before the next connection can be made.

4.5.FIN _ WAIT_2 status

This is the famous semi-closed status, which is the status after the client and server shake hands twice when the connection is closed. In this state, the application can accept data, but data cannot be sent. However, the client is always in the FIN_WAIT_2 state, and the server is always in the WAIT_CLOSE state, the Application Layer determines to close this state.

5. RST. both open and close

RST is another way to close the connection. The application should be able to determine the authenticity of the RST package, that is, whether the connection is aborted abnormally. Both open and close are two special TCP states, with a low probability of occurrence.

6. TCP Server Design

We have previously talked about UDP server design. We can find that UDP servers do not need the so-called concurrency mechanism at all, and they only need to establish a data input queue. But TCP is different. The TCP server needs to establish an independent process (or lightweight, thread) for each connection to ensure the independence of the conversation. Therefore, the TCP server is concurrent. In addition, TCP also requires an incoming connection request queue (which is not required by the UDP server) to establish a dialog process for each connection request, this is why all TCP servers have a maximum number of connections. Based on the IP address and port number of the source host, the server can easily differentiate different sessions for data distribution.

 

Run the following command:

# Netstat-n | awk '/^ tcp/{++ state [$ NF]} END {for (key in state) print key. "\ t ". state [key]}'

The following result is displayed, and the numbers are different:

FIN_WAIT_1 286

FIN_WAIT_2 960

SYN_SENT 3

LAST_ACK 32

CLOSING 1

CLOSED 36

SYN_RCVD 144

TIME_WAIT 2520

ESTABLISHED 352 # almost equal to the number of concurrent connections

This command can classify and summarize the network connection status of the current system.

This statement is seen at Zhang banquet. It is said that it was obtained from Wang daoda, Technical Director of Sina interactive community business department. It is very good.

The return parameters are described as follows:

SYN_RECV indicates the number of requests waiting for processing;

ESTABLISHED indicates the normal data transmission status;

TIME_WAIT indicates the number of requests that have been processed and wait for the timeout to end.

------------------------------------------------------------------
Let's take a look at awk:
/^ Tcp/
Filters records starting with tcp to shield irrelevant records such as udp and socket.
State []
It is equivalent to defining an array named state.
NF
Indicates the number of fields in the record. As shown above, NF is equal to 6.
$ NF
Indicates the value of a field. For the record shown above, $ NF is $6, which indicates the value of the 6th fields, that is, TIME_WAIT.
State [$ NF]
Indicates the value of the array element. The record shown above indicates the number of connections in the state [TIME_WAIT] state.
++ State [$ NF]
Add one number. The record shown above is to add one to the number of connections in the state [TIME_WAIT] state.
END
Indicates the command to be executed in the last stage
For (key in state)
Traverse Arrays
Print key, "\ t", state [key]
Print the keys and values of the array, and use the \ t tab in the middle to beautify it.

 

Reasons for CLOSE_WAIT status and solutions:

When the connection is closed passively, the connection is in the CLOSE_WAIT status when the FIN has been received but has not been sent.
Generally, the CLOSE_WAIT status should last very short, just as the SYN_RCVD status. However, in some special cases, the connection may be in the CLOSE_WAIT status for a long time.
There are a lot of close_wait phenomena, mainly because the other party closes the socket Link under some circumstances, but we are busy with reading or writing, and the connection is not closed.

CLOSE_WAIT will last for up to 2 hours. If the system has Too many CLOSE_WAIT states, it may cause "Too open files"

CLOSE_WAIT first needs to eliminate application causes, and then you can shorten CLOSE_WAIT time by modifying the kernel:

net.ipv4.tcp_keepalive_time = 1800 net.ipv4.tcp_keepalive_probes = 3 net.ipv4.tcp_keepalive_intvl = 15 

 

 

TIME_WAIT status causes and solutions:

This situation is common. Some crawler servers or WEB servers (if the network management system does not optimize Kernel Parameters during installation) often encounter this problem. How does this problem occur?

From the above we can see that TIME_WAIT is the State maintained by the party that actively closes the connection. For the crawler server, it is itself a "client". After a crawling task is completed, he will initiate an active close connection to enter the TIME_WAIT state, and then completely close the Recycle resources after the 2MSL (max segment life time) state is maintained. Why? Obviously, the connection has been closed. Why do we need to keep the resources for a while? This is specified by the TCP/IP designer, mainly for the following two considerations:

1. Prevent the packets in the last connection from appearing again after they get lost, and affect the new connection (after 2MSL, all repeated packets in the last connection will disappear)
2. Close the TCP connection reliably. The last ack (fin) sent by the active shutdown party may be lost. The passive party will re-Send the fin. If the active Party is in the CLOSED state, it will respond to the rst instead of ack. Therefore, the active party must be in the TIME_WAIT state, not the CLOSED state. In addition, the design of TIME_WAIT will regularly Recycle resources and will not occupy a large amount of resources, unless a large number of requests or attacks are received within a short period of time.

It is worth noting that for the TCP-based HTTP protocol, the Server end closes the TCP connection. In this way, the Server enters the TIME_WAIT status, for Web servers with a large number of requests, there will be a large number of TIME_WAIT statuses. If the Server receives 1000 requests in one second, there will be a backlog of 240*1000 = 240,000 TIME_WAIT records, maintaining these statuses puts a burden on the Server. Of course, modern operating systems use Quick search algorithms to manage these TIME_WAIT instances. Therefore, it is not too time-consuming to determine whether a TIME_WAIT instance in hit instances is used for new TCP connection requests, however, it is always difficult to maintain so many statuses.
HTTP 1.1 requires that the default behavior is Keep-Alive, that is, multiple requests/response will be transmitted over TCP connections. One major reason is that this problem has been found.

 

That is to say, the HTTP interaction is different from the figure above. Closing the connection is not a client, but a server. Therefore, a large number of TIME_WAIT requests may occur on the web server. Now let's talk about how to solve this problem. The solution is simple, that is, to allow the server to quickly recycle and reuse those TIME_WAIT resources. Next let's take a look at the modifications made to the/etc/sysctl. conf file by the network management system:

# For a new connection, the kernel determines the number of SYN connection requests to be sent before giving up. It should not be greater than 255. The default value is 5, corresponding to net in the left-right time of 180 seconds. ipv4.tcp _ syn_retries = 2 # net. ipv4.tcp _ synack_retries = 2 # indicates the frequency of keepalive messages sent by TCP when keepalive is in use. The default value is 2 hours, which is changed to 300 seconds. net. ipv4.tcp _ keepalive_time = 1200. net. ipv4.tcp _ orphan_retries = 3 # indicates that if the socket is disabled by the local end, this parameter determines the time it remains in the FIN-WAIT-2 state. net. ipv4.tcp _ fin_timeout = 30 # indicates the length of the SYN queue. The default value is 1024. The length of the queue is 8192, which can accommodate more network connections waiting for connection. Net. ipv4.tcp _ max_syn_backlog = 4096 # indicates enabling SYN Cookies. When a SYN wait queue overflows, cookies are enabled to prevent a small number of SYN attacks. The default value is 0, indicating that net. ipv4.tcp _ syncookies = 1 # is disabled. Allow TIME-WAIT sockets to be re-used for new TCP connections. The default value is 0, indicating to disable net. ipv4.tcp _ tw_reuse = 1 # indicates to enable quick recovery of TIME-WAIT sockets in TCP connections. The default value is 0, indicating to disable net. ipv4.tcp _ tw_recycle = 1 # reduce the number of times before timeout. net. ipv4.tcp _ keepalive_probes = 5 # optimize the network device receiving queue. net. core. netdev_max_backlog = 3000