| Analysis of TCP connection disconnection |
Zheng Yong(Zhengyzy@cn.ibm.com), Software engineer, IBM August 21, 2008
Introduction In official documents, TCP/IP protocol clusters are also known as Internet Protocol clusters. TCP/IP protocol cluster is currently the most widely used Global Internet technology. Its layered structure 1 is shown in: Figure 1. Hierarchy of TCP/IP protocol Clusters As shown in 1, the data link layer is mainly responsible for processing the transmission media and many other physical interface details; the network layer is responsible for processing the activities of data groups in the network, including the upper layer data packet segmentation, routing phost2008-08-21T00: 00: 00; the transport layer is responsible for providing end-to-end communication for the two hosts; the application layer is responsible for handling specific application details. The IP protocol is the core protocol at the network layer to provide unreliable and connectionless data transmission services, while the TCP protocol is at the transport layer, based on the unreliable and connectionless IP protocol, it can provide connection-oriented and reliable communication for the two hosts. UDP? Because TCP is a connection-oriented protocol, a connection must be established before two hosts communicate. Next we will briefly introduce the establishment of the TCP connection and how the communication parties maintain the established TCP connection. Establish and maintain TCP connections The establishment of a TCP connection must be completed through the famous "three-way handshake. The following example shows how to establish a TCP connection. In the following description, the client host is testclient.cn.ibm.com (Linux), and the server host is testserver.cn.ibm.com (Aix ). Run the tcpdump-I eth0 host testserver command on a terminal of the testclient host to start tcpdump to listen to network data (eth0 is the NIC used for communication between the client host and the external network ); at the same time, execute the following command on another terminal of the client host: (root @ testclient/)> Telnet testserver. The output of tcpdump on the client host is shown in List 1. Listing 1. Create a three-way handshake for a TCP connection
Note: We deleted some irrelevant information in the output result of tcpdump. For ease of understanding, we convert the above output to the actual sequence diagram 2. Figure 2. TCP creates the actual sequence of three-way handshakes As shown in figure 2, the following three handshakes are performed to establish a connection between testclient and testserver:
A TCP connection is established after the preceding three handshakes are completed. After that, information can be transmitted between the two ends of the connection. Therefore, a TCP connection can be considered as a communication channel identified by the IP address and port at both ends. The establishment of a TCP connection is the process of registering the above communication channel with both parties. Once a TCP connection is established, as long as the intermediate nodes (including gateways, switches, routers, and other network devices) between the two parties work normally, before either party actively closes the connection, all TCP connections will be retained. This feature enables a idle connection that does not exchange any information for a long time to be maintained for several hours, days, or even months. The intermediate router can crash and restart, and the network cable can be hung up and connected. TCP connections can be kept as long as the hosts at both ends are not restarted.
Causes of TCP connection disconnection Ideally, a TCP connection can be maintained for a long time. However, in actual applications, a seemingly normal TCP connection maintained on the client or server may have been disconnected. The TCP connection is interrupted due to two major impacts: What are the two-party nodes in the communication between the intermediate network node and the client/server node? In actual network applications, communication between two hosts often needs to traverse multiple intermediate nodes, such as routers, gateways, and firewalls. Therefore, the TCP connection between two hosts is also affected by the intermediate node, especially the firewall (software or hardware firewall. A firewall is a device that has multiple implementation methods (software implementation, hardware implementation, or software and hardware combination). It needs to scan inbound and outbound information flows according to a series of rules, it also allows secure (compliant with rules) information interaction and prevents insecure (violating rules) information interaction. The working characteristics of the firewall determine that it takes a lot of resources to maintain a network connection, and the enterprise firewall is often located at the entrance and exit of the enterprise network, long maintenance of inactive TCP connections will inevitably lead to a decline in network performance. Therefore, most firewalls disable connections that are not active for a long time by default, resulting in TCP connection disconnection. Similarly, if an intermediate node is abnormal and the request from the client to close the connection cannot be passed to the server, the connection on the server will also be disconnected. On the other hand, it takes some system resources to create a TCP connection for a host at both ends of a TCP connection. If a connection is no longer used, we always hope that the two hosts that communicate can actively close the connection to release the system resources occupied. However, if the connection fails to be properly closed due to exceptions on the client (for example, crashes or restarts abnormally), the connection on the server may be disconnected. Whether it is a client node or a server node, the disconnected TCP connection cannot transmit any information. Therefore, maintaining a large number of disconnected TCP connections will result in a waste of system resources. This waste of system resources may not cause too many problems to client nodes. However, for server hosts, this may lead to system resources (especially memory resources and socket resources) the service is denied for new user requests. Therefore, in practical applications, the server needs to take appropriate methods to detect whether the TCP connection has been disconnected.
Three common methods for detecting TCP connection disconnection The principle of detecting whether a TCP connection is disconnected or working normally is relatively simple: regularly send a certain format of information to the connected Remote Communication Node and wait for feedback from the remote communication node, if you receive the correct feedback from the remote node within the specified time, the connection is normal. Otherwise, the connection has been disconnected. Based on this principle, there are currently three common detection methods. Application self-detection The application itself comes with the function of detecting its own TCP connections. This method has great flexibility and can select the corresponding detection mechanism and function implementation based on the characteristics of the application. However, in practical applications, most applications do not have the function of self-detection. Third-party application Detection This method is to install a third-party application on the service node to detect whether all TCP connections on the node are normal or have been disconnected. The biggest disadvantage of this method is that all clients that support detection can identify data packets from the application. Therefore, this method is rare in practical applications. TCP protocol layer live Detection The most common test method is to use the active detection function provided by the TCP protocol layer, that is, the active timer of the TCP connection. Although this function is not part of the RFC specification, it is widely used in almost all UNIX-like systems. In the following section, we will focus on the live-preserving detection methods from the TCP protocol layer.
TCP connection life-saving timer on Unix-like systems The retention timer of TCP connections can be implemented at the application layer or in TCP. This issue is controversial, so the TCP connection retention test is not part of the TCP specification. For convenience, almost all UNIX-like systems provide functions in TCP. List 2. Active timer on Common Unix systems
The time units of parameters on different systems are different. On Aix, the time unit of tcp_keeidle/tcp_keepinit/tcp_keepintvl is 0.5 seconds. on Linux, the time unit of net. ipv4.tcp _ keepalive_intvl and net. ipv4.tcp _ keepalive_time is second. In addition, the above parameters are only valid for connection to the server application running on it. Note: On Solaris, Run "NDD/dev/tcp /?" Command to display the above similar parameter information, and on hp unix, you can use the nettune or NDD command to query. Because all UNIX-like systems support this function, we will introduce the meaning and mechanism of the above parameters in the following sections based on the AIX system.
Mechanism and principle of TCP connection active-active detection in Aix As listed in Listing 2, the active-active probe mechanism on AIX is controlled by four parameters. For more information, see listing 3: Listing 3. Active Timer control parameters on AIX
Let's look at a specific example. Use the tcp_keepidel = 240 (2 minutes): tcp_keepcnt = 8: tcp_keepintvl = 150 (75 seconds) parameter value on the testserver (Aix host; start tcpdump on testserver to view the interaction of network packets; initiate a request from testclient to establish a telnet connection with testserver. After the connection is established, unplug the network cable of testclient and observe the data output of the server (see figure 4 ). Listing 4. tcpdump output of Telnet connection on the server
From list 4, we can see that 6th rows of packets are the final data sent by the connection, while 7th rows are the confirmation of 6th rows of data. Then, the connection does not have any data interaction, so that the connection remains inactive. After two minutes (the difference between 8th-row datagram time 04:53:54 and 7th-row datagram time 04:51:54, that is, the value of tcp_keepidle, row 8th is the first live-preserving test datagram initiated by the server. Because the server does not receive the corresponding detection packet from the client, after the time interval (75 seconds) of tcp_keepintvl, Row 3 shows that the server initiates a new test data packet. The server continuously sends tcp_keepcnt detection packets (the above results show that tcp_keepcnt + 1 Detection packet is continuously sent on AIX), but still does not receive any response from the client, therefore, the server sends a reset packet to the client in line 3 and closes the connection on the server. It should be noted that, although the active-active probe does not affect normal TCP connections even though the TCP probe packet is sent. From list 4, we can see that the number of TCP packets sent from line 4 is the 1 byte data starting from line 1, and the data has been sent and confirmed by the client in line 3. For normal connections, the client will return an ACK packet shown in row 7th after receiving the detection packet, and then indicate to the server that the connection works properly. Next, we will analyze the impact of the above mechanism on TCP connection persistence through an actual TCP disconnection example, two optional solutions are proposed for applications that require TCP connection for a long time.
TCP disconnection and data analysis on AIX Figure 3. network topology with TCP disconnection All server hosts are classified as one LAN and placed behind firewall B. Due to work requirements, the testclient host from the LAN in the work zone needs to establish a connection with the database on the testserver in the LAN of the server using TCP/IP, the upper-layer applications on testclient perform operations on the databases on testserver through the connection. In the actual test, we found that when both testclient and testserver are working normally, the client on testclient did not receive any exception information in advance, unexpected disconnection may occur for the connection (the connection is reset by foreign host error will be reported when you try to operate the database through the connection ). As this phenomenon continues to occur and the intermediate nodes (such as routers and switches) in the network work normally, physical factors (such as power loss and downtime) can be ruled out. To facilitate analysis of the reason for disconnection, we first checked the default retention settings on the testserver machine:
The tcp_keepidle on testserver is 14400, that is, 2 hours. Since the intermediate node works normally, why does the active mechanism not work? For analysis, we use tcpdump to capture the packet information on testclient and testserver, as shown in listing 5 and 6. Listing 5. server-side tcpdump data output
Listing 6. tcpdump data output of the Client
From list 5, we can see that the time when the connection is inactive reaches the 2 hours set by tcp_keepidle, the server host sends the first active connection detection packet (the first line in listing 5 ). Then, the server host receives the Connection Reset packet from testclient (Row 5 in listing 5 ). Then, the server closes the connection (you can view it through netstat-Ni ). However, the tcpdump data in Listing 6 shows that the testclient does not send any packets. So who sent the reset message to testserver? To view the sender of the reset packet, use the tcpdump command to capture the packet information of server and firewall B again (note: capture the egress nic and entry nic data of network data on the firewall host). The result is displayed, after receiving the first probe packet from testserver, firewall B immediately sends a reset packet to testserver. The above analysis shows that the connection has been terminated by firewall B after the last interactive data is transmitted to the server for the first active-active detection, any packets transmitted Based on the connection will be discarded by the firewall and sent a reset packet when attempting to pass through the firewall.
Two common solutions There are two common solutions for TCP disconnection: Solution 1: extend the time for the firewall to terminate inactive TCP connections. For example, in the above case, you can adjust the firewall settings to set the time to 2 hours later than the server-side setting. Solution 2: shorten the TCP connection retention time on the server. The purpose of shortening this time is to send a live detection packet before the connection is terminated by the firewall, which can detect the client status and make the connection active. In the first solution, increasing the TCP connection retention time may lead to a decrease in the firewall performance, especially when a large number of connections are in inactive state for a long time; in the second solution, if the TCP connection retention time of the server is shortened, the number of data packets in the network is increased and the extra network bandwidth is occupied. Therefore, the two solutions have their own advantages and disadvantages and need to be selected based on different actual application situations.
Summary This article introduces the concepts of TCP connection establishment and persistence and common factors that affect TCP connection persistence. The configuration parameters for TCP connection life-saving test on Common Unix-like systems are provided, and an actual TCP disconnection case is analyzed based on the tcpdump tool of Aix. Finally, two feasible solutions are provided for TCP disconnection. References
Author Profile
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
