Linux TCP traffic is close_wait, causing service-side process to hang up

This article describes the reasons for the close_wait state of Linux server-side TCP communication, which is illustrated by an example of a "bad" effect: direct the server process down.

CentOS server to establish listening port

1 CentOS server to establish listening port

As shown in the figure above, open a terminal interface in the virtual machine CentOS7 Server (192.168.1.178) to establish a 8000-port listening Service (pid:13035). The code used is the same as the program in the previous article, with just one more sigpipe signal processing and automatic reply (auto response from server.) Part.

The code is as follows

Copy Code

#include <stdio.h> #include <string.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <arpa/inet.h> #include <errno.h> #include <signal.h> Whether add a signal handle. #define SIGNAL_HANDLE 0 void Sig_handle (int signal) { printf ("Receive a signal=[%d].\n", signal); Return } int main (int argc, char **argv) { int SERVER_SOCKFD; int CLIENT_SOCKFD; int Len; int llopt = 1; struct sockaddr_in my_addr; struct sockaddr_in remote_addr; int sin_size; Char Buf[bufsiz]; memset (&my_addr, 0, sizeof (MY_ADDR)); my_addr.sin_family = af_inet; MY_ADDR.SIN_ADDR.S_ADDR = Inaddr_any; My_addr.sin_port = htons (8000); #if Signal_handle struct sigaction new_act, old_act; New_act.sa_handler = Sig_handle; new_act.sa_flags = 0; Sigemptyset (&new_act.sa_mask); Sigaction (Sigpipe, &new_act, &old_act); Sigaction (SIGINT, &new_act, &old_act); #endif if ((SERVER_SOCKFD = socket (af_inet, sock_stream, 0)) < 0) { Perror ("socket"); return 1; } if (setsockopt (SERVER_SOCKFD, Sol_socket, so_reuseaddr, &llopt, sizeof (llopt))) { Close (SERVER_SOCKFD); return errno; } if (Bind (SERVER_SOCKFD, (struct sockaddr *) &my_addr, sizeof (struct sockaddr)) < 0) { Perror ("bind"); return 1; } Listen (SERVER_SOCKFD, 5); sin_size = sizeof (struct sockaddr_in); printf ("Server is listening with pid=[%d].\n", Getpid ()); while (1) { if ((CLIENT_SOCKFD = Accept (SERVER_SOCKFD, (struct sockaddr *) &remote_addr, &sin_size)) < 0) { Perror ("accept"); return 1; } Print the IP address and port of client. printf ("Accept client[%s:%u].\n", Inet_ntoa (REMOTE_ADDR.SIN_ADDR), Ntohs (Remote_addr.sin_port)); Send (CLIENT_SOCKFD, "Auto response from server.", strlen ("Auto response from server."), 0); memset (buf, 0x00, Bufsiz); while (len = recv (CLIENT_SOCKFD, buf, Bufsiz, 0)) > 0) { Buf[len]= ' "; printf ("Message from client=[%s]\n", buf); } Close (CLIENT_SOCKFD); } Close (SERVER_SOCKFD); return 0; }

2 Create a client using the Telnet command in Linux

Create a new shell script netstat_nap.sh that contains only one valid command Netstat-nap|head-n 2;netstat-nap|grep 8000.

Open a Linux terminal interface again, and then enter the command telnet 192.168.1.177 8000 as a client to establish a TCP connection to the server. Then execute the script./netstat_nap.sh can see that TCP traffic to the Linux client (pid:13045) and server (pid:13035) has become established, as shown in the following illustration:

3 Create a client using the Telnet command in Windows

Open a PowerShell terminal interface in Windows Host (192.168.1.110) and enter the command Telnet 192.168.1.177 8000 as a client to establish a TCP connection to the Linux server.

Then execute the script./netstat_nap.sh, you can see that TCP traffic to Windows client (port: 64012) and server side (PID:13035) has become established state. Using command lsof-i:8000, you can see the files that the process opens.

4 directly shut down the Windows Telnet Client interface and use the Wireshark grab package

After you close the Telnet interface, continue to use the netstat_nap.sh script and the lsof command to discover that the TCP traffic that you just established appears to be in a close_wait state.

After waiting 2 minutes, use Wireshark grab in Windows to find that the TCP link has been reset because the client sent the Rst+ack packet to the Linux server:

Using Wireshark in Windows to grab packages

The netstat_nap.sh script and the lsof command are used again in Linux to discover that the close_wait state no longer exists.

5 Shut down the Linux telnet client

After Windows closes the Telnet client interface and sends the RST+ACK message, close the Telnet client opened in section 2 in Linux. At this point the Linux server process executes the close () function at line 90th, and then performs a normal four wave-waving shutdown of the TCP connection.

The Linux server-side process then continues to remove the completed connection from the completed connection queue in the kernel, so that the client connection that Windows Telnet established in section 3 is read. As shown in the following illustration, the server-side process prints the 80th row of data (Accept client[192.168.1.110:64012].), but the service-side process is dead.

CentOS server to establish listening port

Then use the netstat_nap.sh script and lsof command again in Linux:

6 Causal Analysis

Because the TCP link for the Windows client is closed in section 4 due to RST, there is no read end. Then when the Linux server executes the 82-line Send () function, sending a 26-byte message data to the previous socket descriptor, it receives the sigpipe signal sent from within, causing the server-side process to shut down by default.

So if you want to capture this sigpipe signal, you can change the Signal_handle macro definition value in the 17 line of the program to 1, and you'll get the situation as shown in the following figure (the process works correctly).

7 Problem Extension

What happens if you close the Windows Client interface in section 4th and turn off the Linux Telnet client interface directly as shown in section 5th? And then again the test, the process ibid, below is the test results and analysis.

Use the Netstat and lsof commands to view the status of the TCP service and discover that the listening service is normal:

The TCP communication packets are then crawled using tcpdump and Wireshark respectively, and are intercepted as follows. You can see that after the Linux Telnet client has finished waving four times, the server process continues to send 26 bytes of message data to the socket descriptor that was established by the Windows Telnet client.

Because the Windows client is in the Fin_wait2 state at this time (the Linux server is in the close_wait state), the server can continue to send its sending data (that is, the push+ack message in the diagram), and then the Windows client responds to the Rst+ack message. Thereby both TCP links are reset.

Using tcpdump to grab packages in Linux

Using Wireshark in Windows to grab packages

This allows the Linux server-side process to perform the listening task normally:

8 Other

Someone on the internet that this kind of client or server-side abnormal shutdown of the connection called TCP semi-shutdown (half-close), such as network cable unplugged, sudden power outages, at this point to the end of the connection is still considered both sides of the connection is open.