Troubleshooting of DDoS attacks on hosts at the source of LAN Paralysis

After receiving help from the customer, we recently conducted a network "visit ". This is a network fault caused by a DDos attack on the slave host. It is a typical case and the troubleshooting process is also twists and turns. I will restore the process and share it with you.

1. Network Environment

This customer is a chemical company with a small network size. A lan consisting of more than 10 switches has about 150 nodes. No VLAN division.-Some hosts run the IPX protocol, while others run the TCP/IP protocol. Only a few hosts can access the Internet. In the access mode, the ADSL Router directly connects to a vswitch in the network. The built-in firewall is enabled on the ADSL Router, and anti-virus software is installed on all hosts that can access the Internet.

2. fault description

On the last day, the entire network was suddenly paralyzed. We can see that the indicator lights of all vswitches flash rapidly. The test shows that any two hosts in the network cannot ping each other, And all network applications cannot run normally. After unplugging some network cables (hierarchical connections between switches), the symptoms slowed down and finally recovered to normal. Plug the unplugged Network cables one by one back to the original location, and the fault does not occur again. Since then, this phenomenon has become irregular and irregular.

3. Fault Analysis

When a fault occurs, the switch port indicator is flashing rapidly, and any two hosts in the network cannot ping each other. It is preliminarily concluded that the network is filled with a large number of broadcast packets, network Resources are exhausted. So where did these sudden massive broadcast packages come from? To find the broadcast source, use Sninffer software to capture data packets when a fault occurs. It is found that the original broadcast packet is not displayed in the network, but there are a large number of abnormal Single IP packets.

4. troubleshooting

The analysis shows that these packets are sent to the host 172. *. 1l of the host 219. *. 88, and the sending speed is no less than packets per second. Ask the Administrator to learn that 172. *. *. 1l is an intranet host that can access the Internet. This is obviously abnormal. After the suspicious host is disconnected, the problem is solved.

5. In-depth Analysis

Although network faults are troubleshooting, I feel that everything is not that simple. Generally, packets sent to host 219. *. *. 88 are not broadcast packets and should not be sent to the switch port where the Sniffer host is running. Obviously, these packets are sent in the form of broadcast on the network. Such a number of broadcast packets flood the network, which is the culprit of network paralysis. (1 ). Lan host becomes a zombie

To identify the cause of the fault, install Sniffer on the suspicious host to analyze network behavior. The packet capture analysis finds that internetis connected to an ftpserver on the Internet and the hosts file ddos.txt is downloaded. What are the contents of the ddos.txt file? It turns out to be an IP address and an 8O port. Then, the data package is analyzed in one step to find that the target region of all data packets is the IP address specified in ddos.txt. These meaningless data packets use the 8O port to break through the firewall restrictions. In the past, this host was lucky enough to become a zombie of Distributed Denial of Service (DDoS) attacks. Each time an EIP is connected, A ddos.txt file is automatically downloaded to an ftpserver. If the file is empty, the file is downloaded at a certain interval until the obtained file contains the IP address and port of the target host, and DoS attacks are initiated to the target host. This is the reason why network faults occur irregularly and irregularly.

(2 ). ADSL Router "Drowned"

Another problem is that these attack packets should be sent from the slave machine to the ADSL Router and then to the target host on the Internet. They are not generally called broadcast packets, why do vswitches send broadcast packets in the form of broadcasts? There is only one possible cause: At this time, the address Forwarding Table (CAM) of the vswitch does not have the physical address of the internal network interface of the ADSL Router, causing the switch to broadcast a single packet to all vswitch ports.

At the beginning, the vswitch address forwarding table contains the physical address of the ADSL Router. After the slave machine starts the attack, a stable attack data stream is formed in the Network: slave machine → several switches → ADSL Router → target host attacked. At this time, the impact on the Intranet is only the ports of some switches and the ADSL Router. Because the ADSL Router is busy handling a large number of attack packets, it is "Drowned", which will cause problems in accessing the Internet of hosts in the intranet.

(3 ). Broadcast storm caused by Interaction

What causes the vswitch address forwarding table to discard the physical address of the internal network interface of the ADSL Router? There are two scenarios: 1. By default, if the switch is not connected to a data frame sent by a device within five minutes, the device is considered to be down. in order to save resources, this address will be deleted from the CAM table. Second, when the STP protocol detects a change in the network topology, all non-refreshed CAM table items will be cleared. Assume that the host is restarted or the network cable is plugged in because someone cannot access the Internet at this time, the status of the switch port will change. In this case, the switch determines that the network topology has changed. The next action is to notify all switches and clear the address forwarding table items that have not been refreshed within 15 seconds. "Not refreshed" indicates that the switch does not receive a data frame with the physical address as the source address. That is to say, the device does not send data frames to other devices, the physical address of the device will be cleared in the address forwarding table of the switch. All data frames with the physical address of the device as the destination address will be sent to all ports of the switch even if they are not broadcast frames.

(4 ). Fault Formation Process

Through the above analysis, we finally return to our example to understand the fault formation process. After the attack on the slave machine starts, the small ADSL Router receives and processes no less than l5o00 packets per second. Even if it has three heads and six arms, it has no chance to send data packets to the switch. That is to say, it only accepts and does not send, and cannot refresh the related table items in the switch cAM table. Then, if it is 15 seconds later, if it is 5 minutes later, the vswitch will clear the physical address record of the ADSL Router. Do not forget that the attack data stream is not stopped at this time, and these attack data frames are exactly the destination address of the physical address of the ADSL Router. In this way, a disaster occurs, all data frames are broadcast to each port in each vswitch in the network.

6. Solution

On the surface, this network fault is caused by a slave host and has a certain chance. However, basically, it is inevitable that an unreasonable network structure is the key cause of this failure. The solution provided by the author is:

(1 ). Disconnect the computer that acts as a zombie from the network and reinstall the system to eliminate potential risks.

(2 ). While enhancing the local network security prevention measures, adjust the original network structure: according to different applications into several VLANs, machines that can access the Internet are divided into a specific VLAN, limits the impact of such faults.

(3 ). Configuring the port connecting to the host as the STP speed port does not participate in the STP protocol can reduce unnecessary topology change operations on the switch in the network.

The author's inspiration for this online troubleshooting is that the online troubleshooting is similar to treating diseases by doctors. Normal doctors often treat patients with headaches and pains, however, brilliant doctors often have a cure. Why is network troubleshooting not?