Case study-network performance reduction caused by attacks
Fault description
The manager of donghuantuo Mining Network reported that many users in the region had access to the Intranet of the group company or the Internet was slow or inaccessible. The linnancang mining report said that the Headquarters network was faulty.
This report is used to check related networks and has the following features:
First, the CPU load of C7609 is as high as 99%. ping donghuantuo from the Headquarters and the C3550 of linnan warehouse cannot be connected;
Second, the two mining households reported that they could access their respective internal websites;
Third, donghuantuo mine can ping the gateway 192.168.60.1, but the packet loss is serious;
Fourth, linnan warehouse cannot ping the gateway 192.168.130.1;
Fifth, the Headquarters users also reflected that the Internet access was obviously slower than usual.
Basic Environment Description
Shows the user's basic network topology.
The East huantuo mine is about 15 kilometers away from the group's headquarters and uses one hundred m optical fiber connections. The linnancang mine is more than kilometers away from its headquarters and only 10 kilometers away from the East huantuo mine, therefore, Lin nancang's access to the Headquarters network is connected to the C3550 nearest to the east huantuo mine through a C3550 communication line of 2 m, and connected to the Headquarters network through the East huantuo mine network.
The network of the East huantuo mine is vlan 23, the gateway 192.168.60.1 refers to C7609, the network of the forest south warehouse mine is vlan 22, and the gateway 192.168.130.1 also refers to C7609. The network is accessed by enabling OSPF routing.
Analysis Scheme Design Analysis Objectives
It is preliminarily determined that an abnormal network behavior causes the CPU load of the vro to increase, resulting in a decrease in the processing capacity, thus affecting network performance. Therefore, we can analyze the cause of high CPU load on the vro C7609 to analyze the cause of slow network access.
Analysis device deployment
Because the mining area is far away from the Government headquarters, network monitoring and analysis work cannot be carried out directly in the faulty mining area and can only be performed at the headquarters. Therefore, the pc used for the colais Network Analysis System Monitoring is deployed in C7609, connect to port G2/42.
In order to further analyze the cause of the fault, the source port is G4/5 on C7609, And the destination port is G2/42, and packets are captured by monitoring pc connected to the G2/42 port.
Because the connection from ports G4/5 of C7609 to C3550 is 2010 MB, the "Network File" Solution uses MB when the kelai network analysis system flagship version is used for packet capture, add the network segments 192.168.60.0/23 and 192.168.130.0/23 to the "local subnet" Settings, and set the data packet cache to 50 MB. The packet capture time is 2.11 seconds, the packet size is 15.629 MB, and the packet name is dht.
Similarly, for captured data packet analysis, the data packet is generally analyzed in the order of "my chart", "summary", and "diagnosis; perform detailed Fault Analysis on data packets based on the protocol, IP endpoint, physical endpoint, and IP/TCP/UDP session. Finally, combine the "diagnosis" content to summarize the host that causes the fault and the cause of the fault.
Basic Traffic Analysis
First, analyze and view the basic traffic information.
Three outstanding features are shown: The bandwidth utilization is as high as 66%; the maximum number of data packets per second is 13256; the number of large packet bytes is 12.903 MB, accounting for about 83% of the total data volume (12.903 MB/15.629 MB ).
Then, analyze the basic traffic of the donghuantuo mine network in the problematic CIDR block, and make a brief analysis of the CIDR block, as shown in:
According to the statistical data shown in the figure, the sent data volume and received traffic in the east huantuo mine network are obviously out of proportion. The sent data volume is much larger than the received data volume, and a large number of packets may cause network congestion in a short time, an exception occurs when the user accesses the internet.
According to the IP address, the highest traffic of the network segment is the internal host 192.168.60.45, accounting for 40% of the total traffic, and its sent data packets are far from receiving data packets, the sending/receiving ratio reaches 234, there are obvious exceptions.
Key host Analysis
192.168.60.45 sent more than data packets in 2 seconds. Because we caught data on the Headquarters router, it is not necessarily all data, that is, the host may send more data packets. Its data packet decoding is as follows:
It can be seen that almost all data packets have the same source IP address and source port, target IP address and target port, both of which are UDP communication packets with the source IP address 192.168.60.45: 5444 and the target IP address 59.34.198.72: 80, these Data packets are separated very short, completely equal in size, all of which are 1066 bytes, and all the "Extra Data" Data items are fill Block 41.
It can be preliminarily determined that these packets are forged data packets. The host initiates an attack to a host on the Internet through a large number of forged UDP packets at high speed, this attack greatly consumes the CPU resources of the core switch C7609 and the bandwidth resources from the East huantuo mine to the headquarters.
Other Traffic Analysis
Further analyze other host traffic to see if there are other causes that may cause network performance degradation.
Because C7609 is the core switch and serves as the gateway for up to 60 or 70 Direct Connected Networks, the traffic in these subnets will also be captured by the listener, therefore, we need to analyze the traffic of other 192 and 172 network segments except donghuantuo and linnancang to determine whether there are possible attacks.
For the 172 CIDR Block, after sorting by byte, we can see that the host traffic in this segment is very small and there is no obvious abnormal traffic, so we can exclude it from the fault source.
After sorting 192 network segments by byte, it is found that the network traffic is high, about 7.952 MB, accounting for 51% of the packet capture files (7.952/15.629 MB), but after careful observation, we found that, except for the high traffic of host 192.168.43.176, although the traffic is large, the number of hosts sending packets is large, and the traffic of each host is balanced, without the typical abnormal traffic characteristics, UDP download package.
For example, if the traffic of host 192.168.43.176 is significantly higher than that of other hosts, the communication data packets are displayed, all of which are UDP packets, with varying sizes, random segment lengths, and different IP addresses. Although the traffic of this host is significantly higher than that of other hosts, the traffic should also be the UDP download package.
No obvious abnormal traffic is found on other hosts through analysis.
Analysis conclusion
After analysis, we preliminarily determined that this fault was mainly caused by East huantuo? Too many? 92.1660.45 send a large number of forged UDP packets to the Internet host 59.34.198.72 through the core switch C7609, which on the one hand causes the M Line Blocking from donghuantuo mine to the Authority, the network access to the Headquarters and the Internet by Lin nancang and donghuantuo mining households fails. At the same time, the C7609 CPU reaches 99% due to a large number of UDP Attack Packets, causing serious performance degradation, it affects the route forwarding and connection response services of other local networks of the Group Company, resulting in slow access to the Internet by users of the entire network.
We will notify donghuantuo Mining Network Administrator to forcibly deprecate a user whose IP address is 192.168.60.45 from the local machine.
After the user is forcibly deprecated, the CPU of C7609 immediately drops to the normal range, which is about 23%. Linnan warehouse users can connect to the headquarters and access the Internet normally. At this time, the 192.168.60.2 switch of the East huantuo mine can be pinged. Other Internet functions have been restored, and the access speed of Headquarters users has also significantly increased. This confirms our analysis conclusions.