Real case: DOS attacks on websites
DOS attacks on the ghost website I. Event background
Long holidays are a short vacation period for IT staff, but it systems cannot be stopped for a while. The more holidays, the more likely a major problem may occur, the following is a case of DOS attacks.
After the Spring Festival holiday, Xiao Li's Web server went wrong. One o'clock P.M., after dinner, Xiao Li habitually checked the Web server. The Web server's traffic monitoring system displays a red downward curve, and receives an email alarm to determine the server's status.
According to the above problems, Xiao Li immediately began to check the Web server logs and try to find some clues about the interruption. During the process of querying clues, the department manager told Mr. Li that he had received a complaint from the customer saying he could not access their website.
No suspicious information is found in the log file of the Web server. Therefore, Mr. Li looks at the firewall log and router log in detail. Print the records of the server's problems, filter out normal traffic, and keep suspicious records. Table 1 shows the printed results.
Table 1 firewall log statistics
Source IP Address |
Destination IP address |
Source Port |
Destination Port |
Protocol |
172.16.45.2 |
192.168.0.175 |
7843 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
10.168.45.3 |
192.168.0.175 |
34511 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
192.168.89.111 |
192.168.0.175 |
1783 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
10.231.76.8 |
192.168.0.175 |
29589 |
7 |
17 |
192.168.15.12 |
192.168.0.175 |
17330 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
172.16.43.131 |
192.168.0.175 |
8935 |
7 |
17 |
10.23.67.9 |
192.168.0.175 |
22387 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
192.168.57.2 |
192.168.0.175 |
6588 |
7 |
17 |
172.16.87.11 |
192.168.0.175 |
21453 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
10.34.67.89 |
192.168.0.175 |
45987 |
7 |
17 |
10.65.34.54 |
192.168.0.175 |
65212 |
7 |
17 |
192.168.25.6 |
192.168.0.175 |
52967 |
7 |
17 |
172.16.56.15 |
192.168.0.175 |
8745 |
7 |
17 |
10.18.18.18 |
192.168.0.175 |
19 |
7 |
17 |
He did the same job on the router log and printed the abnormal records. In table 5-1, the router logs generated after the website is attacked are normalized.
For more information, Xiao Li went on to view the comprehensive statistics of NetFlow in the vro. The details are as follows:
For reference, he also printed the cached data (normal data) that he stored in the previous weeks before the Web server began to experience problems ). The normal routing log is as follows: the two lines under the IP packet size distribution title show the percentage of data packets distributed by size range. The content shown here indicates that only 2% of data packets are in the size of 33 ~ Between 64 bytes.
Note: The website traffic decreases linearly. Obviously, no one can access his Web server during this time. Mr. Li began to study what happened and how to fix the fault as soon as possible.
Ii. FAQ
1. What happened to Xiao Li's Web server? What are possible attack types?
2. if the address is not disguised, how can Xiao Li trace the attacker?
3. If the address is disguised, how can he trace the attacker?
Iii. Event Reasoning
What kind of attacks does Xiao Li's Web Server suffer? This attack is achieved by constantly sending UDP packets to the echo port (echo port number is 7. The attack seems to come from two places, probably because two attackers use different tools at the same time. In any case, overloaded data streams will drag the Web server down. However, the attack Address Source is uncertain. I don't know whether the attack source is distributed, or whether the same real address disguise many different fake IP addresses. This problem is hard to judge. If the source IP address is not disguised, You can consult the arini us Internet number registry to find out which network the intrusion IP address belongs to from its "Whois" database. Next, you only need to contact the administrator of the network to obtain further information. However, this is unlikely to cause DOS attacks.
If the source address is disguised, it would be much more difficult to trace the attacker. If you are using a Cisco router, you also need to query the NetFlow cache. However, to track this disguised address, you must query the NetFlow cache on each vro to determine which interface the traffic enters. Then, you can trace the traffic one by one through these vro interfaces, until the IP Address Source is found. However, this is very difficult, because there may be many routers between the Web Server and the attacker's initiating PC, and they belong to different organizations. In addition, these analyses must be performed while the attack is ongoing. It is difficult to find the source without the involvement of the judicial department.
After analysis, the information in the firewall log and the router log is associated and some interesting similarities are found, such as the bold black mark in Table 5-1. The target of the attack is obviously the Web server (192.168.0.175, port is UDP 7. This looks like a Denial of Service attack (but it cannot be determined because the source IP address of the attack is randomly distributed ). The address looks random. Only one source address remains fixed, and its source port number remains unchanged. This is interesting. He then concentrated his attention on the router log.
He found that there were a large number of 64-byte packets in the router log when the attack occurred, and there was no problem in the Web server log at this time. He also found that there were a large number of "UDP-other" packets in the router log at the time of the incident, and the Web server logs were all normal. This phenomenon is consistent with the assumption of UDP-based DoS attacks.
In this case, it can be assumed that the attacker uses many small UDP packets to launch a flood attack on the echo 7 port of the Web server, therefore, Xiao Li's next task is to stop this attack. First, Xiao Li blocks attacks on the vro. Quickly set a filter rule for the vro. Because the source address is random, they think it is difficult to restrict an address or a range of addresses to prevent attacks. Therefore, they decided to prohibit all UDP packets sent to 192.168.0.175. This will cause the server to lose some features, such as DNS, but at least make the Web server work normally.
The initial temporary DOS access control list (ACL) of the router is as follows:
Access-list 121 remark Temporary block DoS attack on web server 192.168.0.175
Access-list 105 deny udp any host 192.168.0.175
Access-list 105 permit ip any
This method reduces the burden on the Web server, but the attack can still reach the Web, thus reducing the network performance to a certain extent. The next step is to contact the upstream bandwidth provider and ask them to temporarily limit the incoming UDP traffic on port 7 of Xiao Li's website. This will significantly reduce the traffic on the network to the server.
Iv. Targeted measures
There is no panacea for preventing and mitigating bandwidth-related DOS attacks. In essence, this is an attack that "uses a thick pipe to defeat a thin pipe. Attackers can "direct" more bandwidth, sometimes even a huge bandwidth, to defeat networks with insufficient bandwidth. In this case, prevention and mitigation should complement each other.
There are many ways to make the attack more difficult, or reduce the impact when the attack occurs, as shown below:
Network entry filtering network service providers should set entry filtering on their downstream networks to prevent fake information packets from entering the network. This will prevent attackers from disguising IP addresses for easy tracking. The network traffic filtering software filters out unnecessary network traffic. This can also prevent DOS attacks, but in order to achieve the effect, these filters should be set as far as possible in the upstream network.
Network Traffic rate limit. Some routers have the maximum traffic rate limit. These restrictions will enhance the bandwidth policy and allow a given type of network traffic to match a limited bandwidth. This measure can also mitigate ongoing attacks in advance.
Intrusion Detection System and host monitoring tools. IDS can warn the network administrator of the Attack Time and the attack tool used by the attacker, which can help prevent the attack. The host monitoring tool can warn the administrator of the existence of the DOS tool.
Spof RPF (Reverse Path Forwarding), which is used by CEF (Cisco Express Forwarding function for short) to check another characteristic of packets received on the interface. If the source IP address CEF table does not have the same route as the interface pointing to the received data packet, the router will lose the data packet. The magic of dropping RPF is that it blocks all attacks that disguise source IP addresses.
1) DOS attack detection
Using host monitoring system and IDS system joint analysis, you can quickly find problems, such as using EtherApe (an open-source tool for monitoring connections). Of course, sniffer Pro and colai network analysis tools can achieve the same effect. Sniffer can display network connection conditions in real time. In case of DOS attacks, it can preliminarily determine the attack type from its internal dense connections and IP addresses, in this case, we can use the traffic monitoring software in the Ossim system, such as Ntop and IDS systems, for careful judgment. The latter two will be explained in detail in "Unix/Linux Network Log Analysis and traffic monitoring. The quickest way is to use the command line. Enter the following command:
# Netstat-an | grep SYN_RECV | wc-l
The results show that a large number of TCP synchronization data packets exist in the network, but few TCP connections are successfully established. According to the analysis of the TCP three-way handshake principle, this is certainly not a normal phenomenon, and the network must have problems, further investigation is required. If the value is very high, for example, if it reaches thousands of values, it is likely to be attacked. 1.
In Figure 1, the Snort in the OSSIM system detects DOS attacks and displays a large number of alarms graphically. For example, when a website is under DOS attack, the TCP connection is as follows:
To count the number of SYN_RECV states, run the following command:
# Netstat-na | grep SYN_RECV | wc-l
1989
In combination with the above 5-1 figure, we can determine that the website is under DOS attack.
TIPS: You can also use the following Shell command to display which IP Address has the most connections.
# Netstat-nta | awk '{print $5}' | cut-d: f1 | sort | uniq-c | sort-n
1 192.168.150.10
2 192.168.150.20
... ...
1987 192.168.150.200
The information obtained by this command is more detailed. The number reaches 1989, and there are nearly two thousand records, which clearly indicates that it was under DOS attack. At this time, we can use Wireshark tool for data packet decoding to solve more problems. Currently, all communication uses the TCP protocol. Check that all data packets sent by the TCP flag are SYN set to 1, that is, TCP synchronous request data packets, these packets usually point to the same IP address. So far, we can verify the above judgment: This host is under DOS attack, and the attack method is SYN Flood attack.
V. troubleshooting
1. Xiao Li's server is under DOS attack, which is achieved by continuously sending small UDP packets to port 7. The attack appeared to have originated from two locations, probably because two attackers used different tools. A large amount of data streams quickly drag the Web server. The difficulty lies in the uncertainty of the attack address source, the distribution of the attack source, and the uncertainty of many different IP addresses disguised by the same address.
2. If the IP address is not disguised, Xiao Li queries ARIN and finds out from its Whois database which network the intrusion IP address belongs.
3. if the IP address is disguised, This tracing is troublesome. You need to query the NetFlow data on each vro to determine the interfaces for inbound and outbound traffic, then, one interface of these routers is traced back and forth until the IP Address Source is found. However, this involves multiple AS (Autonomous Systems). If you are looking for the attack source in China
The process often involves many operators, as well as the judiciary, and the workload and time will be extended. If cross-country tracing is involved, it will be more complicated. The most difficult thing is that accurate analysis can be performed only during the attack period. Once the attack ends, you have to go to the log system for query.
After reading the actual cases above, we also learned that many DoS attacks are very difficult to cope with, because the number of requests sent by the host involved in the destruction is completely legal and compliant with the standard, but the number is too large. We can first Block ICMP echo requests on the vro with the appropriate ACL.
Router (config) # ip tcp intercept list 101
Router (config) # ip tcp intercept max-incomplete high 3500
Router (config) # ip tcp intercept max-incomplete low 3000
Router (config) # ip tcp intercept one-minute high 2500
Router (config) # ip tcp intercept one-minute low 2000
Router (config) # access-list 101 permit any
If Context-Based Access Control (CBAC) can be used, you can use its timeout and threshold settings to cope with SYN flood and UDP spam flood. For example:
Router (config) # ip inspect tcp synwait-time 20
Router (config) # ip inspect tcp idle-time 60
Router (config) # ip inspect udp idle-time 20
Router (config) # ip inspect max-incomplete high 400
Router (config) # ip inspect max-incomplete low 300
Router (config) # ip inspect one-minute high 600
Router (config) # ip inspect one-minute low 500
Router (config) # ip inspect tcp max-incomplete host 300 block-time 0
Warning TCP interception and CBAC defense are not recommended because this may cause router overload.
Enabling the Cisco Express Forwarding (CEF) function helps the router defend against the flood of packets with random source addresses. You can make some settings for the scheduler to avoid the router's CPU overload under the impact of a flood:
Router (config) # sched1_allocate 3000 1000
After configuration, IOS will process network interface interruption requests in 3 seconds, and then execute other tasks in 1 second. For earlier systems, the command scheduler interval <milliseconds> may be required.
Another method is to use Iptables to prevent DOS scripts.
#! /Bin/bash
Netstat-an | grep SYN_RECV | awk '{print $5}' | awk-F: '{print $1}' | sort | uniq-c | sort-rn | awk '{if ($1> 1) print $2 }'
For I in $ (cat/tmp/dropip)
Do
/Sbin/iptables-a input-s $ I-j DROP
Echo "$ I kill at 'date'">/var/log/ddos
Done
This script calculates the number of SYN_RECV IP addresses that have reached 5, and sets the INPUT chain written to Iptables as rejected.
Vi. Case Summary
No matter the purpose of launching a larger scale attack or other DOS/DDoS attacks, we must pay attention to it. To prevent such attacks, install patches from the vendor in a timely manner. In addition, you must disable the service with vulnerabilities or use the access control list to restrict access. Common DOS attacks, especially DDOS attacks, are more difficult to prevent. If the bandwidth is exhausted by the Ping flood, what we can do is limited. To defend against DOS attacks, we must first analyze the attack methods, such as ICMP Flood, UDP Flood, SYN Flood, and other traffic attacks, or similar to TCP Flood and CC attacks, then find a relatively effective response policy. The following methods can be used for this attack:
1) use the "Honey network" protection to enhance the immediate analysis and response to attack tools and malicious samples. Large-scale deployment of honey network devices to track the dynamics of botnets and capture malicious code. Deploy website operation monitoring devices to enhance the monitoring of webpage Trojans, access redirection mechanisms, and domain name resolution, and cut off the main paths of malicious code infection. Use automatic malicious code analysis devices with sandbox and various shelling technologies to strengthen research on new malicious code and improve the timeliness of research.
2). The Apache Dos Protection Policy provided by the Ossim system can be used for monitoring.
3) Improve the efficiency of detecting and protecting new attacks, especially application-layer attacks and low-rate attacks, by using new technology platforms such as cloud computing and virtualization. Some foreign scholars have begun to use the Hadoop platform to study the Http Get Flood detection algorithm.
4). Use the IP reputation mechanism. Credibility mechanisms are introduced in all aspects of information security protection to improve the efficiency and accuracy of security protection. For example, you can evaluate the security credibility of applications and files to guide network users in downloading behaviors. by publishing authoritative IP address credibility information, you can instruct security devices to automatically generate protection policies, for details, see section 2.1 of Unix/linux Network Log Analysis and traffic monitoring.
5) using a passive policy means purchasing large bandwidth, which can effectively reduce the harm of DDOS attacks.
6 ). build a distributed system, deploy your business in multiple data centers, distribute access from different regions to the corresponding data centers, and deploy CDN, deploying firewalls (such as Cisco and Juniper firewalls) in data centers of important IDC nodes is only one of the data centers, and does not affect the entire business even if attackers perform DOS attacks.
7 ). if the scale is not big, the machine room condition is general, it can consider using some anti-DDos gadgets in the system, such as DDoS Deflate, its official website address is http://deflate.medialayer.com, it is a free script used to defend against and mitigate DDOS attacks. It uses the built-in netstat command to monitor and track the IP addresses that create a large number of network connections, when detecting that a node exceeds the preset limit, the program will disable or block these IP addresses through the filters or IPTABLES. Of course, this tool is only a relief tool and cannot completely prevent attacks.
At last, we need to use different vendors, different AS paths, and support more than one connection to the Internet for Server Load balancer, but there is still a gap between this and the requirements for dealing with high-bandwidth conventional DOS/DDOS flood. We can always use CAR (Committed Access Rate, promised Access Rate) or NBAR (Network-Based Application Recognition, Network Application Recognition) to discard data packets or limit the speed of Network streams that initiate attacks, reduces the burden on the vro CPU and the occupation of the buffer zone and the host after the vro.