Real case: A Dos attack on the website

Web site encounters a Dos attackI. Background of the event

Long vacation for IT staff is a short period of recuperation, but it system can not stop, the more holidays, the more likely to be a big problem, the following is a case of a Dos attack.

Spring Festival holiday just finished, Xiao Li Company's web server on the fault. 1 o'clock in the afternoon, after dinner back, Xiao Li habitually checked the Web server. Web server traffic Monitoring system displays the downward red curve, at the same time received a mail alarm, you can determine the status of the server.

According to the above questions, Xiao Li immediately began to check the Web server log, trying to find some clues about the interruption caused. In the course of inquiry clues, the department manager told Xiao Li that he had received a complaint from customers, saying that they could not access their website.

There was nothing suspicious in the log files of the Web server, so Xiao Li looked closely at the firewall logs and router logs. Print out the record of the server when the problem occurs, and filter out the normal traffic, keeping the suspicious records. Table 5-1 shows the printed results.

Table 1 Firewall Log statistics

Source IP Address	Destination IP Address	Source Port	Destination Port	Agreement
172.16.45.2	192.168.0.175	7843	7	17
10.18.18.18	192.168.0.175	+	7	17
10.168.45.3	192.168.0.175	34511	7	17
10.18.18.18	192.168.0.175	19	7	17
192.168.89.111	192.168.0.175	1783	7	17
10.18.18.18	192.168.0.175	19	7	17
10.231.76.8	192.168.0.175	29589	7	17
192.168.15.12	192.168.0.175	17330	7	17
10.18.18.18	192.168.0.175	19	7	17
172.16.43.131	192.168.0.175	8935	7	17
10.23.67.9	192.168.0.175	22387	7	17
10.18.18.18	192.168.0.175	19	7	17
192.168.57.2	192.168.0.175	6588	7	17
172.16.87.11	192.168.0.175	21453	7	17
10.18.18.18	192.168.0.175	19	7	17
10.34.67.89	192.168.0.175	45987	7	17
10.65.34.54	192.168.0.175	65212	7	17
192.168.25.6	192.168.0.175	52967	7	17
172.16.56.15	192.168.0.175	8745	7	17
10.18.18.18	192.168.0.175	19	7	17

He did the same work on the router log and printed a record that looked unusual. In table 5-1, the router log information is normalized after the site has been compromised.

In order to get more information, Xiao Li then looked at the router's NetFlow comprehensive statistics, details as follows:

650) this.width=650; "Style=" background-image:none;padding-left:0px;padding-right:0px;border-top-width:0px; border-bottom-width:0px;border-left-width:0px;padding-top:0px; "title=" clip_image002 "border=" 0 "alt=" clip_ image002 "src=" http://img1.51cto.com/attachment/201410/17/350944_1413554642WHuQ.jpg "width=" 501 "height=" 581 "/ >

In order to have a reference, he also prints the cached data (which is the normal state of the data) that he saved in the first few weeks of the Web server beginning to issue. The normal routing log is as follows:

650) this.width=650; "Style=" background-image:none;padding-left:0px;padding-right:0px;border-top-width:0px; border-bottom-width:0px;border-left-width:0px;padding-top:0px; "title=" clip_image003 "border=" 0 "alt=" clip_ image003 "src=" http://img1.51cto.com/attachment/201410/17/350944_1413554644Y3JU.jpg "width=" 416 "height=" 536 "/ >

IP packet size distribution The two lines under this heading show the percentage of packets distributed by size range. The content shown here shows that only 2% of the packet size is between 33~64 bytes.

Note that the number of visits to the website has plummeted. It is clear that no one has access to his Web server during this period of time. Xiao Li began to study exactly what had happened and how to fix the fault as soon as possible.

Second, difficult questions and answers

1. What happened to Xiao Li's Web server? What are the possible types of attacks?

2. If the address is not disguised, how can Xiao Li trace the attacker?

3. If the address has been disguised, how can he track the attacker?

Event Inference

What kind of attack did Xiao Li's Web server suffer from? This attack is done by continuously sending UDP packets through a echoing port (Echo Port number 7). The attack appears to have originated from two locations, possibly two attackers using different tools at the same time. In any case, overloaded traffic will drag down the Web server. However, the attack address source is not certain, do not know whether the attack source itself is distributed, or the same real address disguised many different false IP addresses, this problem is more difficult to judge. If the source IP address is not spoofed, you can consult the Arini US Internet number registry to find out which network the intrusion IP address belongs to from its "Whois" database. Then just contact the administrator of that network for further information, but this is unlikely to be a Dos attack.

If the source address is disguised, it is much more troublesome to trace the attacker. If you are using a Cisco router, you also need to query the NetFlow cache. However, in order to trace the spoofed address, you must query the NetFlow cache on each router to determine which interface the traffic is entering, and then trace back through these router interfaces until the IP address source is found. This is difficult, however, because there may be many routers between the Web server and the attacker's originating PC and belong to different organizations. In addition, these analyses must be done while the attack is in progress. It's hard to find the source if it wasn't for the Justice Department involved.

After analysis, the firewall logs and the information in the router log were correlated, and some interesting similarities were found, as shown in table 5-1 in bold black markings. The target of the attack is obviously the Web server (192.168.0.175, the port is UDP 7. This looks much like a denial of service attack (but is not yet certain because the source IP address of the attack is distributed randomly). The address appears to be random, and only one source address is fixed, and its source port number is not changed. This is interesting. He then focused on the router log.

He found that there were a large number of 64-byte packets on the router log when the attack occurred, and there were no problems with the Web server logs. He also found that there were a lot of "udp-other" packets in the router log when the incident occurred, and that the Web server logs were all normal. This behavior is consistent with the assumption of UDP-based denial-of-service attacks.

At this point, it can be assumed that the attacker was using many small UDP packets to flood the Echo 7 port of the Web server, so Xiao Li's next task was to stop the attack. First, Xiao Li intercepted the attack on the router. Quickly set up a filtering rule for the router. Because source addresses are randomly sourced, they find it difficult to block an attack by restricting an address or a range of addresses, and therefore decide to prohibit all UDP packets sent to 192.168.0.175. This can cause the server to lose some functionality, such as DNS, but at least allow the Web server to work properly.

The router's initial temporary DOS access control List (ACL) is as follows:

Access-list 121 remark Temporary block DoS attack on Web server 192.168.0.175

Access-list deny UDP any host 192.168.0.175

Access-list Permit IP any any

This reduces the burden on the Web server, but the attack can still reach the web, reducing network performance to some extent. So the next step is to contact the upstream bandwidth provider, and want them to temporarily limit all of the small Lee's website Port 7 UDP traffic, this will significantly reduce the network to the server traffic.

Targeting measures

There is no magic bullet for preventing and mitigating this bandwidth-related DOS attack. Essentially, this is a "thick pipe to defeat the thin pipe" attack. Attackers can "instruct" more bandwidth, and sometimes even huge bandwidth, to overwhelm networks with insufficient bandwidth. In this context, prevention and mitigation should complement each other.

There are many ways to make an attack more difficult to take place, or to reduce its impact when an attack occurs, as follows:

The Network portal filtering Network service provider should set up ingress filtering on his downstream network to prevent false packets from entering the network. This prevents attackers from disguising IP addresses, making them easy to track. Network traffic filtering software filter out the network does not need traffic is always not wrong. This also prevents Dos attacks, but in order to achieve the effect, these filters should be set up upstream of the network as far as possible.

Network traffic rate limit. Some routers have the highest traffic rate limit. These restrictions will strengthen the bandwidth policy and allow a given type of network traffic to match a limited bandwidth. This measure can also pre-empt ongoing attacks.

Intrusion detection system and host monitoring tool. IDS can warn network administrators of the timing of attacks and attack tools used by attackers, which can help prevent attacks. The host monitoring tool can warn the administrator if a DOS tool appears in the system

Unicast RPF (Reverse Path Forwarding), which is the CEF (router's Cisco Express Forwarding feature abbreviation), is used to check for another feature of the packets received on the interface. If the source IP address on the CEF table does not have a route that is consistent with the interface to the receiving packet, the router loses the packet. The beauty of discarding RPF is that it blocks all attacks that disguise the source IP address.

1 ) Detection DOS Attack

With the host monitoring system and the IDS system federated analysis, problems can quickly be identified, such as through the EtherApe tool (an open source tool for monitoring connectivity), and of course, the same effect can be achieved with sniffer pro as well as with the KE Network analysis tool. Sniffer can display the network connection situation in real-time, if encounter Dos attack, from its inside dense line, as well as the IP address can initially determine the attack type, then can use the OSSIM system of traffic monitoring software such as NTOP, and IDS system to carefully judge. The latter two will be explained in detail in the "Unix/linux Network log analysis and Traffic monitoring" book. The quickest way is the command line, we enter the following command:

# Netstat-an|grep Syn_recv|wc–l

Through the results can be found in the network there are a large number of TCP synchronization packets, and the successful establishment of TCP connection is very few, according to the TCP three handshake principle analysis, this is certainly not normal phenomenon, network must have problems, need to further verify, if the value is high, for example, to reach thousands of values, Then there is a good chance of being attacked. As shown in 1.

650) this.width=650; "Style=" background-image:none;margin:0px;padding-left:0px;padding-right:0px; border-top-width:0px;border-bottom-width:0px;border-left-width:0px;padding-top:0px; "Title=" clip_image005 " Border= "0" alt= "clip_image005" src= "Http://img1.51cto.com/attachment/201410/17/350944_1413554646b5au.png" width= " 535 "height=" 376 "/>

Figure 1 Ossim Discovery Dos attack

In Figure 1, snort in the Ossim system detects a Dos attack and graphically displays a large amount of alarm information. For example, a Web site with a Dos attack has the following TCP connection:

650) this.width=650; "Style=" background-image:none;padding-left:0px;padding-right:0px;border-top-width:0px; border-bottom-width:0px;border-left-width:0px;padding-top:0px; "title=" clip_image006 "border=" 0 "alt=" clip_ image006 "src=" Http://img1.51cto.com/attachment/201410/17/350944_1413554647tx0Q.png "width=" 535 "height=" 172 "/ >

We count the number of "syn_recv" states, with the following command:

#netstat –na |grep syn_recv |wc–l

1989

Such a large number, in conjunction with the above 5-1 graphics can be judged by a Dos attack site.

Tip: You can also use the following Shell command to show which IP connection is the most.

#netstat-nta |awk ' {print $} ' |cut–d:f1 |sort|uniq–c |sort–n

1 192.168.150.10

2 192.168.150.20

... ...

1987 192.168.150.200

The information given in this command is more detailed. The value reached 1989, with nearly 2000, which clearly indicates a Dos attack. At this time we use the Wireshark tool for packet decoding can be more problematic, the current communication is all using the TCP protocol, view TCP flag sent all packets are SYN 1, that is, TCP synchronous request packets, and these packets tend to point to the same IP address. It is possible to verify the above judgment: this host suffers a Dos attack, and the attack is SYN flood attack.

Troubleshooting

1. Xiao Li's server was attacked by a Dos attack that was implemented by sending small UDP packets to port 7 continuously. The attack appears to originate in two locations, and it is likely that two attackers use a different tool. A large amount of traffic quickly drags down the Web server. The difficulty is that the attack address source is not certain, the attack source itself is distributed, or the same address is disguised as many different IP addresses are not OK.

2. Assuming that the address is not disguised, Xiao Li queries arin to find out from its WHOIS database which network the intrusion IP address belongs to.

3. If the IP address is spoofed, this trace is cumbersome, it is necessary to query the NetFlow data on each router to determine which interfaces the traffic is in and out, and then to trace the query back to the router one interface at a time until the originating IP address source is found. But this involves multiple as (autonomous systems), if the source of its attack is sought at home

Process often involves many operators, as well as the judiciary, where the workload and time are prolonged and if cross-border tracing is involved is more complex. The hardest thing to do is to have an accurate analysis during an attack, and once the attack is over, you have to check the log system.

See the actual case above we also understand that many Dos attacks are difficult to deal with, because the destruction of the host to make the request is completely legal, standard, but the number is too large. We can block ICMP echo requests with appropriate ACLs on the router first.

Router (config) #ip TCP intercept list 101

Router (config) #ip tcp intercept Max-incomplete high 3500

Router (config) #ip tcp intercept Max-incomplete low 3000

Router (config) #ip tcp intercept One-minute high 2500

Router (config) #ip TCP intercept One-minute low 2000

Router (config) #access-list 101 Permit any

If you can take context-based access control (contextual Based access Control,cbac), you can use its time-out and threshold settings to handle the SYN torrent and udp garbage torrent. For example:

Router (config) # IP inspect TCP synwait-time 20

Router (config) # IP inspect TCP idle-time 60

Router (config) # IP inspect UDP idle-time 20

Router (config) # IP inspect Max-incomplete high 400

Router (config) # IP inspect max-incomplete low 300

Router (config) # IP inspect one-minute high 600

Router (config) # IP inspect one-minute low 500

Router (config) # IP inspect TCP max-incomplete host Block-time 0

Warning: It is not recommended to use both TCP intercept and CBAC defenses, as this may cause the router to overload.

The Cisco Express Forwarding (Cisco Express FORWARDING,CEF) feature is turned on to help the router defend the torrent of packets for random source addresses. The scheduler can be set up to avoid the full CPU overload of the router under the impact of torrent:

Router (config) #scheduler allocate 3000 1000

After the configuration, iOS takes 3 seconds to process the network interface interrupt request, and then takes 1 seconds to perform other tasks. For older systems, you may have to use the command scheduler interval<milliseconds>.

Another approach is to use iptables to prevent DOS scripting

#!/bin/bash

Netstat-an|grep Syn_recv|awk ' {print$5} ' |awk-f: ' {print$1} ' |sort|uniq-c|sort-rn|awk ' {if (>1) print $ '

For I in $ (CAT/TMP/DROPIP)

Do

/sbin/iptables-a input-s $i-j DROP

echo "$i kill at ' Date '" >>/var/log/ddos

Done

The script counts the IP in SYN_RECV and the number of 5, and sets the input chain written to Iptables to reject.

Case Summary

Dos/ddos attacks must be taken seriously, whether for any purpose, for a larger attack or for other purposes. The main way to prevent this attack is to hit the patch from the manufacturer in time. Also, either close the vulnerable service or restrict access with the access control list. Regular Dos attacks, especially DDoS attacks, are more difficult to guard against. If the entire bandwidth is exhausted by the ping torrent, we can do it very limited. For DOS attack, the first is to analyze its attack mode, is ICMP Flood, UDP Flood and Syn Flood and other traffic attacks, or similar to TCP Flood, CC and other ways, and then look for a relatively effective response strategy. There are several approaches to this attack:

1). Use "Honey net" protection to enhance the first time analysis and response to attack tools and malicious samples. Large-scale deployment of honey network devices to track botnet dynamics and capture malicious code. Deploy the site to run monitoring equipment, strengthen the Web page horse, access redirection mechanism and domain name resolution monitoring, cut off the main infection path of malicious code. The use of the sandbox technology and a variety of shelling technology of malicious code automation analysis equipment, strengthen the research of new malicious code, improve the timeliness of research.

2). The Apache DOS protection strategy provided by the OSSIM system can be used as a monitoring function.

650) this.width=650; "title=" tmp-1.jpg "alt=" wkiom1rbkoetcnllaamyj_kcq90126.jpg "src=" http://s3.51cto.com/wyfs02/ M01/4c/9d/wkiom1rbkoetcnllaamyj_kcq90126.jpg "/>

3). Leverage new technology platforms such as cloud computing and virtualization to improve the efficiency of detection and protection of new attacks, especially application-level attacks and low-rate attacks. Foreign scholars have begun to use the Hadoop platform for HTTP Get flood detection algorithm research.

4). Use the IP reputation mechanism. In all aspects of information security protection to introduce a reputation mechanism, improve the efficiency and accuracy of security protection. For example, the application software and files to give security reputation evaluation, guide network users download behavior, through the issuance of authoritative IP reputation information, guide security equipment to automatically generate protection strategy, see "Unix/linux Network log analysis and Traffic Monitoring" section 2.1.

5). Using a passive strategy to buy large bandwidth can also effectively mitigate the threat of DDoS attacks.

6). Build a distributed system, deploy your own business in a multi-ground room, spread the access of each area to the corresponding room, consider deploying CDN, deploy a firewall in the important IDC node room (such as Cisco, Juniper Firewall, etc.) so that even if an attacker Dos attack, The scope of destruction may also be just one of the rooms, without impacting the entire business.

7). If the scale is small, the room condition is general, that may consider in the system to use some anti-DDoS the small tool, like the DDoS Deflate, its official website address is http://deflate.medialayer.com, It is a free script to defend against and mitigate DDoS attacks through the system's built-in netstat command to monitor IP addresses that create a large number of network connections, which are banned or blocked by APF or iptables when a node is detected that exceeds a preset limit. Of course, this tool is only mitigated and cannot be fully protected against attacks.

Finally, there is more than one connection to the Internet with different vendors, different as paths, and support for load balancing, but there is still a gap between the requirements of a conventional dos/ddos torrent that consumes high bandwidth. We can always use car (Committed access rate, committed access rates) or Nbar (network-based application recognition, network application recognition) to abandon the packet or limit the speed of the network stream that launched the attack, Reduces the burden on the router's CPU, reducing the footprint of the buffer and the host behind the router.

This article from "Lee Chenguang original Technology blog" blog, declined reprint!

Real case: A Dos attack on the website