Counterfeit Google crawlers have become the third-largest DDoS attack tool

Counterfeit Google crawlers have become the third-largest DDoS attack tool

In the article Prolexic released the first quarter of 2014 Global DDoS attack report published by quickshield, we learned that the attack traffic initiated by the "Reflection amplification" technology increased by 39% compared with the previous quarter, at the same time, attackers are constantly exploring other basic Internet services to launch DDoS attacks. For example, in March this year, the security company Sucuri found that hackers used the Pingback function of over 0.162 million WordPress websites to launch large-scale DDoS amplification attacks.

Recently, new research shows that counterfeit Google crawlers have become the third-largest DDoS attack tool. Details are as follows:

Incapsula researchers found that more than 10 thousand of counterfeit Google crawlers were used for DDoS attacks after investigating 0.4 billion searches by search engines on 23% websites, 10.8% malware, spam, and scanners used to steal data.

Some highlights of the analysis results are very interesting for many SEO professionals and website operators:

Google's web crawlers are much more active than its competitors (such as MSN/Bing, Baidu and Yandex bots.

For websites that have been visited by Google Crawlers for a large number of times, their natural traffic shares will not grow, which means that Google does not take special care of the website.

On average, each website is accessed 187 times by Google crawlers every day, and the average crawling depth of each access is 4 pages. Content-intensive and frequently-updated websites, such as forums, news sites, and large e-commerce websites, are visited by crawlers more often.

Because Google is still the world's No. 1 search engine, most website operators will not block Google crawlers, but unfortunately, this also leads to the popularity of fake Google crawlers, launch DDoS attacks, plagiarize content, send junk information, and even intrude into the system.

Counterfeit Google crawlers can obtain website information as Google's identity. They use the HTTP (S) User proxy feature of Google crawlers, which is equivalent to a visitor's ID. According to the data collected by Incapsula, more than 4% of crawlers using user proxies are not actually Google crawlers.

By analyzing the data of 50 million fake Google crawlers, Incapsula found that up to 34.3% of counterfeit crawlers are malicious, of which 23.5% are used for layer-7 DDoS attacks.

Anti-DDoS attacks initiated by Google crawlers make it very difficult for website operators: they either shield all Google crawlers, disappear from search engines, or buy more bandwidth to prevent DDoS attacks.

Access to fake Google crawlers is usually from botnets. The top traffic powers are the United States (25.2%), China (15.6%, Editor's note: How ironic), Turkey (14.7% ), brazil (13.49%) and India (8.4%), while 98% of the Official Google crawlers come from the United States.

The good news is that people can now precisely identify fake Google crawlers through a series of security measures, including IP and ASN verification-a technical process for identifying crawlers by source, but unfortunately, small and Medium websites generally do not master these methods.