How to prevent unfriendly search engine bots and spider crawlers-php Tutorial

Source: Internet
Author: User
How can we prevent unfriendly search engine robot spider crawlers? Today, we found that MYSQL traffic is high on the server. Then I checked the log and found an unfriendly Spider crawler. I checked the time & nbsp; and accessed the page 7 or 8 times in one second, and accessed the website's entire site receiving page. It is not listening to query the database. I would like to ask you how to prevent such problems? & Nbsp; now I have stopped this IP address. how can I block unfriendly search engine robot spider crawlers?
Today, we found that MYSQL traffic is high on the server. Then I checked the log and found an unfriendly Spider crawler. I visited the page 7 or 8 times in one second, and accessed the website's whole site receiving page. It is not listening to query the database.
I would like to ask you how to prevent such problems? Now I have static this IP address.


------ Solution --------------------
There is no good way. You can observe which ip addresses are frequently accessed and block them if possible.

In the program, you can set the interval between two accesses from the same ip address.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.