How can we prevent unfriendly search engine robot spider crawlers? Today, we found that MYSQL traffic is high on the server. Then I checked the log and found an unfriendly Spider crawler. I checked the time & nbsp; and accessed the page 7 or 8 times in one second, and accessed the website's entire site receiving page. It is not listening to query the database. I would like to ask you how to prevent such problems? & Nbsp; now I have stopped this IP address. how can I block unfriendly search engine robot spider crawlers?
Today, we found that MYSQL traffic is high on the server. Then I checked the log and found an unfriendly Spider crawler. I visited the page 7 or 8 times in one second, and accessed the website's whole site receiving page. It is not listening to query the database.
I would like to ask you how to prevent such problems? Now I have static this IP address.
------ Solution --------------------
There is no good way. You can observe which ip addresses are frequently accessed and block them if possible.
In the program, you can set the interval between two accesses from the same ip address.
