How HTTP Web content is prevented from being fetched in bulk

I have a website where some of the important content is being crawled in batches elsewhere
And also can update my content in real-time update speed can also be faster than my normal page loading display
Ask how to solve AH.
PS: Content is mainly text and a small number of pictures, backstage in PHP written

Reply content:

I have a website where some of the important content is being crawled in batches elsewhere
And also can update my content in real-time update speed can also be faster than my normal page loading display
Ask how to solve AH.
PS: Content is mainly text and a small number of pictures, backstage in PHP written

Try the Nginx:
1. Ngx_http_limit_conn_module can be used to limit the number of connections for a single IP
Http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html
2. Ngx_http_limit_req_module can be used to limit the number of requests per second for a single IP
Http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
3. Nginx_limit_speed_module can be used for IP speed limit
Https://github.com/yaoweibin/nginx_limit_speed_module

Or use the Nginx Fail2ban module: http://www.fail2ban.org/wiki/index.php/Main_Page

In addition to the above can also use pictures/url anti-theft chain

