Collection and prevention of collection is a prolonged topic, on the one hand want to engage in other people's things, on the other hand do not want their own things to get away.
This article describes how to use Nginx shielding IP to achieve the prevention of collection, of course, can also be achieved through iptable.
1. Find the IP to be screened
awk ' {print $} ' Nginx.access.log |sort |uniq-c|sort-n
Nginx.access.log is a log file,
will be to the following results, the previous IP access number, followed by IP, it is obvious that we need to visit the number of IP and not the spider's IP shielding, in this case, we block out
165.91.122.67 ...
13610 202.112.113.192
95772 180.169.22.135
337418 219.220.141.2
558378 165.91.122.67
2. Under the Nginx installation directory, the new screen IP file, named Blockip.conf, after the new increase shielding IP only need to edit this file. Add the following content
3. Add the following configuration to the Nginx configuration file nginx.conf, which can be placed in HTTP, server, location, limit_except statement block, and need to pay attention to relative path, In this case, nginx.conf,blocksip.conf is in the same directory.
4. Restart the Nginx service:/usr/local/nginx/nginx-s reload can be effective.
Advanced usage:
A configuration file that masks IP can either mask a single IP or block IP segments, or only allow an IP or an IP segment to be accessed.
# Mask Single IP access
deny IP;
# Allow single IP access to
allow IP;
# Masks All IP access
deny all;
# Allow all IP access
allow all;
#屏蔽整个段即从123.0.0.1 to 123.255.255.254 Access command
deny 123.0.0.0/8
#屏蔽IP段即从123.45.0.1 to 123.45.255.254 Access command
Deny 124.45.0.0/16
#屏蔽IP段即从123.45.6.1 to 123.45.6.254 Access command
deny 123.45.6.0/24
If you want to implement such an application, except for a few IP, all the other Rejects,
That would require you to write that in blockip.conf.
Allow 1.1.1.1;
Allow 1.1.1.2;
Deny all;
Individual website shielding IP method, include blocksip.conf; Put to the URL corresponding in the server{} statement block,
All Web site shielding IP methods, include blocksip.conf; to the HTTP {} statement block.
Mask access too frequent IP scripts need to be modified according to the actual nginx log format, remove access to IP and user-agent.
Add a configuration to the Nginx configuration file
Include./vhost/blockip.conf;
Script content:
#!/bin/bash
nginx_home=/usr/local/webserver/nginx
log_path=/usr/local/webserver/nginx/logs
Tail- N50000/usr/local/webserver/nginx/logs/access.log \
|awk ' {print $1,$12} ' \
|grep-i-v-e ' google|yahoo|baidu| Msnbot| Feedsky|sogou "\
|awk ' {print} ' |sort|uniq-c|sort-rn \
|awk ' {if ($1>1000) print" Deny "$"; "} ' > $nginx _ home/conf/vhost/blockip.conf
/etc/init.d/nginx Reload