Nginx + lua + redis implement verification code anti-collection

Nginx + lua + redis implement verification code anti-collection

I have introduced how to embed the lua module in nginx. Using nginx + lua can develop nginx business logic well and achieve high concurrency.

The following describes how to use nginx + lua + redis to implement the anti-collection function.

Symptom:

Websites are constantly crawled by search engines and collectors while providing services to users. This may cause the website to be overwhelmed and cause the error of returning the page to 5XX. In this case, we need to control the access of collectors and search engines. Of course, the control of search engines may affect the indexing of websites.

Function Description:

Nginx + lua implements Client Access Control on the front end, and records client access information in redis. If the Access frequency limit is exceeded, the verification code page generated by php is displayed; if the verification succeeds, you can continue the access for half an hour. If the verification fails, the access will be blocked for half an hour. The collector's ip address may change, so it will not be blocked until now.

1. Install nginx_lua Module

See the previous blog http://blog.csdn.net/yanggd1987/article/details/46679989 for nginx and lua

2. lua-resty-redis module Installation

 

Cd/usr/local/srcwget https://github.com/openresty/lua-resty-redis/archive/master.zipunzip master.zip cd lua-resty-redis-mastermkdir-p/usr/local/nginx/lua # copy lib to the lua folder under the nginx installation directory cp-rf lib/usr/local/nginx/luacd/usr/local/nginx/lua/libln-s redis. lua resty/redis. lua

3. Write a lua script in the nginx directory

 

 

Cd/usr/local/srcwget https://github.com/openresty/lua-resty-redis/archive/master.zipunzip master.zip cd lua-resty-redis-mastermkdir-p/usr/local/nginx/lua # copy lib to the lua folder under the nginx installation directory cp-rf lib/usr/local/nginx/luacd/usr/local/nginx/lua/libln-s redis. lua resty/redis. luacd/usr/local/nginx/luavim access_test.luapackage.path = "/usr/local/nginx/lua /?. Lua;/usr/local/nginx/lua/lib /?. Lua; "package. cpath ="/usr/local/nginx/lua /?. So;/usr/local/nginx/lua/lib /?. So; "-- ban ip time ip_bind_time = 300 -- ip Access Frequency period ip_time_out = 60 -- maximum ip Access Frequency connect_count = 60 -- connect to redislocal redis = require" resty. redis "local cache = redis. new () local OK, err = cache. connect (cache, "10.10.10.8", "6381") cache: set_timeout (60000) -- if the connection fails, jump to the label if not OK then goto labelend -- disable keyis_bind from the ip address, err = cache: get ("bind _".. ngx. var. remote_addr) -- whitelist -- after the verification code is passed, set white_ngx.var.remote_addr to 1 and set the expiration time. The is_white, err = cache: get ("white _".. ngx. var. remote_addr) if tonumber (is_white) = 1 then goto labelend -- query whether the ip address is in the blocked period. if so, go to the Verification code page if tonumber (is_bind) = 1 then -- ngx. say ("block, jump to verification code page") -- base64 encoding local source = ngx. encode_base64 (ngx. var. scheme .. "://".. ngx. var. host .. ngx. var. request_uri) local dest = "http: // 10.10.10.8/authcode.html ".. "? Continue = ".. source -- url_args encoding -- local source = ngx. encode_args ({continue = ngx. var. scheme .. "://".. ngx. var. host .. ngx. var. request_uri}) -- local dest = "http: // 10.10.10.8/authcode.html ".. "? ".. Source ngx. redirect (dest, 302) goto labelend -- ip record time keystart_time, err = cache: get ("time _".. ngx. var. remote_addr) -- ip count keyip_count, err = cache: get ("count _".. ngx. var. remote_addr) -- if the key of the ip record time does not exist or the current time minus the ip record time is greater than the specified time interval, reset time key and count key -- if the current time minus the ip record time is less than the specified time interval, ip count + 1, and ip count is greater than the specified ip Access frequency, set the ip address ban key to 1, and set the ban key expiration time to the ban ip address time if start_time = ngx. null or OS. time ()-tonumber (start_time)> ip_time_out then res, err = cache: set ("time _".. ngx. var. remote_addr, OS. time () res, err = cache: set ("count _".. ngx. var. remote_addr, 1) else ip_count = ip_count + 1 res, err = cache: incr ("count _".. ngx. var. remote_addr) if ip_count> = connect_count then res, err = cache: set ("bind _".. ngx. var. remote_addr, 1) -- submit the following steps to php. If the Verification Code fails, set the bind expiration time. If the verification code passes, set white_ip to 1 and set its expiration time -- res, err = cache: expire ("bind _".. ngx. var. remote_addr, ip_bind_time) endend: label: local OK, err = cache: close ()

 

10.10.10.8/authcode.html is the verification code page, which must be written in other languages!

Note:

1. In the production environment, because the backend php needs to record the remote client ip address, you need to enable the relevant settings on the nginx Proxy:

Proxy_set_header X-Real-IP $ remote_addr;
Proxy_set_header X-Forwarded-For $ proxy_add_x_forwarded_for;

2. In the application production environment, multiple domain names are supported. The formats of ngx.execand ngx.redirectare different, ngx.exe c is internal, and ngx. redirect is external;

3. white_ip is a whitelist. If verification succeeds, the whitelist is added and the expiration time of the whitelist is set. If verification fails, the bind_ip expiration time is set directly. If verification is not performed, it will be blocked all the time;

4. Check that white_ip must be placed on bind_ip, because the subsequent judgment part will be skipped after the whitelist has been added;

5. After you jump to the Verification code page, you need to record the url of the page to be accessed to go to the page to be accessed after the verification is passed;

6. cout_ip and time_ip will be reset if the limit is not reached during the access time.

4. Add the lua script to the corresponding location

   location /test {                access_by_lua_file '/usr/local/nginx1.6/lua/access.lua';                content_by_lua 'ngx.header.content_type = "text/plain"                ngx.say("hello,world")                ';        }

When the number of visits reaches 100 within one minute, the verification code page is displayed.