Nginx prohibits specific UA access

First, what is UA?

The user agent is referred to as UA, which is the proxy. Usually we use a browser to access the site, in the site's log, our browser is a kind of UA.

Ii. prohibition of specific UA access

Recently, there is a website (www.C.com) Plagiarism of the company's main station (www.A.com) content, in addition to different domain names, content, pictures, etc. are completely the content of our main station.

In this case, there are two possible ways:

One is that it uses crawlers to crawl the contents of the company's main station (www.A.com) and put it on its own server display;

The other is by the access agent to the company Master Station (Www.A.com), and the domain name (www.C.com) is a misappropriation, fraudulent traffic.

In any case, the continuation of such acts should be prohibited.

By analyzing the logs, there is no information on crawling the company Master (www.A.com) content page.

Test whether it is a proxy, test steps:

1) Access www.C.com and specify parameters

Url:www.c.com/chaojikaifangfu.html/?cccc

2) Check the Nginx log on the Web server of the company master station

A record was found stating that the visit was being proxied to the company's master Web site.

UA (the full name of UA is inside double quotes): Resty/http 0.03 (Lua)

There are two ways to solve this:

1) Disable IP

2) Prohibit UA

From the Nginx log observation, the visitor's proxy IP is often changed, but access to the UA is fixed, so you can prohibit UA.

Prohibit specific UA:

1) Nginx configuration information

Nginx Configuration directory:/usr/local/nginx/conf/

2) in the Nginx configuration directory, create a new configuration file, agent_deny.conf, and add the following:

# Prohibit specific UA access
# ~ "Lua": represents user ua matching "LUA" string, case-sensitive
# when the user UA is matched successfully, a 403 error is returned and no access is given.  if"Lua") {     403;}

3) Add the configuration to the appropriate server or location segment

Include  agent_deny.conf;

Note: You cannot add to an HTTP segment, and the If for HTTP segment will report a syntax error.

4) Make the configuration effective

Nginx-s Reload

5) Check that the functionality of a specific UA is not successful

Using the Curl tool, specify UA to access the company Master Station (www.A.com)

Do not disable specific UA before accessing the result:

# Curl-i-A"resty/http 0.03 (Lua)"Www.A.comHTTP/1.1  $okcontent-type:text/html; charset=utf-8Connection:keep-Alivedate:thu,GenevaNov .  the: -: -GMTContent-length:361168vary:accept-Encodingage:294Pragma:publicServer:nginxAccept-ranges:bytes

After you disable a specific UA, access the results:

" resty/http 0.03 (Lua) " Www.A.comHTTP/1.1403:17  gmtcontent-type:text/htmlcontent162connection:keep-alive

Comparing two times, it is found that Nginx has been able to prohibit specific UA access.

Nginx prohibits specific UA access