Below is the access log file
14:43:22
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;. Net CLR 2.0.50727;. Net CLR 1.1.4322)
14:43:27
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;. Net CLR 2.0.50727;. Net CLR 1.1.4322)
14:44:18
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
14:44:26
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; sv1; Maxthon; qqdownload 1.7;. Net CLR 1.1.4322;. Net CLR 2.0.50727;. Net CLR 3.0.04506.648;. Net CLR 3.5.21022)
14:45:17
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Open this file, which is used to record normal visitors or robot spider accessing the search bar, extracted from the spider: Mozilla/5.0 (compatible; googlebot/2.1; + http://www.google.com/bot.html)
Gigabot/3.0 (http://www.gigablast.com/spider.html)
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
Mozilla/5.0 (compatible; yodaobot/1.0; http://www.yodao.com/help/webmaster/spider /;)
Sogou web spider/4.0 (+ http://www.sogou.com/docs/help/webmasters.htm#07)
Msnbot/1.1 (+ http://search.msn.com/msnbot.htm)
Baiduspider + (+ http://www.baidu.com/search/spider.htm)
CodeIt is also very simple:Copy codeThe Code is as follows: string agent = system. Web. httpcontext. Current. Request. servervariables ["http_user_agent"];
If (agent = NULL)
{
Response. Write ("other ");
}
Else
{
Response. Write (agent );
}
You can try this code and find that the output is similar to: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; tencenttraveler 4.0;. Net CLR 2.0.50727;. Net CLR 1.1.4322)
With this, you can write a spider access record. How can you determine whether a normal user or spider is used? You can roughly determine this: by judging the user's access Source Page, such as from the jb51.net/index.htmfile to show.asp? Id = 11 so we know its source page is jb51.net/index.htm, which means he is a normal customer. The source of the spider is not. Record this part and choose from it. It can be recorded in conjunction with the database.