The following is an access log file
2008-8-13 14:43:22
mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1. NET CLR 2.0.50727;. NET CLR 1.1.4322)
2008-8-13 14:43:27
mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1. NET CLR 2.0.50727;. NET CLR 1.1.4322)
2008-8-13 14:44:18
mozilla/5.0 (compatible; Yahoo! Slurp; HTTP://HELP.YAHOO.COM/HELP/US/YSEARCH/SLURP)
2008-8-13 14:44:26
mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; Maxthon; Qqdownload 1.7;. NET CLR 1.1.4322;. NET CLR 2.0.50727;. NET CLR 3.0.04506.648;. NET CLR 3.5.21022)
2008-8-13 14:45:17
mozilla/5.0 (compatible; Yahoo! Slurp; HTTP://HELP.YAHOO.COM/HELP/US/YSEARCH/SLURP)
Open this file, this file is used to record access to the search bar of the normal visitors or robot spiders, extracted from the Spider: mozilla/5.0 (compatible; googlebot/2.1; +http://www.google.com/bot.html)
gigabot/3.0 (http://www.gigablast.com/spider.html)
mozilla/5.0 (compatible; Yahoo! Slurp; HTTP://HELP.YAHOO.COM/HELP/US/YSEARCH/SLURP)
mozilla/5.0 (compatible; yodaobot/1.0; http://www.yodao.com/help/webmaster/spider/; )
Sogou Web spider/4.0 (+http://www.sogou.com/docs/help/webmasters.htm#07)
msnbot/1.1 (+http://search.msn.com/msnbot.htm)
baiduspider+ (+http://www.baidu.com/search/spider.htm)
The code is also very simple:
Copy Code code as follows:
String agent = system.web.httpcontext.current.request.servervariables["Http_user_agent"];
if (Agent = = null)
{
Response.Write ("other");
}
Else
{
Response.Write (agent);
}
You can try this code yourself and find out that it will output something like this: mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Tencenttraveler 4.0;. NET CLR 2.0.50727;. NET CLR 1.1.4322)
Combining this you can write a spider access record, how to judge a normal user or spider? This can be roughly judged: by judging the user's Access source page, such as from the jb51.net/index.htm file point into show.asp?id=11 then his source page we know is jb51.net/index.htm, then rough judgment he is a normal customer. And the source of spiders is not, this part is not recorded down, and then choose from. Can be recorded in conjunction with the database.
