About Web site IIS Log analysis search engine crawler description

Intermediary transaction http://www.aliyun.com/zixun/aggregation/6858.html ">seo diagnose Taobao guest cloud host technology Hall

Title: About Web site IIS Log analysis search engine crawler description

Text: IIS default log file in the C:\WINDOWS\system32\LogFiles, the following is Seoer edge of the server log, through the view, you can understand the search engine spider crawling through, such as:

2008-08-19 00:09:12 w3svc962713505 203.171.226.111 get/index.html-80-61.135.168.39 baiduspider+ (+http:// www.baidu.com/search/spider.htm) 200 0 64

1, 203.171.226.111 is the search engine spider to ask the website IP,

2, 61.135.168.39 Baiduspider representative, Baidu search engine spider IP is 61.135.168.39

3, the code in the/index.html on behalf of the search engine Spider-proofing page

4, 2008-08-19 00:09:12 on behalf of the search engine spider crawling Date and time

5, w3svc962713505 represents the Web site log folder

6. http://www.baidu.com/search/spider.htm Baiduspider FAQ Page

7, the code of 200 on behalf of the search engine spider crawling back to the HTTP status code, the code can understand the spider crawling after the reflection, code as follows:

2XX success

200 normal;

201 normal;

202 normal; Accepted for processing, but processing has not yet completed.

203 normal; Partial information-the information returned is only part of the message.

204 normal; No response-received request, but no information to echo.

3xx redirect

301 Moved-The requested data has a new location and the change is permanent.

302 found-The requested data has a different URI temporarily.

303 See other-the response to the request can be found under another URI, and the response should be retrieved using the Get method.

304 unmodified-The document is not modified as expected.

305 Use proxy-the requested resource must be accessed through the agent provided in the Location field.

306 unused-is no longer in use;

Error occurred in 4xx client

400 Error request-There is a syntax problem in the request or cannot satisfy the request.

401 Unauthorized-The client is not authorized to access the data.

402 Payment required-Indicates that the billing system is valid.

403 prohibited-access is not required even if authorized.

404 Not Found-the server could not find the given resource;

407 Proxy authentication Request-The client must first use the proxy authentication itself.

410 The requested Web page does not exist (permanent);

415 Media type not supported-server denial of Service request because the format of the request entity is not supported.

Error occurred in 5xx server

500 INTERNAL Error-The server cannot complete the request because of an unexpected condition.

501 not executed-the server does not support the requested tool.

502 Error Gateway-server received an invalid response from the upstream server.

503 cannot get service-the server cannot process the request due to temporary overload or maintenance.