Baidu Lee gives the meaning and suggestion of the HTTP return code

Baidu Crawler in the crawl and processing, is based on the HTTP protocol specification to set the corresponding logic, so please webmaster also try to reference the HTTP protocol on the meaning of the return code to set the definition.

　Baidu Spider to the commonly used HTTP return code processing logic is this:

1, 404

404 Return Code meaning is "not FOUND", Baidu will think that the Web page has been invalidated, then usually removed from the search results, and in the short term spider again found that the URL will not crawl.

2, 503

503 Return code meaning is "Service unavailable", Baidu will think that the Web page temporarily inaccessible, usually temporary site shutdown, limited bandwidth, etc. will produce this situation. For Web page return 503, Baidu Spider will not delete this URL directly, will visit again shortly. If the page has been restored, then the normal crawl, if you continue to return 503, in the short term will be repeatedly visited several times. However, if the Web page returns for a long time 503, then this URL will still be considered as a failure link Baidu, removed from the search results.

3, 403

403 Return code meaning is "forbidden", Baidu will think that the Web page is currently prohibited access. For this situation, if it is a new found URL, Baidu Spider will not crawl, in the near future to check again, if Baidu has been included in the URL, the current will not be directly deleted, the same will be again in the near future visit. If the page allows access, then the normal crawl, if still not allow access, in the short term will be repeatedly visited several times. However, if the Web page returns for a long time 403, Baidu will also be considered a failure link, from the search results deleted.

4, 301

301 Return code meaning is "moved permanently", Baidu will think that the page is currently jump to the new URL. When encountered site migration, domain name replacement, site revision of the situation, the recommended use of 301 return code to minimize the loss of the flow of the revision. Although Baidu Spider now to 301 jump response cycle is longer, but we still recommend everyone to do so.

　Our suggestions

1, if the site is temporarily closed, when the Web page can not be opened, do not immediately return 404, recommend the use of 503 states. 503 can tell Baidu spider this page temporarily inaccessible, please try again after a period of time.

2, if Baidu Spider to your site crawl pressure is too large, please try not to use 404, the same proposal to return 503. So Baidu Spider will try to crawl this link over a period of time, if that time site is idle, it will be successfully crawled.

3, some sites want Baidu to include only part of the content, such as the content after the audit, accumulated a period of time the new user page and so on. In this case, it is recommended that the new content temporarily return 403, such as audit or processing, and then return to the normal state return code.

4, site migration, or domain name replacement, please use 301 return code.

Article Finishing: Beijing SEO, article address: http://www.pangziseo.com/seo_resources/117.html reprint please keep.