Common HTTP Status Codes

First, you need to know what the HTTP status code is: the HTTP status code is a three-digit code used to indicate the HTTP response status of the Web server. When a client sends an HTTP request to the Web server, the server returns a response code. These response codes are mainly divided into five types: 100 status codes, 200 status codes, 300 status codes, 400 status codes, and 500 status codes, the following describes the meanings of common HTTP status codes:
I. 100 status code:
The HTTP status code from 100 to 199 is an information report code. For various reasons, we rarely see the code in most cases. Not detailed.

2: The Request status code is successfully processed 2 ×.
200 the server has successfully processed the request and provided the requested webpage.
204 the server successfully processed the request but did not return any content.

3. Redirect 3 × the number of redirection requests must not exceed 5.
301 the requested webpage has been permanently moved to a new location. Code 301 is used when the URLs changes. Save the new URL in the search engine index.
302 the requested webpage is temporarily moved to a new location. The original URL is saved in the search engine index.
304 if the web page is not updated after the last request, use code 304 to notify the search engine robot, which can save bandwidth and costs.

4. Client errors 4××
It indicates that the request may fail, which hinders server processing.
400 the server does not understand the request syntax.
403 the server rejects the request.
404 The server cannot find the requested webpage. This code is often returned for webpages that do not exist on the server.
410 after the requested resource is permanently deleted, the server returns this response. This code is similar to the 404 (not found) code, but is sometimes used to replace the 404 code when the resource exists before and does not exist. If the resource has been permanently deleted, use 301 to specify a new location for the resource.

V. Server Error 5××
An internal error occurs when the server processes the request. These errors may be server errors rather than request errors.

500 the server encountered an error and could not complete the request.

503 the server is currently unavailable (due to overload or downtime maintenance ). Generally, this is only a temporary status.

 

 

Baidu official explanation: Meaning of common HTTP return Status Codes

When crawling and processing, Baidu crawlers set the corresponding logic according to the HTTP protocol specification, therefore, the webmaster should also try to refer to the definition of the status code in the HTTP protocol for setting.
Baidu Spider's processing logic for common HTTP status codes is as follows:

1. 404 (What should I do on the 404 page ?)
The Status Code 404 indicates "not found". Baidu will think that the webpage has expired, so it will usually be deleted from the search results, and the spider will not capture the URL again in the short term.

2. 503
The status code 503 indicates "Service unavailable". Baidu will think that the webpage is temporarily unavailable. This is usually the case if the website is temporarily closed or the bandwidth is limited. Baidu spider will not directly Delete the URL for returning 503 to the webpage, but will access it again in a short period of time. If the web page is restored, it will be crawled normally. If 503 is returned, it will be accessed several times in the short term. However, if the web page returns 503 for a long time, the URL will still be considered invalid by Baidu and deleted from the search results.

3. 403
The status code 403 indicates "forbidden". Baidu considers that the webpage is currently not accessible. In this case, if it is a newly discovered URL, Baidu spider will not capture it for the moment and will check it again in the short term. If it is a URL already included by Baidu, it will not be deleted directly at present, in the short term, access will be resumed. If the webpage is allowed to be accessed, the webpage will be crawled normally. If the webpage is still not allowed to be accessed, it will be accessed several times in a short period of time. However, if the web page returns 403 for a long time, Baidu will also consider the link as invalid and delete it from the search results.

4. 301 (301 redirection)
The status code 301 indicates "moved permanently". Baidu will assume that the webpage is currently redirected to the new URL. When you encounter site migration, domain name replacement, and site revision, we recommend that you use a 301 status code to minimize the traffic loss caused by the revision. Although Baidu spider has a long response cycle for 301 redirection, we recommend that you do so.

Our Suggestions
1. if the site is temporarily closed, do not return 404 immediately when the webpage cannot be opened. We recommend that you use 503. 503 you can tell Baidu spider that the page is temporarily inaccessible. Please try again later.

2. If Baidu spider is under too much grabbing pressure on your site, try not to use 404. We recommend that you return 503. In this way, Baidu spider will try again later to capture this link. if the site is idle at that time, it will be captured successfully.

3. Some websites want Baidu to include only part of the content, such as the content after review and new user pages accumulated over a period of time. In this case, it is recommended that 403 be returned for the new content temporarily. After review or processing, the status code of the normal status is returned.

4. Use the 301 status code when migrating the site or changing the domain name.