Web Crawler Exception Handling

Http://blog.csdn.net/wxg694175346/article/details/8923725

The response of each HTTP Response object on the server contains a number "status code ".
 

Sometimes the status code indicates that the server cannot complete the request. The default processor will process some of these responses for you.

For example, if response is a "redirection", the client needs to obtain the document from another address, and urllib2 will process it for you.

Otherwise, urlopen generates an httperror.

Typical errors include "404" (page not found), "403" (request prohibited), and "401" (with verification request ).

The HTTP status code indicates the status of the response returned by the HTTP protocol.

For example, a client sends a request to the server. If the requested resource is successfully obtained, the returned status code is 200, indicating that the response is successful.

If the requested resource does not exist, Error 404 is usually returned.

HTTP status codes are generally divided into 5 types, with 1 ~ It starts with five digits and consists of three integers:

Bytes ------------------------------------------------------------------------------------------------

200: Successful request processing method: Obtain the response content for processing

201: the request is complete. The result is that a new resource is created. The URI of the newly created resource can be processed in the response object.

202: the request is accepted, but the processing has not been completed. Processing Method: Blocking wait

204: the server has implemented the request, but no new message is returned. If the customer is a user agent, you do not need to update the document view for this. Processing Method: discard

300: the status code is not used by HTTP/1.0 applications.ProgramIt is used directly. It is only the default explanation for 3xx type responses. Multiple available requested resources exist. Processing Method: if it can be processed in the program, it will be further processed. If it cannot be processed in the program, it will be discarded.
301: all requested resources will be allocated with a permanent URL. In this way, you can use this URL to access this resource in the future. Processing Method: redirect to the allocated URL.
302: the requested resource is temporarily saved in a different URL. Processing Method: redirect to a temporary URL.

304 request resource not Updated Handling Method: discard

400 Illegal Request Handling Method: discard

401 unauthorized handling method: discard

403 Forbidden processing method: discard

404 no handling method found: discard

5xx responseCodeThe status code starting with "5" indicates that the server finds an error and cannot continue to execute the request processing method: discard

 

Bytes ------------------------------------------------------------------------------------------------

After an httperror instance is generated, an integer 'code' attribute is generated, indicating the error code sent by the server.

Error Codes error code
Because the default processor processes redirection (a number other than 300) and the number in the range of-299 indicates success, you can only see the error number.
Basehttpserver. basehttprequesthandler. response is a useful answer number dictionary that displays all the response numbers used by the HTTP protocol.

When an error code is generated, the server returns an HTTP Error code and an error page.

You can use the httperror instance as the response object response returned by the page.

This indicates that, like the error attribute, it also contains the read, geturl, and info methods.