The requests of Python

Get started quickly

Can't wait for it? This page provides a good guide on how to get started with requests. It is assumed that you have installed the requests. If not, go to install a section to see it.

First, make sure that:

• Requests already installed
  

• Requests is the newest
  

Let's start with some simple examples.

Send Request

Sending network requests using requests is straightforward.

To import the requests module at the outset:

>>> Import Requests

Then, try to get a webpage. In this example, we're going to get GitHub's public timeline

>>> r = requests.get (' Https://github.com/timeline.json ')

Now, we have a Response object named R. You can get all the information we want from this object.

Requests's simple API means that all HTTP request types are obvious. For example, you can send an HTTP POST request like this:

>>> r = requests.post ("Http://httpbin.org/post")

Pretty, isn't it? So what about the other HTTP request types: PUT, DELETE, head, and options? Are the same simple:

>>> r = Requests.put (">>> r = Requests.delete (" >>> r = Requests.head (">>> r = Reque Sts.options ("Http://httpbin.org/get")

All very well, but this is only the tip of the iceberg of requests.

Passing parameters for URLs

You might often want to pass some kind of data to the query string of a URL. If you build the URL manually, the data is placed in the URL in the form of a key/value pair followed by a question mark. For example, Httpbin.org/get?key=val. Requests allows you to use the params keyword parameter to provide these parameters in a dictionary. For example, if you want to pass key1=value1 and key2=value2 to Httpbin.org/get, then you can use the following code:

>>> payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}>>> r = Requests.get ("Http://httpbin.org/get", Params=payload)

By printing out the URL, you can see that the URL has been correctly encoded:

>>> print R.urlu ' http://httpbin.org/get?key2=value2&key1=value1 '

Response Content

We can read the contents of the server response. Again, take the GitHub timeline as an example:

>>> Import requests>>> r = Requests.get (' >>> r.text ' [{"repository": {"open_issues": 0, "url" : "https://github.com/...

Requests will automatically decode the content from the server. Most Unicode character sets can be decoded seamlessly.

After the request is issued, requests will make an educated guess based on the HTTP header's encoding of the response. When you visit R.text, requests uses its inferred text encoding. You can find out what encoding requests is using and can use the R.encoding property to change it:

>>> r.encoding ' utf-8 ' >>> r.encoding = ' iso-8859-1 '

If you change the code, the request will use the new value of r.encoding whenever you visit r.text.

Requests can also use custom encodings where you need them. If you create your own code and register with the codecs module, you can easily use the decoder name as the r.encoding value, and then the requests will handle the encoding for you.

Binary response Content

You can also access the request response body in bytes, for non-textual requests:

>>> R.contentb ' [{"repository": {"open_issues": 0, "url": "https://github.com/...

Requests will automatically decode your gzip and deflate transmit encoded response data for you.

For example, to create a picture of the binary data returned by the request, you can use the following code:

>>> from PIL import image>>> from Stringio import stringio>>> i = Image.open (Stringio (R.conten T))

JSON response content

The requests also has a built-in JSON decoder to help you work with JSON data:

>>> Import requests>>> r = Requests.get (' >>> r.json () [{u ' repository ': {u ' open_issues ': 0, U ' URL ': ' https://github.com/...

If the JSON decoding fails, R.json throws an exception.

Original response Content

In rare cases you may want to get the original socket response from the server, then you can access the R.raw. If you really want to do this, make sure that you set the stream=true in the initial request. Specifically, you can do this:

>>> r = requests.get (' Https://github.com/timeline.json ', stream=true) >>> r.raw< Requests.packages.urllib3.response.HTTPResponse object at 0x101194810>>>> R.raw.read (Ten) ' \x1f\x8b\x08\ X00\x00\x00\x00\x00\x00\x03 '

Custom Request Headers

If you want to add an HTTP header for the request, simply pass a dict to the headers parameter.

For example, in the previous example we did not specify Content-type:

>>> import json>>> url = ' >>> payload = {' Some ': ' data '}>>> headers = {' Content-type ' : ' Application/json '}>>> r = requests.post (URL, data=json.dumps (payload), headers=headers)

More complex POST requests

Typically, you want to send some data encoded as forms-much like an HTML form. To implement this, simply pass a dictionary to the data parameter. Your data dictionary is automatically encoded as a form when the request is made:

>>> payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}>>> r = requests.post ("Http://httpbin.org/post",  Data=payload) >>> Print r.text{... "Form": {"Key2": "value2", "Key1": "Value1"}, ...}

Many times the data you want to send is not encoded as a form. If you pass a string instead of a dict, the data will be published directly.

For example, the Github API V3 accepts POST/PATCH data encoded as JSON:

>>> import json>>> url = ' >>> payload = {' Some ': ' data '}>>> r = requests.post (URL, da Ta=json.dumps (payload))

Post a multi-part encoding (multipart-encoded) file

Requests makes it easy to upload multi-part encoded files:

>>> url = ' >>> files = {' file ': Open (' Report.xls ', ' RB ')}>>> r = requests.post (URL, files=file  s) >>> r.text{... "Files": {"file": "<censored...binary...data>"}, ...}

You can explicitly set the file name:

>>> url = ' >>> files = {' file ': (' Report.xls ', open (' Report.xls ', ' RB '))}>>> r = Requests.post  (URL, files=files) >>> r.text{... "Files": {"file": "<censored...binary...data>"}, ...}

If you want, you can also send a string as a file to receive:

>>> url = ' >>> files = {' file ': (' report.csv ', ' some,data,to,send\nanother,row,to,send\n ')}>>& Gt  r = Requests.post (URL, files=files) >>> r.text{... "Files": {"file": "some,data,to,send\\nanother,row,to,send\\n"}, ...}

Response Status Code

We can detect the response status code:

>>> r = Requests.get (' >>> r.status_code200

For easy reference, requests also comes with a built-in status code query object:

>>> R.status_code = = Requests.codes.okTrue

If a failed request (not a 200 response) is sent, we can throw an exception by Response.raise_for_status ():

>>> Bad_r = Requests.get (' >>> bad_r.status_code404>>> bad_r.raise_for_status () Traceback ( Most recent call last): File "requests/models.py", line 832, in Raise_for_status raise Http_errorrequests.exceptions.h ttperror:404 Client Error

However, since the Status_code of R in our example is 200, when we call Raise_for_status (), we get:

>>> r.raise_for_status () None

Everything is very harmonious, huh.

Response header

We can view the server response headers shown in a Python dictionary:

>>> r.headers{' status ': ' $ OK ', ' content-encoding ': ' gzip ', ' transfer-encoding ': ' chunked ', ' Conne    Ction ': ' Close ', ' Server ': ' nginx/1.0.4 ', ' x-runtime ': ' 148ms ', ' etag ': ' e1ca502697e5c9317743dc078f67693f ', ' Content-type ': ' Application/json; Charset=utf-8 '}

But this dictionary is very special: it is only for HTTP headers. According to RFC 2616, the HTTP header is case insensitive.

Therefore, we can use any uppercase form to access these response header fields:

>>> r.headers[' content-type '] ' Application/json; Charset=utf-8 ' >>> r.headers.get (' content-type ') ' Application/json; Charset=utf-8 '

If a response header field does not exist, its default value is None

>>> r.headers[' X-random ']none

Cookies

If a response contains some cookies, you can quickly access them:

>>> url = ' >>> r = requests.get (URL) >>> r.cookies[' example_cookie_name '] ' example_cookie_ Value

To send your cookies to the server, you can use the Cookies parameter:

>>> url = ' >>> cookies = dict (cookies_are= ' working ') >>> r = Requests.get (URL, cookies=cookies ) >>> R.text ' {"cookies": {"Cookies_are": "Working"}} '

Redirect and request history

When you use GET or options, requests automatically handles location redirection.

GitHub redirects all HTTP requests to HTTPS. You can use the history method of the response object to track the redirection. Let's take a look at what GitHub does:

>>> r = Requests.get (' >>> r.url ' >>> r.status_code200>>> r.history[<response [ 301]>]

Response.history is a: class: A list of the request objects that were created to complete the request. This list of objects is sorted by the most recent request.

If you are using GET or options, you can disable redirection with the Allow_redirects parameter:

>>> r = requests.get (' http://github.com ', allow_redirects=false) >>> r.status_code301>>> R.history[]

If you are using Post,put,patch,delete or head, you can also enable redirection:

>>> r = requests.post (' http://github.com ', allow_redirects=true) >>> r.url ' https://github.com/' >>> r.history[<response [301]>]

Timeout

You can tell requests to stop waiting for a response after the number of seconds set by the timeout parameter:

>>> requests.get (' http://github.com ', timeout=0.001) Traceback (most recent call last): File "<stdin>", Line 1, in <module>requests.exceptions.timeout:httpconnectionpool (host= ' github.com ', port=80): Request timed Out. (timeout=0.001)

Note:

Timeout is only valid for the connection process and is not related to the download of the response body.

Errors and exceptions

Requests throws a Connectionerror exception when encountering network problems such as DNS query failure, connection rejection, and so on.

When a rare invalid HTTP response is encountered, requests throws a Httperror exception.

If the request times out, a timeout exception is thrown.

If the request exceeds the maximum number of redirects set, a Toomanyredirects exception is thrown.

All exceptions that are explicitly thrown by requests inherit from Requests.exceptions.RequestException.

The requests of Python