Python Requests Library: HTTP for humans

The module used to process HTTP in the Python standard library is URLLIB2, but the API is fragmented, and requests is a simpler and more user-friendly third-party library.

Download with Pip:

PIP Install requests

or git:

git clone git://github.com/kennethreitz/requests.git

Send request:

Get method

>>> Import requests>>> r = Requests.get (' https://api.github.com/events ')

Post method:

>>> r = requests.post ("Http://httpbin.org/post")

You can also use other methods:

>>> r = requests.put ("Http://httpbin.org/put") >>> r = Requests.delete ("Http://httpbin.org/delete") >>> r = Requests.head ("Http://httpbin.org/get") >>> r = requests.options ("Http://httpbin.org/get")

You can also put the request method in the parameter:

>>> Import requests>>> req = requests.request (' GET ', ' http://httpbin.org/get ')

To pass parameters or upload files:

1. If you want to pass parameters in a URL, use the params parameter, which can be a dictionary or a string:

>>> payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}>>> r = Requests.get ("Http://httpbin.org/get", params=payload) >>> r.urlu ' http://httpbin.org/get?key2=value2&key1=value1 '

2. If you want to pass the parameter in the request body, use the data parameter, which can be a dictionary, a string, or a class file object.

When using a dictionary, form-encoded data is sent:

>>> payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}>>> r = requests.post ("Http://httpbin.org/post", Data=payload) >>> Print (r.text) {  ...  " Form ": {    " Key2 ":" value2 ",    " Key1 ":" Value1 "  },  ...}

Data is sent directly when using a string:

>>> import json>>> url = ' https://api.github.com/some/endpoint ' >>> payload = {' Some ': ' Data ' }>>> r = requests.post (URL, data=json.dumps (payload))

3. If you want to upload a file, you can send the multipart-encoded data using the files parameter, which is a dictionary of {' name ': file-like-objects} format (or {' name ':(' filename ', fileobj )}) ：

>>> url = ' Http://httpbin.org/post ' >>> files = {' file ': Open (' Report.xls ', ' RB ')}>>> r = Reque Sts.post (URL, files=files) >>> r.text{...  " Files ": {    " file ":" <censored...binary...data> "  },  ...}

You can also explicitly set the filename, Content_Type and headers:

>>> url = ' Http://httpbin.org/post ' >>> files = {' file ': (' Report.xls ', open (' Report.xls ', ' RB '), ' Application/vnd.ms-excel ', {' Expires ': ' 0 '})}>>> r = requests.post (URL, files=files) >>> print R.text {  "args": {},   "Data": "",   "files": {    "file": "1\t2\r\n"  },   "form": {},   "headers": {    "Content-type": "Multipart/form-data; boundary=e0f9ff1303b841498ae53a903f27e565 ",     " Host ":" httpbin.org ",     " user-agent ":" python-requests/2.2.1 cpython/2.7.3 WINDOWS/7 ",   },   " url ":" Http://httpbin.org/post "}

Set headers

>>> import json>>> url = ' https://api.github.com/some/endpoint ' >>> payload = {' Some ': ' Data ' }>>> headers = {' Content-type ': ' Application/json '}>>> r = requests.post (URL, data=json.dumps ( Payload), Headers=headers)

Response object:

Gets the Unicode string, which is automatically decoded according to the character encoding (r.encoding) of the response header, and can, of course, set r.encoding itself:

>>> r = requests.get (' Https://github.com/timeline.json ') >>> r.textu ' {"message": "Hello there, Wayfaring Stranger ...

Gets the bytes string that automatically decodes the gzip and deflate data:

>>> r.content ' {"message": "Hello there, wayfaring stranger ...

To store a Web picture, you can:

>>> from PIL import image>>> from Stringio import stringio>>> i = Image.open (Stringio (R.conten T))

You can decode a JSON object:

>>> R.json () {u ' documentation_url ': U ' https://developer ...

To return the raw response, you need to set the stream to true in the requests request:

>>> r = requests.get (' Https://github.com/timeline.json ', stream=true) >>> r.raw< Requests.packages.urllib3.response.HTTPResponse object at 0x101194810>>>> R.raw.read (Ten) ' \x1f\x8b\x08\ X00\x00\x00\x00\x00\x00\x03 '

Of course, the better approach is to iterate over the data:

with open (filename, ' WB ') as FD: For    Chunk in R.iter_content (chunk_size):        fd.write (Chunk)

Get the response code:

>>> r = requests.get (' http://httpbin.org/get ') >>> r.status_code200

Get Response headers:

>>> r.headers{'    content-encoding ': ' gzip ',    ' transfer-encoding ': ' chunked ',    ' connection ': ' Close ',    ' server ': ' nginx/1.0.4 ',    ' x-runtime ': ' 148ms ',    ' etag ': ' "e1ca502697e5c9317743dc078f67693f" ', '    content-type ': ' Application/json '}

Cookies

Gets the cookie that returns the Cookiejar object:

>>> url = ' http://www.baidu.com ' >>> r = requests.get (URL) >>> r.cookies

Convert Cookiejar to Dictionary:

>>> Requests.utils.dict_from_cookiejar (r.cookies) {' Baiduid ': ' 84722199df8edc372d549ec56ca1a0e2:fg=1 ', ' Bd_home ': ' 0 ', ' bdsvrtm ': ' 0 '}

To convert a dictionary to Cookiejar:

Requests.utils.cookiejar_from_dict (Cookie_dict, Cookiejar=none, Overwrite=true)

Uploading your own set of cookies, using cookies, can be a dictionary or Cookiejar object:

>>> url = ' http://httpbin.org/cookies ' >>> cookies = dict (cookies_are= ' working ') >>> r = Requests.get (URL, cookies=cookies) >>> r.text ' {"cookies": {"Cookies_are": "Working"}} '

If you need to keep cookies in your session, you need to use the following session.

Redirection and history

You can use the history property to track redirection

>>> r = requests.get (' http://github.com ') >>> r.url ' https://github.com/' >>> r.status_ Code200>>> r.history[<response [301]>]

Session

To preserve state in a session, you can use request. Session ().

Session can use Get,post, etc., the returned cookie will be automatically retained on the next visit:

>>> Import requests>>> s = requests. Session () >>> s.get (' Http://httpbin.org/get ') 200

You can also set headers,cookies yourself:

>>> Import requests>>> s = requests. Session () >>> s.headers={...}

Verify

Basic Authentication

>>> from Requests.auth import httpbasicauth>>> requests.get (' Https://api.github.com/user ', auth= Httpbasicauth (' user ', ' pass ') <response [200]>

Because HTTP Basic auth is common, you can also verify it directly:

>>> requests.get (' Https://api.github.com/user ', auth= (' user ', ' Pass ')) <response [200]>

Digest Authentication

>>> from requests.auth import httpdigestauth>>> url = ' http://httpbin.org/digest-auth/auth/user/ Pass ' >>> requests.get (URL, auth=httpdigestauth (' user ', ' Pass ')) <response [200]>

OAuth 1 Authentication

>>> Import requests>>> from requests_oauthlib import oauth1>>> url = ' https:// Api.twitter.com/1.1/account/verify_credentials.json ' >>> auth = OAuth1 (' Your_app_key ', ' Your_app_secret ',                  ' User_oauth_token ', ' User_oauth_token_secret ') >>> requests.get (URL, auth=auth) <response [200]>

You can also use validation classes that you write yourself. For example, a Web service accepts authentication that sets the X-pizza header to a password, which can be written to verify the class:

From Requests.auth import Authbaseclass Pizzaauth (authbase): ""    "attaches HTTP Pizza authentication to the given Reque St object.    "" " def __init__ (self, username):        # Setup No auth-related data here        Self.username = Username    def __call__ (self, R):        # Modify and return the request        r.headers[' X-pizza '] = Self.username        return r

Use:

>>> requests.get (' Http://pizzabin.org/admin ', Auth=pizzaauth (' Kenneth ')) <response [200]>

Agent

Using proxies:

Import requestsproxies = {  "http": "http://10.10.1.10:3128",  "https": "http://10.10.1.10:1080",} Requests.get ("http://example.org", proxies=proxies)

You can set environment variables:

$ export http_proxy= "http://10.10.1.10:3128" $ export https_proxy= "http://10.10.1.10:1080" $ python>>> Import Requests>>> requests.get ("http://example.org")

If the agent needs validation:

Proxies = {    "http": "Http://user:[email protected]:3128/",}

　　

Python Requests Library: HTTP for humans