Python Requests Library: HTTP for humans

Source: Internet
Author: User
Tags response code git clone

The module used to process HTTP in the Python standard library is URLLIB2, but the API is fragmented, and requests is a simpler and more user-friendly third-party library.

Download with Pip:

PIP Install requests

or git:

git clone git://github.com/kennethreitz/requests.git
Send request:

Get method

>>> Import requests>>> r = Requests.get (' https://api.github.com/events ')

Post method:

>>> r = requests.post ("Http://httpbin.org/post")

You can also use other methods:

>>> r = requests.put ("Http://httpbin.org/put") >>> r = Requests.delete ("Http://httpbin.org/delete") >>> r = Requests.head ("Http://httpbin.org/get") >>> r = requests.options ("Http://httpbin.org/get")

You can also put the request method in the parameter:

>>> Import requests>>> req = requests.request (' GET ', ' http://httpbin.org/get ')
To pass parameters or upload files:

1. If you want to pass parameters in a URL, use the params parameter, which can be a dictionary or a string:

>>> payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}>>> r = Requests.get ("Http://httpbin.org/get", params=payload) >>> r.urlu ' http://httpbin.org/get?key2=value2&key1=value1 '

2. If you want to pass the parameter in the request body, use the data parameter, which can be a dictionary, a string, or a class file object.

When using a dictionary, form-encoded data is sent:

>>> payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}>>> r = requests.post ("Http://httpbin.org/post", Data=payload) >>> Print (r.text) {  ...  " Form ": {    " Key2 ":" value2 ",    " Key1 ":" Value1 "  },  ...}

Data is sent directly when using a string:

>>> import json>>> url = ' https://api.github.com/some/endpoint ' >>> payload = {' Some ': ' Data ' }>>> r = requests.post (URL, data=json.dumps (payload))

3. If you want to upload a file, you can send the multipart-encoded data using the files parameter, which is a dictionary of {' name ': file-like-objects} format (or {' name ':(' filename ', fileobj )}) :

>>> url = ' Http://httpbin.org/post ' >>> files = {' file ': Open (' Report.xls ', ' RB ')}>>> r = Reque Sts.post (URL, files=files) >>> r.text{...  " Files ": {    " file ":" <censored...binary...data> "  },  ...}

You can also explicitly set the filename, Content_Type and headers:

>>> url = ' Http://httpbin.org/post ' >>> files = {' file ': (' Report.xls ', open (' Report.xls ', ' RB '), ' Application/vnd.ms-excel ', {' Expires ': ' 0 '})}>>> r = requests.post (URL, files=files) >>> print R.text {  "args": {},   "Data": "",   "files": {    "file": "1\t2\r\n"  },   "form": {},   "headers": {    "Content-type": "Multipart/form-data; boundary=e0f9ff1303b841498ae53a903f27e565 ",     " Host ":" httpbin.org ",     " user-agent ":" python-requests/2.2.1 cpython/2.7.3 WINDOWS/7 ",   },   " url ":" Http://httpbin.org/post "}
Set headers
>>> import json>>> url = ' https://api.github.com/some/endpoint ' >>> payload = {' Some ': ' Data ' }>>> headers = {' Content-type ': ' Application/json '}>>> r = requests.post (URL, data=json.dumps ( Payload), Headers=headers)
Response object:

Gets the Unicode string, which is automatically decoded according to the character encoding (r.encoding) of the response header, and can, of course, set r.encoding itself:

>>> r = requests.get (' Https://github.com/timeline.json ') >>> r.textu ' {"message": "Hello there, Wayfaring Stranger ...

Gets the bytes string that automatically decodes the gzip and deflate data:

>>> r.content ' {"message": "Hello there, wayfaring stranger ...

To store a Web picture, you can:

>>> from PIL import image>>> from Stringio import stringio>>> i = Image.open (Stringio (R.conten T))

You can decode a JSON object:

>>> R.json () {u ' documentation_url ': U ' https://developer ...

To return the raw response, you need to set the stream to true in the requests request:

>>> r = requests.get (' Https://github.com/timeline.json ', stream=true) >>> r.raw< Requests.packages.urllib3.response.HTTPResponse object at 0x101194810>>>> R.raw.read (Ten) ' \x1f\x8b\x08\ X00\x00\x00\x00\x00\x00\x03 '

Of course, the better approach is to iterate over the data:

with open (filename, ' WB ') as FD: For    Chunk in R.iter_content (chunk_size):        fd.write (Chunk)

Get the response code:

>>> r = requests.get (' http://httpbin.org/get ') >>> r.status_code200

Get Response headers:

>>> r.headers{'    content-encoding ': ' gzip ',    ' transfer-encoding ': ' chunked ',    ' connection ': ' Close ',    ' server ': ' nginx/1.0.4 ',    ' x-runtime ': ' 148ms ',    ' etag ': ' "e1ca502697e5c9317743dc078f67693f" ', '    content-type ': ' Application/json '}
Cookies

Gets the cookie that returns the Cookiejar object:

>>> url = ' http://www.baidu.com ' >>> r = requests.get (URL) >>> r.cookies

Convert Cookiejar to Dictionary:

>>> Requests.utils.dict_from_cookiejar (r.cookies) {' Baiduid ': ' 84722199df8edc372d549ec56ca1a0e2:fg=1 ', ' Bd_home ': ' 0 ', ' bdsvrtm ': ' 0 '}

To convert a dictionary to Cookiejar:

Requests.utils.cookiejar_from_dict (Cookie_dict, Cookiejar=none, Overwrite=true)

Uploading your own set of cookies, using cookies, can be a dictionary or Cookiejar object:

>>> url = ' http://httpbin.org/cookies ' >>> cookies = dict (cookies_are= ' working ') >>> r = Requests.get (URL, cookies=cookies) >>> r.text ' {"cookies": {"Cookies_are": "Working"}} '

If you need to keep cookies in your session, you need to use the following session.

Redirection and history

You can use the history property to track redirection

>>> r = requests.get (' http://github.com ') >>> r.url ' https://github.com/' >>> r.status_ Code200>>> r.history[<response [301]>]
Session

To preserve state in a session, you can use request. Session ().

Session can use Get,post, etc., the returned cookie will be automatically retained on the next visit:

>>> Import requests>>> s = requests. Session () >>> s.get (' Http://httpbin.org/get ') 200

You can also set headers,cookies yourself:

>>> Import requests>>> s = requests. Session () >>> s.headers={...}
Verify

Basic Authentication

>>> from Requests.auth import httpbasicauth>>> requests.get (' Https://api.github.com/user ', auth= Httpbasicauth (' user ', ' pass ') <response [200]>

Because HTTP Basic auth is common, you can also verify it directly:

>>> requests.get (' Https://api.github.com/user ', auth= (' user ', ' Pass ')) <response [200]>

Digest Authentication

>>> from requests.auth import httpdigestauth>>> url = ' http://httpbin.org/digest-auth/auth/user/ Pass ' >>> requests.get (URL, auth=httpdigestauth (' user ', ' Pass ')) <response [200]>

OAuth 1 Authentication

>>> Import requests>>> from requests_oauthlib import oauth1>>> url = ' https:// Api.twitter.com/1.1/account/verify_credentials.json ' >>> auth = OAuth1 (' Your_app_key ', ' Your_app_secret ',                  ' User_oauth_token ', ' User_oauth_token_secret ') >>> requests.get (URL, auth=auth) <response [200]>

You can also use validation classes that you write yourself. For example, a Web service accepts authentication that sets the X-pizza header to a password, which can be written to verify the class:

From Requests.auth import Authbaseclass Pizzaauth (authbase): ""    "attaches HTTP Pizza authentication to the given Reque St object.    "" " def __init__ (self, username):        # Setup No auth-related data here        Self.username = Username    def __call__ (self, R):        # Modify and return the request        r.headers[' X-pizza '] = Self.username        return r

Use:

>>> requests.get (' Http://pizzabin.org/admin ', Auth=pizzaauth (' Kenneth ')) <response [200]>
Agent

Using proxies:

Import requestsproxies = {  "http": "http://10.10.1.10:3128",  "https": "http://10.10.1.10:1080",} Requests.get ("http://example.org", proxies=proxies)

You can set environment variables:

$ export http_proxy= "http://10.10.1.10:3128" $ export https_proxy= "http://10.10.1.10:1080" $ python>>> Import Requests>>> requests.get ("http://example.org")

If the agent needs validation:

Proxies = {    "http": "Http://user:[email protected]:3128/",}

  

Python Requests Library: HTTP for humans

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.