Python network programming package use tutorial, pythonrequests

Source: Internet
Author: User

Python network programming package use tutorial, pythonrequests

I have long heard that the requests library is powerful, but I haven't touched it yet. Today I have touched on it and found that urllib, urllib2, and other methods used in the past are too tricky ......
Here, we will write some simple operations and use them as a preliminary record.


1. Download

Official project page: https://pypi.python.org/pypi/requests/#downloads
You can download it directly from above.

2. Send a get request without Parameters

>>> r = requests.get('http://httpbin.org/get')>>> print r.text{ "args": {},  "headers": {  "Accept": "*/*",   "Accept-Encoding": "gzip, deflate",   "Connection": "close",   "Host": "httpbin.org",   "User-Agent": "python-requests/2.3.0 CPython/2.6.6 Windows/7",   "X-Request-Id": "8a28bbea-55cd-460b-bda3-f3427d66b700" },  "origin": "124.192.129.84",  "url": "http://httpbin.org/get"}

3. Send a get request with ParametersPut the key and value in a dictionary and pass the params parameter. The function is equivalent to urllib. urlencode.

>>> Import requests >>> pqyload = {'q': 'yang yanxing' >>> r = requests. get ('HTTP: // www.so.com/s', Params = pqyload) >>> r. urlu 'HTTP: // www.so.com/s? Q = % E6 % 9D % A8 % E5 % BD % A6 % E6 % 98% 9f'

4. Send a post request,Passed through the data parameter,

>>> Payload = {'A': 'yang', 'B': 'hello'} >>> r = requests. post ("http://httpbin.org/post", data = payload)> print r. text {"args" :{}, "data": "", "files" :{}, "form": {"a": "\ u6768 ", "B": "hello"}, "headers": {"Accept": "*/*", "Accept-Encoding": "gzip, deflate", "Connection ": "close", "Content-Length": "19", "Content-Type": "application/x-www-form-urlencoded", "Host": "httpbin.org ", "User-Agent": "python-requests/2.3.0 CPython/2.6.6 Windows/7", "X-Request-Id": "c81cb937-04b8-4a2d-ba32-04b5c0b3ba98"}, "json": null, "origin": "124.192.129.84", "url": "http://httpbin.org/post" >>>

As you can see, the post parameter has been passed to form. data not only accepts dictionary-type data, but also supports json and other formats.

>>> Payload = {'A': 'yang', 'B': 'hello'} >>> import json >>> r = requests. post ('HTTP: // httpbin.org/post', data = json. dumps (payload ))

5. The post type of the file to be sent. This is equivalent to uploading an image or document to a website. In this case, the files parameter is used.

>>> url = 'http://httpbin.org/post'>>> files = {'file': open('touxiang.png', 'rb')}>>> r = requests.post(url, files=files)

Custom headers, transmitted using headers Parameters

>>> import json>>> url = 'https://api.github.com/some/endpoint'>>> payload = {'some': 'data'}>>> headers = {'content-type': 'application/json'}>>> r = requests.post(url, data=json.dumps(payload), headers=headers)

 
Vi. Response content
Response status code:

r = requests.get('http://httpbin.org/get')print r.status_code

Response Header:
 

>>> print r.headers{'content-length': '519', 'server': 'gunicorn/18.0', 'connection': 'keep-alive', 'date': 'Sun, 15 Jun 2014 14:19:52 GMT', 'access-control-allow-origin': '*', 'content-type': 'application/json'}

You can also obtain this individual response header for some judgment. The parameters here are case insensitive.

r.headers[‘Content-Type']r.headers.get(‘Content-Type')

Response content, which has already been applied:

r.textr.content

 
7. obtain cookies in response

>>> r = requests.get('http://www.baidu.com')>>> r.cookies['BAIDUID']'D5810267346AEFB0F25CB0D6D0E043E6:FG=1'

You can also customize COOKIES for requests.

>>> url = 'http://httpbin.org/cookies'>>> cookies = {'cookies_are':'working'}>>> r = requests.get(url,cookies = cookies)>>> >>> print r.text{ "cookies": {  "cookies_are": "working" }}>>>

There are still a lot of cookies, because I still don't have a lot of cookies at the moment. let's expand it later.

8. Use the timeout parameter to set the timeout time

>>> requests.get('http://github.com', timeout=1) <Response [200]>

If you set the time to a very small number, for example

requests.get('http://github.com', timeout=0.001)

If there is no connection within the timeout time, a Timeout exception will be thrown.

9. Use session in access
Initialize a session object first,

s = requests.Session() 

Then use this session object for access, r = s. post (url, data = user)
Access Renren's network to obtain the most recent access in the home page, and then access to view more visits to read more recent visits.
More visits are to access the http://www.renren.com/myfoot.do with session

# Coding: utf-8import requestsimport reurl = r 'HTTP: // www.renren.com/ajaxLogin'user = {'e-mail ': 'e-mail', 'Password': 'pass'} s = requests. session () r = s. post (url, data = user) html = r. textvisit = [] first = re. compile (R' </span> <span class = "time-tip first-tip"> <span class = "tip-content"> (. *?) </Span> ') second = re. compile (R' </span> <span class = "time-tip"> <span class = "tip-content"> (. *?) </Span> ') third = re. compile (R' </span> <span class = "time-tip last-second-tip"> <span class = "tip-content"> (. *?) </Span> ') last = re. compile (R' </span> <span class = "time-tip last-tip"> <span class = "tip-content"> (. *?) </Span> ') visit. extend (first. findall (html) visit. extend (second. findall (html) visit. extend (third. findall (html) visit. extend (last. findall (html) for I in visit: print iprint below are more recent visits to 'vm = s. get ('HTTP: // www.renren.com/myfoot.do') fm = re. compile (R' "name ":"(. *?) "') Visitmore = fm. findall (vm. text) for I in visitmore: print I

10. requests-cookies
Cookies store and save the values of each item like a dictionary, such as our username, password, and login information. when a Web page is loaded again, you can find related information from cookies and avoid re-entering the value assignment process.
When using get or other requests in requests, we can also grant cookie information. For example, we can obtain the cookies requested when a webpage is loaded from a browser and grant requests the same permission to use them again.
Cookies can be transmitted by adding the cookie = {key: value} parameter to the requests request.

import requestsurl = 'http://httpbin.org/cookies'cookies = dict(cookies_are='working')r = requests.get(url, cookies=cookies)r.text#'{"cookies": {"cookies_are": "working"}}'

It is very easy to query the cookies of a request, just like using the cookies property like getting headers:

url = 'http://example.com/some/cookie/setting/url'r = requests.get(url)r.cookies['example_cookie_name']# 'example_cookie_value'

The following function can break down the cookie string obtained by the browser into a dictionary to help us simulate requests.

def browsercookiesdict(s): '''Covert cookies string from browser to a dict''' ss=s.split(';') outdict={} for item in ss: i1=item.split('=',1)[0].strip() i2=item.split('=',1)[1].strip() outdict[i1]=i2 return outdict

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.