The Python requests installation _python the win and Linux system

Under Windows System, you can install it simply by entering the command PIP install requests.

Under Linux, you only need to enter the command sudo pip install requests to install.

Or

=================

Window

1. Download requests by Wall

Open this URL, http://www.lfd.uci.edu/~gohlke/pythonlibs on this site has a lot of Python third-party library files, ctrl+f find requests download

After downloading the. whl file, change the suffix name from. WHL to. zip, and then unzip the file, we can get two folders

Copy the Requests folder to the Lib directory under the Python installation directory

Requests already installed, enter import requests command to try to install successfully.

Import requests no error, indicating that requests has been successfully installed.

2. Quick Guide

2.1 Send Request
To send a request is simple, first import the requests module:

>>>import requests

Next, let's get a page, such as the homepage of my personal blog:

>>>r = requests.get('http://www.zhidaow.com')


Next, we can use the various methods and functions of this r.
In addition, there are many types of HTTP requests, such as Post,put,delete,head,options. can also be implemented in the same way:

>>> r = requests.post ("Http://httpbin.org/post")
>>> r = requests.put ("Http://httpbin.org/put" )
>>> r = Requests.delete ("Http://httpbin.org/delete")
>>> r = Requests.head ("http:// Httpbin.org/get ")
>>> r = requests.options (" Http://httpbin.org/get ")

Because I don't have any of these at the moment, so I haven't studied it in depth.

2.2 Passing parameters in URLs
Sometimes we need to pass parameters in the URL, such as in the acquisition of Baidu search results, our WD parameters (search term) and RN parameters (number of searches), you can be composed of url,requests also provides a look very NB method:

>>> payload = {' WD ': ' Zhang Yanan ', ' rn ': ' m '}
>>> r = Requests.get ("http://www.baidu.com/s", params= Payload)
>>> print r.url
u ' http://www.baidu.com/s?rn=100&wd=%E5%BC%A0%E4%BA%9A%E6%A5%A0 '

The above wd= garbled is "Zhang Yanan" of the transcoding form. (as if the parameters were sorted according to the first letter.) ）

2.3 Getting response content
You can get the contents of a Web page by R.text.

>>> r = requests.get (' https://www.zhidaow.com ')
>>> r.text
u ' <! DOCTYPE html>\n 
 

The document says that requests will automatically transfer the content to the code. Most Unicode fonts will be seamlessly transcoding. But when I use under the Cygwin always appear unicodeencodeerror error, depressed. It's perfectly normal in Python's idle.
In addition, you can get the page content by R.content.

>>> r = requests.get (' https://www.zhidaow.com ')
>>> r.content
B ' <! DOCTYPE html>\n 
 

The document says R.content is displayed in bytes, so it starts with B in idle. But I did not use it in the Cygwin, download the page just right. So instead of Urllib2 's Urllib2.urlopen (URL), read () function. (Basically the one feature I use most.) ）

2.4 Getting the page encoding

You can use r.encoding to get the page encoding.

>>> r = requests.get (' http://www.zhidaow.com ')
>>> r.encoding
' Utf-8 '

When you send a request, requests will guess the page encoding based on the HTTP header, and when you use R.text, requests will use the code. Of course you can also modify the requests code form.

>>> r = requests.get (' http://www.zhidaow.com ')
>>> r.encoding
' utf-8 '
>>> r.encoding = ' iso-8859-1 '

As in the above example, the modified encoding will directly use the modified code to get the content of the Web page.

2.5 JSON

Like Urllib and URLLIB2, if you use JSON, you'll introduce new modules, such as JSON and Simplejson, but there are already built-in functions in requests, R.json (). For a query IP API:

>>>r = Requests.get (' http://ip.taobao.com/service/getIpInfo.php?ip=122.88.60.28 ')
>>> R.json () [' Data '] [' country ']
' China '

2.6 Page Status Code
We can use R.status_code to check the status code of the page.

>>>r = Requests.get (' http://www.mengtiankong.com ')
>>>r.status_code
>> >r = Requests.get (' http://www.mengtiankong.com/123123/')
>>>r.status_code
404
>> >r = Requests.get (' http://www.baidu.com/link?url=QeTRFOS7TuUQRppa0wlTJJr6FfIYI1DJprJukx4Qy0XnsDO_ S9baoo8u1wvjxgqn ')
>>>r.url
u ' http://www.zhidaow.com/
>>>r.status_code
200

The first two examples are normal, return 200 that can normally be turned on, and return 404 that cannot be turned on normally. But the third is a bit strange, that is Baidu search results in the 302 jump address, but the status code display is 200, then I used a trick to let his true colours:

>>>r.history
(<response [302]>,)

Here you can see that he is using the 302 jump. Perhaps some people think that this can be judged and regular to get the status code of the jump, in fact, there is a simpler way:

>>>r = Requests.get (' http://www.baidu.com/link?url=QeTRFOS7TuUQRppa0wlTJJr6FfIYI1DJprJukx4Qy0XnsDO_ S9baoo8u1wvjxgqn ', allow_redirects = False)
>>>r.status_code
302

As long as the addition of a parameter allow_redirects, prohibit the jump, directly appear jump status code, easy to use it? I also use this in the last palm made a simple to get the Web page status code small application, the principle is this.

2.7 Response Header Content

The response header content can be obtained by r.headers.

>>>r = Requests.get (' http://www.zhidaow.com ')
>>> r.headers
{'
 content-encoding ': ' Gzip ',
 ' transfer-encoding ': ' chunked ',
 ' content-type ': ' text/html; Charset=utf-8 ';
 ...
}

You can see that everything is returned in the form of a dictionary, and we can also access some of the content.

>>> r.headers[' content-type ']
' text/html charset=utf-8 '

>>> r.headers.get (' Content-type ')
' text/html; Charset=utf-8 '

2.8 Setting Timeout time

We can set the timeout through the Timeout property, which prompts an error if the response content has not been received for more than this time.

Copy Code code as follows:

>>> requests.get (' http://github.com ', timeout=0.001)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Requests.exceptions.Timeout:HTTPConnectionPool (host= ' github.com ', port=80): Request timed out. (timeout=0.001)

2.9 Proxy Access
In order to avoid being blocked IP when collecting, agents are often used. Requests also has the corresponding proxies attribute.

Import requests

proxies = {
 "http": "http://10.10.1.10:3128",
 "https": "http://10.10.1.10:1080",
}

requests.get ("http://www.zhidaow.com", proxies=proxies)

This is required if the agent requires an account and password:

Proxies = {
 "http": "http://user:pass@10.10.1.10:3128/",
}

2.10 Request Header Content
The request header content can be obtained using r.request.headers.

>>> r.request.headers
{' accept-encoding ': ' Identity, deflate, compress, gzip ',
' Accept ': ' */* ', ' User-agent ': ' python-requests/1.2.3 cpython/2.7.3 windows/xp '}

2.11 Custom Request Headers
The disguise request head is often used when gathering, we can use this method to hide:

r = Requests.get (' http://www.zhidaow.com ')
print r.request.headers[' user-agent ']
#python-requests/1.2.3 cpython/2.7.3 windows/xp

headers = {' user-agent ': ' Alexkh '}
r = Requests.get (' http://www.zhidaow.com ', headers = headers)
print r.request.headers[' user-agent ']
#alexkh

2.12 Persistent Connection Keep-alive

The requests keep-alive is based on URLLIB3, and the persistent connection within the same session is completely automatic. All requests within the same session will automatically use the appropriate connection.

In other words, you do not need any settings, requests will automatically implement Keep-alive.

3. Simple Application

Get page return code

def get_status (URL):
 r = requests.get (URL, allow_redirects = False) return
 r.status_code

print get_status ( ' http://www.zhidaow.com ') 
#200
print get_status (' http://www.zhidaow.com/hi404/')
#404
Print get_ Status (' http://mengtiankong.com ')
#301
print get_status (' http://www.baidu.com/link?url= Qetrfos7tuuqrppa0wltjjr6ffiyi1djprjukx4qy0xnsdo_s9baoo8u1wvjxgqn ')
#302
print get_status (' http:// Www.huiya56.com/com8.intre.asp?46981.html ')
#500