Python:requests Module

The Requests module is a module that I learned when I learned the crawler, its API is relatively simple and easy to use, here is the introduction of the method.

In fact, this is good to use, a few lines of code can get the content of a Web page:

Import Requestsurl = ' http://www.juzimi.com/ju/252304 ' response = requests.get (URL) print (response.text)

It supports many HTTP request Types : Get,post,put,Delete ,head,options

There is a 2 display method for the response content

. Content displayed in bytes, Chinese as characters

. Text in the form of text, two pictures will make sense.

this is content.

This is text

Requests will automatically decode the content from the server. Most Unicode character sets can be decoded seamlessly.

You can see What encoding the requests uses by. Encoding

You can also manually change the encoding that it uses r.encoding= ' gbk2312 '

Get requests can pass parameters

Import Requestsurl = ' http://www.juzimi.com/article/33125 ' payload = {' page ': ' 1 '}response = Requests.get (url,params= Payload) print (Response.text)

You can print a. URLto view the constructed URL

Customizing the request Header

Pass a dict to the heads parameter

headers = {' user-agent ': ' my-app/0.0.1 '}

r = Requests.get (URL, headers=headers)

send a post request

Payload = {' Key1 ': ' value1 ', ' key2 ': ' value2 '}

r = Requests.post ("Http://httpbin.org/post", Data=payload)

The Get method also has a cookie parameter

Timeout parameter

Access Proxy

Proxies = {           "http""http://10.10.10.10:8888" ,            " HTTPS " " http://10.10.10.100:4444 " ,           = Requests.get ('http://m.ctrip.com', proxies=proxies)

Python:requests Module