Python Crawler---Requests library get started quickly

I. Introduction of Requests Library

Requests is an HTTP-related library for Python

Requests Installation:

PIP Install requests

Second, GET request

 Import Requests   #  First imported reqeusts module  = Requests.get (     #  Send a GET request using the requests emulation browser    url="https://www.baidu.com",   #  Specify the URL to visit )#  Print response content: Web site source code Print(res.text) # print binary response content; We are using music, video, etc. Print (res.content)

Reqeusts First Experience

When we send a GET request, if the URL parameter needs to carry parameters, then the data is placed in the URL in the form of a key/value pair followed by a question mark, for example: Ttps://www.baidu.com/s?wd=hello

ImportRequests#GET request, carrying parameter information#http://httpbin.org/get?key1=value1&key2=value2Payload = {'Key1':'value1','Key2':'value2'}r= Requests.get ("Http://httpbin.org/get", params=payload)Print(R.text)

GET request URL carrying parameters

You can also pass in a list as a value:

 #  http://httpbin.org/get?key1=value1& Key2=value2&key2=value3  payload = { "  Key1   ": "   value1   ", "   Key2   ": ["   value2   ", "   Value3   " ]}r  = Requests.get ( " http://httpbin.org/get   ", Params=payload) 

transmission of multiple worth of timeThird, POST request

Usually, if you want to send some data encoded as a form, simply pass a dictionary to the request. Your data is automatically encoded as a form when you send a request:

 import   Requestspayload  = { " key1  " :  " value1  " ,  " key2   ": "  value2   "  " #   similar to the data in our request body  r = requests.post ( " http://httpbin.org/ Post  , Data=payload)  print  (R.text) 

POST Request

Most of the time, the data you want to send is not encoded as a form, and if you want to transfer a JSON string instead of a dictionary, the data will be sent directly past

For example, the Github API V3 accepts Post/patch data encoded as JSON:

Import Requests Import  = {'some'data'  Https://api.github.com/some/endpoint'= requests.post (URL, data=json.dumps ( Payload))print(r.text)

POST request sends a JSON stringIv. Summary of requests parameters

:p Aram Method: Request methods:p Aram URL: URL of destination address:p Aram params: Request parameter: Can be string, byte, dictionary:p Aram data: Can be dictionary, string, byte, file object , it carries:p Aram JSON in the request body when it is sent: serializes the corresponding data in the JSON into a string, sends it to the server in the request body, and the content-type is {'Content-type':'Application/json'}:p Aram headers: request header data:p Aram cookies: The requested cookie:p Aram files: Send file data to the server:p Aram Auth:auth tuple to enable basic SummaryCustom HTTP auth.    :p Aram Timeout: Waits for the server response time, which can be a floating-point number:p Aram Allow_redirects: Whether automatic redirection is allowed, True (default), Flase:p Aram proxies: Dictionary mapping Protocol to the proxy URL. :p Aram Verify: Control Whether we are verifying the TLS certificate of the server or the string, in which case it must be the path of the CA package to be used.    The default is "True". :p Aram Stream: Whether the data from the server is read as a stream:p Aram cert: The path to the SSL client certificate file (. PEM) If it is a string. If it is a tuple, (' cert ', ' key ') pairs. 

V. Response

ImportRequestsImportJsonpayload= {'some':'Data'}url='Https://api.github.com/some/endpoint'R= Requests.post (URL, data=json.dumps (payload))#Get response HeaderPrint(r.headers)Print(r.headers['Content-type'])#' Application/json 'Print(R.headers.get ('Content-type'))#' Application/json '#Get Response CookiePrint(r.cookies['Example_cookie_name'])Print(R.cookies.get_dict ())#get the response bodyPrint(R.text)#Get response body (binary form)Print(r.content)#redirect and request HistoryPrint(R.url)Print(R.status_code)#Response Status CodePrint(r.history)#JSON response ContentPrint(R.json ())#[{u ' repository ': {u ' open_issues ': 0, U ' url ': ' https://github.com/...

Python Crawler---Requests library get started quickly