"Python" HTTP client library requests & URLLIB2 and IP address processing ipy

Requests

Requests is a httpclient library that is more compact and easy to use than the URLLIB,URLLIB2 modules.

GET request

As an example, talk about how requests initiates and processes a GET request

R = Requests.get ("http://www.baidu.com")    # can add timeout parameter to set time-out

R is a response object, you can view a lot of information with R

If R.status_code view the HTTP return code for this request

R.headers header information (is a class Dictionary object)

R.url URL for this request

R.encoding Encoding method

R.text body

R.content body

The difference between *text and content is that text is in memory in the form of Unicode, and content is the product of encode after text is encoded with r.encoding. So when the returned content is HTML text, type (text) is Unicode and type (content) is str. When returning something like a picture, the encoding can be a binary encoding of Base64, so if you want to download a picture or a file, you should write to the local file that is also content instead of text.

　　　　R.cookies the content of the cookie in the returned content, if you want to add a cookie to the request, simply add cookie={to the parameter ...} Can

The R object also has several methods:

R.raise_for_status () #当返回码不是200的时候抛出一个HTTPError异常

In addition, the Get method can integrate a dictionary into a URL to form a GET request with parameters. Like what

par = {'wd':'word'= Requests.get ("  http://www.baidu.com/s", params=par)>>>r.urlu'http ://www.baidu.com/s?wd=word'

If you return a JSON string, then R can also use the JSON () method to construct the Python dictionary corresponding to this string.

R = Requests.get ("url") #  Assuming that the JSON returned is {"Code": "$", "MSG": "OK"}   = R.json () print res.get ("code"), Res.get ("    msg")# got the OK

　　

In addition to the params parameter, the Get method can have the proxies parameter to set the proxy, and the headers parameter to forge the request header

For example headers={' user-agent ': ' Test '};r = Requests.get (url,headers=headers)

Add cookie={A dictionary} to increase the cookie information in the request

POST

Post requests use the Post method, and get similar settings.

When you add the data to the post, note that the parameter is data not the params!! ==> r = requests.post (Url,data=par)

Urllib2

Urllib2 is closer to the bottom than requests, can achieve more personalized httpclient?

Basic usage

req = Urllib2. Request ('... ')    # create a Request object with a URL that can be either an HTTP protocol or an FTP protocol.  = Urllib2.urlopen (req)    # Open this Request object  = Response.read ()    #  response can read content like a file

If you are using the HTTP protocol for communication, users can enrich their HTTP request by post or get method, such as attaching some form data themselves, or adding headers information of HTTP

① Add form data

First, a Python dictionary is needed to abstract the data to be sent, and then make sure to use Urllib.urlencode to process the dictionary into something Urlopen can recognize (as if it were a key=value&key2=value2, The form that follows the URL of the GET request). Then there are two ways to create the request object, corresponding to the POST request and the GET request, respectively.

Par = {'WD':'Word'}processed_par= Urllib.urlencode (PAR)#The dictionary is processed using the UrlEncode method in Urllib.req= Urllib2. Request ('URL', Data=processed_par)#Write a parameter data= processing good data this way it corresponds to the POST requestreq = Urllib2. Request ("'Url'+'?'+processed_par) #直接加上处理好的数据是指GET请求的方式 (don't forget the middle question mark!!!) Urllib2.urlopen (req)#Open the Request object for Access

② Add header information

Header is the head, the HTTP header is part of the HTTP message, and its logical form is roughly a dictionary of content types, encoding, language, certificate information that the client can accept, what the user's proxy (the browser's brand) is, and so on. From the application logic, the header is divided into general Header,request Header,response Header and Entity header section Four

Because URLLIB2 is primarily an HTTP client library, it focuses on the part of the request header

You can add user-agent information to the header (what the user's browser is), as follows:

　　　　

URL ='http://www.xxxxxxxx ...'useragent='Mozilla 4.0 xxxxx'    #simulate Firefox's informationHeaders= {'user-agent': useragent}#Set Header dictionaryData= Urllib.urlencode (...)#form data for post requestsreq = Urllib2. Request (url,data=data,headers=headers) Res= Urllib2.urlopen (req)#This is the request to open the header information with your own definition .

Fragmented records.

* About error Urlerror & Httperror

These two errors are common errors when applying the HTTP client library. First, Httperror is a subclass of Urlerror, which happens when httperror occurs Urlerror

In addition, Urlerror usually refers to an error that does not have a network connection (no specific route to the specified server), or the server does not exist. Corresponding, httperror refers to the situation where the server points to the correct and the connection is OK, but the request cannot be completed.

Urlerror objects usually have a reason property, a tuple,[0] is an error number, [1] is an error message, so you can print Urlerror to view specific information when except E.reason. A similar httperror has a code attribute that records an HTTP return code, such as a well-known 404 representative that cannot find a file, 401 for authentication, and 500 for an internal server error, and so on.

　　

* Decoding and encoding of URLs

　　some symbols such as ' {', ' \ ' are not recognized, need to encode it into a%7b,%5c kind of something I also said not very clear what code , you can use Urllib.quote (URL) to encode the URL, The same urllib.unquote (URL) is a well-coded URL that decodes the form that adults can understand.

IPy

A small module: There is no place to put, anyway, this article seems to be with the network in place.

I wanted to find a module that could verify the legitimacy of the IP address, and the ipy seemed to be more powerful. ipyhttps://github.com/haypo/python-ipy/

Handling of IP

First use IPY.IP to create an IP object

ip = IP ('x.x.x.x')    # when constructing IP objects, the IP legitimacy is already judged, and if not, an error is made. My first requirement was fulfilled. 

The IP object has a __repr__ method that can be directly print into a string.

Ip.make_net (' subnet mask ') returns the address of the network segment where the IP is located, such as

ip = IP (127.0.0.1= ip.make_net ('255.0.0.0')print  net  ##输出结果就是 IP (' 127.0.0.0/8 ')

Print inch Net # #还可以直接用 in statement to determine that an IP is not in a network segment

Processing of the network segment address

At the time of construction, if a network is written, the IP class will automatically recognize it as a network

NET = IP ('127.0.0.0/24')

When this net object is iterable, it can iterate over all the IP addresses in this network segment, and with Len (net) it is natural to see how many IPs there are in this network segment.

In print net is the same time will be printed out of the network, just to indicate that a network has a number of methods, in the end what to take?

can use Net.strnormal (n) This method to change the printed format, such as the above net, if the set of n is 0, only print out the 127.0.0.0 (network address)

N 1 of the occasion to print out a form similar to 127.0.0.0/24

N 2 of the occasion to print out the form of 127.0.0.0/255.255.255.0

N 3 of the occasion to print out the form of 127.0.0.0-127.0.0.255

"Python" HTTP client library requests & URLLIB2 and IP address processing ipy