Httplib urllib urllib2 Pycurl Comparison

Recent online interviews have seen questions about this, as these libraries have been used more or less in the last two months, and are now summed up in their own experience and on-line presentations.

Httplib

A client protocol that implements HTTP and HTTPS is not generally used directly, and its HTTP implementation is used in the Python's higher-level encapsulation module (URLLIB,URLLIB2). So the efficiency is a little higher than the two libraries. Http://www.cnblogs.com/beiyeren/p/4046139.html

#-*-coding:utf-8–*- fromUrlparseImportUrlparseImportHttplib fromHds.utilsImportGet_loggerImportJSON fromexceptionImportHttpExceptionclassHttpclienttmonitor (object):def __init__(Self, logger=None): Self.logger= LoggerorGet_logger ("Httpclienttmonitor")    defsend_request_with_body (self, URL, content, method, token): Self.logger.info ("URL:"+URL) self.logger.info (method+": "+Str (content))assertLen (URL)! = 0,"URL is empty"        assertLen (token)!=0,"token is empty"o=urlparse (URL) content= Json.dumps (content). Encode ('Utf-8') Headers= {"Content-type":"Application/json"} headers["Authorization"] =token httpClient=Httplib. Httpconnection (O.hostname, O.port)Try: Httpclient.set_debuglevel (0) Httpclient.request (method, O.path+"?"+o.query, content, headers) Response=Httpclient.getresponse ()if  notResponse:return "No response from HttpClient"Self.logger.info ("Status:"+str (response.status)) Self.logger.info ("Reason:"+Response.reason)returnResponse.StatusexceptException as E:RaiseHttpException (e)finally:            ifhttpClient:httpClient.close ()classHttp_request_method (): Get="GET"Post="POST"Delete="DELETE"put="PUT"Patch="PATCH"

Urllib2AndUrllibEach has its own characteristics:

• urllib2Can accept a Request object to set the headers for a URL request, urllib accepts only a URL. That's means, you cannot masquerade your User Agent string etc.

Urllib2 can accept an instance of the request class to set the URL request headers,urllib only the URL can be accepted. This means that you can not disguise your user agent string and so on.

• urllibProvides urlencode the method which is used for the generation of GET query strings, urllib2 doesn ' t has such a function. This is one of the reasons, why are urllib often used along with urllib2 .

Urllib provides the UrlEncode method for generating a Get query string, and Urllib2 does not. This is why Urllib often uses it with URLLIB2.

Pycurl

About Python network programming, when using Urllib to communicate with the server, the client's data is added to the head through the URL, sent to the server side, the Urllib package implementation of the client upload file, there will be a dead state (in fact, the time-out setting problem).
The Pycurl package is a Libcurl Python interface that is written in the C language. It is much faster than urllib.
Libcurl is a client URL transfer library that supports FTP, FTPS, HTTP, HTTPS, GOPHER, TELNET, DICT, FILE, and LDAP. Libcurl also supports HTTPS authentication, HTTP post,http put,ftp on Transmission, Proxy, Cookies, Basic authentication, FTP file breakpoint relay, HTTP proxy channel and so on.

classCurltest (object):defpost (self): provisioncontent["metadata"] =Metadata B=Stringio.stringio () C=Pycurl. Curl () c.setopt (Pycurl. VERBOSE,1) c.setopt (Pycurl. URL, Remoteserviceurl) c.setopt (Pycurl. Httpheader, ["Accept:application/json",                                     "Content-type:application/json"                                     ])
C.setopt (Pycurl. Writefunction, B.write) c.setopt (Pycurl. Followlocation,1) c.setopt (Pycurl. Maxredirs,5) c.setopt (C.postfields, Json.dumps (provisioncontent)) C.perform ()PrintC.getinfo (C.http_code), C.getinfo (C.effective_url), C.postfieldsPrintb.getvalue () self.response=eval (B.getvalue ()) B.close () c.close ()

Among them c.setopt (Pycurl. Writefunction, B.write) #把StringIO的写函数注册到pycurl的WRITEFUNCTION中, that is, pycurl all acquired content is written to Stringio, without this sentence, Pycurl will output all the content in the default output.

Pycurl common operations are as follows:

Pycurl. Curl () #创建一个pycurl对象的方法
Pycurl. Curl (Pycurl. URL, http://www.google.com.hk) #设置要访问的URL
Pycurl. Curl (). setopt (Pycurl. Maxredirs, 5) #设置最大重定向次数
Pycurl. Curl (). setopt (Pycurl. ConnectTimeout, 60)
Pycurl. Curl (). setopt (Pycurl. TIMEOUT, #连接超时设置
Pycurl. Curl (). setopt (Pycurl. UserAgent, "mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1,. NET CLR 1.1.4322) #模拟浏览器
Pycurl. Curl (). Perform () #服务器端返回的信息
Pycurl. Curl (). GetInfo (Pycurl. Http_code) #查看HTTP的状态 similar to the status attribute in Urllib
Pycurl. Namelookup_time Domain Name resolution time
Pycurl. Connect_time Remote Server Connection time
Pycurl. Pretransfer_time the time after the connection is over to the start of the transfer
Pycurl. Starttransfer_time the time that the first byte was received
Pycurl. Total_time the total time of the previous request
Pycurl. Redirect_time If there is a turn, the time spent
Pycurl. Effective_url
Pycurl. Http_code HTTP Response Code
Pycurl. Number of Redirect_count redirects
Pycurl. Size_upload size of uploaded data
Pycurl. Size_download the size of the downloaded data
Pycurl. Speed_upload upload Speed
Pycurl. Header_size Head Size
Pycurl. Request_size Request Size
Pycurl. Content_length_download Download Content Length
Pycurl. Content_length_upload Upload Content Length
Pycurl. Types of Content_Type Content
Pycurl. Response_code Response Code
Pycurl. Speed_download Download speed
Pycurl. Ssl_verifyresult
Time information for the Pycurl.info_filetime file
Pycurl. Http_connectcode HTTP Connection Code
Pycurl. Httpauth_avail
Pycurl. Proxyauth_avail
Pycurl. Os_errno
Pycurl. Num_connects
Pycurl. Ssl_engines
Pycurl.info_cookielist
Pycurl. Lastsocket
Pycurl. Ftp_entry_path

Httplib urllib urllib2 Pycurl Comparison