Python common modules

Urllib

1. Urllib.urlopen () Open web page

From urllib import Requestimport jsonresponse = Request.urlopen ("http://www.baidu.com") #获取的网页信息html = Response.read (). Decode ("Utf-8") print (HTML)

Urlopen returns an object that supports operations:

• Read () ReadLine () ReadLines () Fileno () Close () These methods are used in exactly the same way as file objects
• Info () returns a httplib. Httpmessage object that represents the header information returned by the remote server
• GetCode () returns the HTTP status code. If it is an HTTP request, 200 request completed successfully; 404 URL not Found
• Geturl () returns the requested URL

Urlopen returns the properties of an object:

• Status return state?
• Reason Returning status information

Examples of operations:

#header头信息info = Response.info () print (info)  #返回码code = Response.getcode () print (code) #访问url信息url = Response.geturl () print (URL)                 #遍历所有header信息 for k,v in Info.items ():     print ("%s with%s"% (k,v)) # Get specific content information in the header, including date,server, etc. if "date" in Info:     print (info["date"])

Another way to do this:

From urllib import requestwith request.urlopen (' https://api.douban.com/v2/book/2129650 ') as f:    data = F.read () Print (    ' Status: ', F.status, F.reason)                       for K, V in F.getheaders ():        print ('%s:%s '% (k, v))    print (' Data: ') , Data.decode (' Utf-8 '))

　

2. Urllib.urlretrieve () Save the Web page to a local

3. Urllib.urlencode (query)

The key-value pairs in the URL are separated by connectors & can be combined with urlopen to implement the Post method and the Get method

Get:

>>> Import urllib>>> params=urllib.urlencode ({' spam ': 1, ' eggs ': 2, ' bacon ': 0}) >>> params ' Eggs=2&bacon=0&spam=1 ' >>> f=urllib.urlopen ("http://python.org/query?%s"% params) >>> Print F.read ()

Post:

>>> import urllib>>> Parmas = Urllib.urlencode ({' spam ': 1, ' eggs ': 2, ' bacon ': 0}) >>> f= Urllib.urlopen ("Http://python.org/query", Parmas) >>> F.read ()

　　

Case:

1. Simple download of images via Urllib

url = "Http://n.sinaimg.cn/tech/transform/20170512/P0He-fyfeutp7573474.jpg" from urllib import requestresponse = Request.urlopen (URL) html = Response.read () with open ("A.img", "WB") as F:    f.write (HTML)

2. Call Baidu's online translator via post:

From urllib import request, Parseimport Jsonreq = Request. Request ("Http://fanyi.baidu.com/v2transapi") Req.add_header ("User-agent", "mozilla/5.0" (Windows NT 10.0; Win64; x64) applewebkit/537.36 (khtml, like Gecko) chrome/51.0.2704.79 safari/537.36 edge/14.14393 ") Req.add_header (" Content-type "," application/x-www-form-urlencoded; Charset=utf-8 ") Req.add_header (" Referer "," http://fanyi.baidu.com ") Post_data = {} #英文到中文的翻译 # post_data[" from "] =" en "# post_data["Query"] = "Hello World" # post_data["simple_means_flag") = 3# post_data["to"] = "en" #中文到英文的翻译post_data ["from" ] = "zh" post_data["query"] = "Hello" post_data["simple_means_flag"] = 3post_data["to"] = "en" Post_data = Parse.urlencode ( Post_data) Response = Request.urlopen (req, Data=post_data.encode ("Utf-8")) HTML = Response.read (). Decode ("Utf-8") target = json.loads (HTML) print (target["Trans_result" ["Data"][0]["DST"]) #print (HTML)

3. Get Sina News information via get:

From urllib import Request,parseimport  json#http://feed.mix.sina.com.cn/api/roll/get?pageid=1&lid= 21values = {    "PageID": 1,    "lid": 21}param = Parse.urlencode (values) req = Request. Request ("http://feed.mix.sina.com.cn/api/roll/get?%s"% param) req.add_header ("User-agent", "mozilla/5.0 (Windows NT 10.0; Win64; x64) applewebkit/537.36 (khtml, like Gecko) chrome/51.0.2704.79 safari/537.36 edge/14.14393 ") req.add_header (" Referer " , "http://tech.sina.com.cn/") result = Request.urlopen (req) HTML = Result.read (). Decode ("Utf-8") target = Json.loads ( HTML) #print (target) News_time = target["Result" ["timestamp"]all_data = target["Result"] ["Data"]print (News_time) for I in All_data: print ("    title:", i["title"])    print ("\ t rollup:", i["Summary"])    print ("\ t info:", i["Intro"])    Print ("\turl:", i["url"])    print ("\timg:", (i["img" ["U"]))

　

4.

Python common modules