5th Chapter Web Downloader and URLLIB2 module

Web Downloader: Download the Web page URL on the Internet to a local tool

Store URL pages in the Internet via the Web downloader to a local or memory string

What kinds of Web downloader do python have?

1.urllib2 Python official Base module

2.requests Python third-party package is more powerful

Urllib2 download Page Method 1: The most concise method

URL-------------------------->urllib2.urlopen (URL)

Code:

Import Urllib2

/#直接请求

Response = Urllib2.urlopne (' http://www.baidu.com ');

/#获取状态码 If 200 indicates a successful acquisition

Print.response.getcode ();

/#读取内容

Cont = Response.read ();

Method 2:data, HTTP, header

Code:

Import Urllib2;
#创建Request对象
Request = Urllb2. Request (URL)
#添加数据
Request.add_data (' A ', ' a ')
#添加http的header
Request.add_header (' user-agent ', ' mozilla/5.0 ')
#发送请求获取结果
Response = Urllib2.urlopen (Request)

Method Three: Add a processor for a special scenario

Import Urllib2,cookielib
#创建cookie容器
CJ = Cookielib. Cookiejar ()
#创建1个opener
Opener = Urllib2.build_opener (urllib2. Httpcookieprocessor (CJ))
#给urllib2安装opener
Urllib2.install_opener (opener)
#使用带有cookie的urllib2访问网页
Response = Urllib2.urlopen (' http://www.baidu.com ')

5th Chapter Web Downloader and URLLIB2 module