Web Downloader: Download the Web page URL on the Internet to a local tool
Store URL pages in the Internet via the Web downloader to a local or memory string
What kinds of Web downloader do python have?
1.urllib2 Python official Base module
2.requests Python third-party package is more powerful
Urllib2 download Page Method 1: The most concise method
URL-------------------------->urllib2.urlopen (URL)
Code:
Import Urllib2
/#直接请求
Response = Urllib2.urlopne (' http://www.baidu.com ');
/#获取状态码 If 200 indicates a successful acquisition
Print.response.getcode ();
/#读取内容
Cont = Response.read ();
Method 2:data, HTTP, header
Code:
Import Urllib2;
#创建Request对象
Request = Urllb2. Request (URL)
#添加数据
Request.add_data (' A ', ' a ')
#添加http的header
Request.add_header (' user-agent ', ' mozilla/5.0 ')
#发送请求获取结果
Response = Urllib2.urlopen (Request)
Method Three: Add a processor for a special scenario
Import Urllib2,cookielib
#创建cookie容器
CJ = Cookielib. Cookiejar ()
#创建1个opener
Opener = Urllib2.build_opener (urllib2. Httpcookieprocessor (CJ))
#给urllib2安装opener
Urllib2.install_opener (opener)
#使用带有cookie的urllib2访问网页
Response = Urllib2.urlopen (' http://www.baidu.com ')
5th Chapter Web Downloader and URLLIB2 module