Python crawler Ideas

Python2
Crawler: Take data from a webpage
Crawler module: URLLIB,URLLIB2,RE,BS4,REQUESTS,SCRAPY,XLML
1.urllib
2.request
3.bs4
4. Regular RE
5 Types of data
(1) Number
(2) Strings string
(3) List list[] Chinese in an iterative object is a Unicode object
(4) tuple tuples ()
(5) Dictionary set{}
Crawler ideas:
1. Static Urlopen Open the webpage------get the source read
2.requests (module) Get/post request----Get the source text () method content () method (recommended)
3.BS4 can parse HTML and XML
#- -Coding:utf-8--
From BS4 import BeautifulSoup
#1
#html = "<div>2018.1.8 14:03</div>"
#soup =beautifulsoup (HTML, ' Html.parser ') #解析网页
#print Soup.div
#2从文件中读取
Html= "
Soup=beautifulsoup (Open (' index.html '), ' Html.parser ')
Print soup.prettify ()
4. Get the information you need

Python crawler Ideas