Python Crawl Tour Discover albums

With the urllib.request provided by Python3 it is easy to crawl things on the web.

1. Urllib.request.urlopen (URL) opens the webpage and reads read ()

2, Python regular analysis of image links, such as <photo= ' http://img3A.hualvtu.com/272492/20150223/2143e9d2b51b397cda16.jpg ' >

3, Urllib.request.urlretrieve (URL, filename) download the corresponding URL image, save to filename

Plus create file directory Os.makedirs (), Log.txt text record

See the code in detail:

# coding = utf-8# by Qiuimport Re, osimport urllib.requestpage = ' http://fm.hualvtu.com/viewQuark.action?id= 10150223231300000165&un=woshiyyh&reply=false ' # download htmldef download_html (URL): HTML = Urllib.request.urlopen (URL). Read () return Html.decode () def getImage (HT): REG = R ' photo=\ ' (. *?\.jpg) \ ' dt= ' obj =        Re.compile (reg) imglist = Re.findall (obj, ht) folder = ' g:/download/photos/' If not os.path.exists (folder): Os.makedirs (folder) logfile = open (folder+ ' log.txt ', ' wt ') logfile.write (' Image download source ' + page + ' \ n ') s = 1 for I In Imglist:try:print (' Download%d pictures ...            '% s ') Urllib.request.urlretrieve (i, folder+ ' pic%s.jpg '% s) except:print ("Download Error") Logfile.write (i+ ' download error \ n ') continue Logfile.write (' picture%d link--'% s+ i + ' \ n ') s + = 1 logfile.close () print (' End of download ') HTML = download_html (page) getImage (HTML)

Painting Journey

Python crawl tour Discover albums