Python Basic Learning Web Crawler

Python Basic Learning Web Crawler

# Baidu Post Bar image web crawlers # import reimport urllib def getHtml (url): page = urllib. urlopen (url) html = page. read () return html def getImg (html): reg = r'src = "(. +? \. Jpg) "pic_ext 'imgre = re. compile (reg) imglist = imgre. findall (html) x = 0 l = len (imglist) print "% d images in total" % (l) print "-------------------" for imgurl in imglist: print "% d image" % (x + 1) urllib. urlretrieve (imgurl, 'e: \ Pythoncode \ picture \ % s.jpg '% x) x = x + 1 html = getHtml ("http://tieba.baidu.com/p/3093487131") getImg (html)

There are 38 images in total ------------------- 1st images 2nd images 3rd images 4th images 5th images 6th images 7th images 8th images 9th images 10th images 11th images 12th Images images, 13th images, 14th images, 15th images, 16th images, 17th images, 18th images, 19th images, 20th images, 21st images, 22nd images, 23rd images, 24th images, 25th Images 26th images 27th images 28th images 29th images 30th images 31st images 32nd images 33rd images 34th images 35th images 36th images 37th images 38th images Images