Crawling Blogs with Python

Crawl blogs by Wusheying in Python
Take the blog to climb Wang Yin for example:

Import reimport urllib2def gethtmlcode (URL): return Urllib2.urlopen (URL). Read () def findtitleurl (htmlstring):    Regtitleurl = Re.compile ("Href=\" (. +?) \ "")    return Regtitleurl.findall (htmlstring) def findtitlecontent (htmlstring): Regtitlecontent = Re.compile ("\" > (. +?) </a> ") return Regtitlecontent.findall (htmlstring) Htmlcode = Gethtmlcode (' http://www.yinwang.org/') Titlecontent = Findtitlecontent (htmlcode) Titleurl = Findtitleurl (htmlcode) for I in range (0, Len (titleurl)):p rint Titlecontent[i+3]print titleurl[i+8]htmlpage = Gethtmlcode (titleurl[i+8]) F = open ("%s.html"% (titlecontent[i+3]), ' WB ') F.write (htmlpage) f.close

Crawling Blogs with Python