Python crawler crawls a Han-han Sina Blog

Online to see the great god to Python crawler crawling to very much useful information, think very powerful. Suddenly want to learn Python crawler, although he did not learn python. But on the internet to look for some information to see the climb to the Han-Han Sina blog video. A total of three episodes, the first section to crawl a blog, the second section to crawl a page of blog. The third episode tells you to crawl all blogs.

Watched the video. Also left the code.

Crawler First step: Check the source of the Web page:

The code for the first blog is blue-bottomed section <a title= "" target= "_blank" href= "Http://blog.sina.com.cn/s/blog_ 4701280b0102eo83.html ">" on the seven elements of film--about my ...</a> on electricity

Find the Common section "< title=" ' href= ', '. html ', against the code of other blogs

The code is:

#-*-Coding:-utf-8-*-import urllibstr0 = ' <a title= ' "target=" _blank "href=" http://blog.sina.com.cn/s/blog_ 4701280b0102eo83.html ">" on the seven elements of film--About me ...</a> ' title = Str0.find (R ' <a title ') #print Titlehref = Str0.find (R ' href= ') #print hrefhtml = Str0.find (R '. html ') #print Htmlurl = str0[href + 6:html + 5]print urlcontent = Urllib.urlopen (u RL). Read () #print contentfilename = Url[-26:]print filenameopen (filename+ '. html ', ' W '). Write (content) print ' Download successful! '

Execution Result:

saved files:

Python crawler crawls a Han-han Sina Blog