Python Crawl page link __python

http://blog.csdn.net/cscmaker/article/details/8730153

(1) For project needs, the need to crawl from the Web related pages. Just want to learn python, first read the Python concise tutorial, not much, but to enable you to quickly get started, I always think that instance-driven learning is the most effective way. So it's much better to learn more about Python by doing a hands-on job of crawling Web pages.

Python offers a wide variety of libraries, making it easy to operate on a variety of operations. The Python urllib2 and sgmllib libraries are used here. A total of three modules were provided for processing Html,python: Sgmllib htmllibhtmlparser. Sgmllib is used in this article, but finding the relevant data finds that the Third-party tool BeautifulSoup is the best and can handle poor HTML. So I have to learn beautifulsoup later.

(2) Script code[Python] View Plain copy Class linksparser (sgmllib. Sgmlparser):    urls = []    def do_a (self, attrs):      for name, value in attrs:      if name  ==  ' href '  and value not in self.urls:        If value.startswith (' http '):         self.urls.append (value)           print value       else:         continue       return      P =  linksparser ()    f = urllib2.urlopen (' http://www.baidu.com ')    #f  = urllib2.urlopen (' https://www.googlestable.com/search?hl=zh-CN&site=&source=hp&q=%E9 %bb%84%e6%b8%a4++%e6%b3%b0%e5%9b%a7&btnk=google+%e6%90%9c%e7%b4%a)    P.feed (F.read ())    for url in p.urls:     print url    f.close ()    p.close ()