How Python extracts hyperlinks in Web pages

Many people begin to learn python at the beginning, and are going to use it as a crawler development. Since you want to be a crawler, you must first crawl the Web page and extract the hyperlink address from the Web page. This article for everyone to share a simple method, there is a need to refer to the reference.

Here is the simplest way to do this, first grab the target page back and then get the hyperlink by matching the href attribute in the a tag.

The code is as follows:

Import urllib2import Re url = ' http://www.sunbloger.com/' req = urllib2. Request (URL) con = urllib2.urlopen (req) doc = Con.read () con.close () links = Re.findall (R ' href\=\ "(http\:\/\/[a-za-z0-9\ . \/]+) \ "', doc) for a in Links:  print a