Extracting site Address _ Regular expressions from search results based on Python regular expressions

Regular expressions are not unique to Python, and have recently exported all of the site addresses in the results of Google search, and have thought of using Python regular expressions to extract the site addresses from search results.

This involves several issues that need to be addressed:

1, get the results of the search text

To get more addresses, I use Google's Advanced search feature, which displays 100 results per page.

Get the results of the display, you can view the source code, and keep the text file has the results of the search text

2, analysis of how to extract the site information

First you need to analyze the retrieved pages to see how you can extract the site information.

I use the profiler feature in the IE8 's own development tool (which will bounce out of the F12) to see what special formatting is available for what you want to care about.

From the image above you can see that I need the site in the label <cite></cite>, so I use the regular expression to extract the text is it OK?

3, write regular expressions to get the site address

The next is to write the expression, I use Python3.2 written, convenient and easy (~_~)

The code is as follows: First, keep the search results page in E:/t3.txt and execute the following code

Import re
p = re.compile (R ' <cite> ([^<>\/].+?) </cite> ')
f = open ("E:/t3.txt", encoding= ' utf-8 ')
content = F.read ()
print (\ n). Join (P.findall ( Content)))

Run as follows:

We can compare the operation of the graph, to see all the site address is to get the.