The Python standard library needs to be learned continuously for a long time. Next we will look at how we can better master the relevant technical information. I hope this will be helpful for your future use and learning. The following describes how to use it.
If the keyword I entered is passed to a program as the address parameter, the program will return a page with the top logo and search UI)/result/bottom copyright information ), what we need to get is the intermediate result section. We can use the urlopen method in the urllib of the Python standard library to obtain the strings of the entire page, and then parse these strings, there is a way to extract the intermediate result part, extract the string, and add your own header, top, and bottom, so that the prototype of searching for thieves is probably complete, next, write a test code.
- [code]
- # Search Thief
- # creator: Singo
- # date: 2007-8-24
- import urllib
- import re
- class SearchThief:
- " " "the google thief " " "
- global path,targetURL
- path = "pages\\ "
- # targetURL = "http://www.google.cn/search?complete=1&hl=zh-CN&q= "
- targetURL = "http://www.baidu.com/s?wd= "
- def __init__(self,key):
- self.key = key
- def getPage(self):
- webStr = urllib.urlopen(targetURL+self.key).read() # get the page string form the url
- self.setPageToFile(webStr)
- def setPageToFile(self,webStr):
- rereSetStr = re.compile( "\r ")
- self.key = reSetStr.sub( " ",self.key) # replace the string "\r "
- targetFile = file(path+self.key+ ".html ", "w ") # open the file for "w "rite
- targetFile.write(webStr)
- targetFile.close()
- print "done "
- inputKey = raw_input( "Enter you want to search --> ")
- obj = SearchThief(inputKey)
- obj.getPage()
- [/code]
Here, users are only required to enter a keyword, submit a request to the search engine, and save the returned page to a directory. This is just a test example. If you want to make a real search thief, you can add the extracted strings to the pre-designed template without saving this page. The extracted strings are directly displayed on the client in the form of web, in this way, you can steal the results of some search engines and construct new page rendering.
Let's take a look at the source code of the baidu search result page. There is a <DIV id = Div> </DIV> label in front of the table tag in the search structure, based on this label, we can get the result set of moving down two rows, So we add a method.
- getResultStr()
- [code]
- def getResultStr(self,webStr):
- webStrwebStrList = webStr.read().split( "\r\n ")
- line = webStrList.index( " <DIV id=Div> </DIV> ")+2 # get the line from " <DIV id=Div> </DIV> " move 2 line
- resultStr = webStrList[line]
- return resultStr
- [/code]
Now that we get the result list, we need to put the result list in the Custom page. We can say that this page is called a template:
- [Code]
- <! DOCTYPE html PUBLIC "-// W3C // dtd xhtml 1.0 Transitional // EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
- <Html xmlns = "http://www.w3.org/1999/xhtml">
- <Head>
- <Http-equivhttp-equiv = "Content-Type" content = "text/html; charset = gb2312"/>
- <Title> SuperSingo search-% title % </title>
- <Link href = "default/css/global.css" type = text/css rel = stylesheet>
- </Head>
- <Body>
- <Div id = "top">
- <Div id = "logo"> </div>
- <Div id = "searchUI">
- <Input type = "text" style = "width: 300px;"/>
- <Input type = "submit" value = "Search"/>
- </Div>
- <Div class = "clear"/>
- </Div>
- <Div id = "result_info">
- Job found: ××× records, time consumed ××× seconds
- </Div>
- <Div id = "result"> % result % </div>
- <Div id = "foot">
All the search structures here are from Baidu! Here, % title % and % result % are characters awaiting replacement. to replace these characters, we will add another method,
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
