Python實現抓取網頁並且解析的執行個體

本文以執行個體形式講述了Python實現抓取網頁並解析的功能。主要解析問答與百度的首頁。分享給大家供大家參考之用。

主要功能代碼如下：

#!/usr/bin/python#coding=utf-8import sys import reimport urllib2from urllib import urlencodefrom urllib import quoteimport timemaxline = 2000wenda = re.compile("href=\"http://wenda.so.com/q/.+\?src=(.+?)\"")baidu = re.compile("更多知道相關問題.*?")f1 = open("baidupage.txt","w")f2 = open("wendapage.txt","w")for line in sys.stdin:  if maxline == 0:    break  query = line.strip();  time.sleep(1);  recall_url = "http://www.so.com/s?&q=" + query;  response = urllib2.urlopen(recall_url);  html = response.read();                                                     f1.write(html)  m = wenda.search(html);  if m:    if m.group(1) == "110":      print query + "\twenda\t0";    else:      print query + "\twenda\t1";  else:    print query + "\twenda\t0";  recall_url = "http://www.baidu.com/s?wd=" + query +"&ie=utf-8";  response = urllib2.urlopen(recall_url);  html = response.read();  f2.write(html)  m = baidu.search(html);  if m:    print query + "\tbaidu\t1";  else:    print query + "\tbaidu\t0";  maxline = maxline - 1;f1.close()f2.close()

希望本文所述對大家Python程式設計的學習有所協助。

