Python抓取電影天堂電影資訊的代碼

Python2.7Mac OS

抓取的是電影天堂裡面最新電影的頁面。連結地址： http://www.dytt8.net/html/gndy/dyzz/index.html

擷取頁面的中電影詳情頁連結

import urllib2import osimport reimport string# 電影URL集合movieUrls = []# 擷取電影列表def queryMovieList(): url = 'http://www.dytt8.net/html/gndy/dyzz/index.html'  conent = urllib2.urlopen(url) conent = conent.read() conent = conent.decode('gb2312','ignore').encode('utf-8','ignore')  pattern = re.compile ('
.*?>
'+      '(.*?) ',re.S) items = re.findall(pattern,conent)   str = ''.join(items) pattern = re.compile ('(.*?).*?(.*?)',re.S) news = re.findall(pattern, str) for j in news:    movieUrls.append('http://www.dytt8.net'+j[0])

抓取詳情頁中的電影資料

def queryMovieInfo(movieUrls): for index, item in enumerate(movieUrls): print('電影URL: ' + item) conent = urllib2.urlopen(item) conent = conent.read() conent = conent.decode('gb2312','ignore').encode('utf-8','ignore')  movieName = re.findall(r'
(.*?)
', conent, re.S) if (len(movieName) > 0):  movieName = movieName[0] + ""  # 截取名稱  movieName = movieName[movieName.find("《") + 3:movieName.find("》")] else:  movieName = "" print("電影名稱: " + movieName.strip()) movieContent = re.findall(r'(.*?)',conent , re.S) pattern = re.compile('
 

  (.*?)', re.S) movieDate = re.findall(pattern,movieContent[0]) if (len(movieDate) > 0):  movieDate = movieDate[0].strip() + '' else:  movieDate = "" print("電影發布時間: " + movieDate[-10:]) pattern = re.compile('
  

  
(.*?)
  

  
 0):  movieInfo = movieInfo[0]+''  # 刪除
  
標籤  movieInfo = movieInfo.replace("
  
","")  # 根據 ◎ 符號拆分  movieInfo = movieInfo.split('◎') else:  movieInfo = "" print("電影基礎資訊: ") for item in movieInfo:  print(item) # 電影海報 pattern = re.compile('', re.S) movieImg = re.findall(pattern,movieContent[0]) if (len(movieImg) > 0):  movieImg = movieImg[0] else:  movieImg = ""  print("電影海報: " + movieImg) pattern = re.compile('.*?', re.S) movieDownUrl = re.findall(pattern,movieContent[0]) if (len(movieDownUrl) > 0):  movieDownUrl = movieDownUrl[0] else:  movieDownUrl = "" print("電影下載地址：" + movieDownUrl + "") print("------------------------------------------------\n\n\n")
 

執行抓取

if __name__=='__main__':  print("開始抓取電影資料");   queryMovieList()  print(len(movieUrls))  queryMovieInfo(movieUrls)  print("結束抓取電影資料")

總結

學好Regex很重要,很重要,很重要!!!! Python的文法好有感覺, 對比Java …

