Python requests grab cat's Eye movie

1. Website: HTTP://MAOYAN.COM/BOARD/4?

2. Code:

1 ImportJSON2  fromMultiprocessingImportPool3 ImportRequests4  fromRequests.exceptionsImportrequestexception5 ImportRe6 7 8 defget_one_page_html (URL):9     Try:TenResponse =requests.get (URL) One         ifResponse.status_code = = 200: A             returnResponse.text -         returnNone -     exceptrequestexception: the         returnNone -  - defparse_one_page (HTML): -Pattern = Re.compile ('<dd>.*?board-index.*?> (\d+) </i>.*?alt.*?src= "(. *?)". *?name "><a' ++'.*?> (. *?) </a>.*?star "> (. *?) </p>.*?releasetime "> (. *?) </p>' -+'. *?integer "> (. *?) </i>.*?fraction "> (. *?) </i>.*?</dd>', Re. S#. Can match any line break +  AItems =Re.findall (pattern,html) at     #(' 1 ', ' http://p1.meituan.net/movie/[email protected]_220h_1e_1c ', ' Farewell My Concubine ', ' \ n Starring: Leslie Cheung, 张丰毅secret, gong li \ n ') , ' Release date: 1993-01-01 (Hong Kong, China) ', ' 9. ', ' 6 '), -      forIteminchItems: -         yield { -             'Index': item[0], -             'Image': item[1], -             'title': item[2], in             'actor': Item[3].strip () [3:], -             ' Time': Item[4].strip () [5:], to             'score': item[5] + item[6] +         } -  the defwrite_to_file (content): *With open ('Result.txt','a', encoding='Utf-8') as F: $F.write (json.dumps (content, Ensure_ascii=false) +'\ n')#Import shortcut See alt+enter,content content is a dictionary, we want to make it into a string to write to the file, add line breaks, each line aPanax Notoginseng f.close () -  the defMain (offset): +URL ='http://maoyan.com/board/4?offset='+Str (offset) AHTML =get_one_page_html (URL) the      forIteminchparse_one_page (HTML): +         Print(item) -Write_to_file (item)#will become Unicode encoding, if you want to result.txt inside is Chinese, you need to modify the Write_to_file function, plus encoding= ' utf-8 ' and Ensure_ascii=false $  $ if __name__=='__main__': -     #For I in Range (Ten): -     #Main (i*10) the  -Pool =Pool ()WuyiPool.map (Main, [i*10 forIinchRange (10)])

View Code

3. Results:

Attention:

1. Regular match to take a good look at

2. Format the contents of the output into a dictionary of generators

3. When writing to a file, turn the Unicode encoding into a Chinese display

4. Process Pools pool. Achieve seconds capture

Python requests grab cat's Eye movie