Python makes the most beautiful crawler for applications

Android is the most beautiful application page crawler, crawler is very simple, the design of things to quite a lot of
File operations
Regular expressions
String substitution and so on

Import Requestsimport Reurl = "http://zuimeia.com" R = Requests.get (' http://zuimeia.com/community/app/hot/?platform= 2 ') pattern = Re.compile (r ') urllist = Pattern.findall (r.content) def requestsurl (URL): r = Requests.get (URL) title = re.fi Ndall (R ' "App-title" >
(.*?)
', r.content) #print title category = Re.findall (R ' (. *?) ', r.content) #print category describe = Re.findall (R ' (. *?) ', r.content) #print type (describe[0]) Strdescribe = Srtreplace (describe[0]) #print strdescribe downloadurl = Re.findall (R ' ', '
', '
', '
', '
', '
', '
', '
', '
   

    ','
  
    ','
    
', '
', '
 
   
', '', '', '', '', '', '','','', '' for eachlistreplace in listreplace:string = String.Replace (str (eachlistreplace), ' \ n ') string = String.Replace (' \ n ') , ') return stringdef categornfinal (category): Categoryfinal = ' for eachcategory in category:categoryfinal = Categoryfi Nal+str (eachcategory) + '--' return categoryfinaldef urlreplace (URL): url = url.replace (' & ', ' & ') return Urlrequestsurl ("http://zuimeia.com/community/app/27369/?platform=2") for eachurl in Urllist:eachurl = Url+eachurl Content = Requestsurl (eachurl) categoryfinal = ' title = content[0][0] Category = Categornfinal (content[1]) Strdescribe = CONTENT[2] DownloadURL = Urlreplace (content[3][0]) with open (' C:/wqa.txt ', ' A + ') as Fd:fd.write (' title: ' +title+ ' \ n ' + ') Category: ' +category+ ' \ n ' + ' Strdescribe: ' +strdescribe+ ' \ n ' + ' DownloadURL: ' +downloadurl+ ' \n\n\ N------------------------------------------------------------------------------------------------------------------------ -----\n\n\n ')

