1 ImportUrllib.request,json2 3 4 5country=['ID','th','vn','inch','BR','Me','ru','na','HK']6url_lst=[7 '. server.voga360.com/nclient/sjson/playpicture/playpicture.htm?mtypecode=1',8 '. server.voga360.com/nclient/sjson/playpicture/playpicture.htm?mtypecode=2',9 '. Server.voga360.com/nclient/sjson/recommend/recommend.htm?a=recmd_app_home_1',Ten '. Server.voga360.com/nclient/sjson/recommend/recommend.htm?a=recmd_game_home_1', One '. Server.voga360.com/nclient/sjson/recommend/recommend.htm?a=recmd_app_releases_1', A '. Server.voga360.com/nclient/sjson/recommend/recommend.htm?a=recmd_game_releases_1', - '. server.voga360.com/nclient/sjson/subject/subjectinfo.htm?subjectinfoid=393', - '. server.voga360.com/nclient/sjson/subject/subject.htm?mtypecode=1', the '. Server.voga360.com/nclient/sjson/rank/topranklist.htm?mtypecode=2&typecode=0', - '. Server.voga360.com/nclient/sjson/rank/topranklist.htm?mtypecode=1&typecode=0', - '. Server.voga360.com/nclient/sjson/rank/topnewranklist.htm?mtypecode=1&typecode=0', - '. Server.voga360.com/nclient/sjson/rank/topnewranklist.htm?mtypecode=2&typecode=0', + '. Server.voga360.com/nclient/sjson/rank/toptrendranklist.htm?mtypecode=1&typecode=0', - '. Server.voga360.com/nclient/sjson/rank/toptrendranklist.htm?mtypecode=2&typecode=0', + '.server.voga360.com/nclient/sjson/category/categoriesdownload.htm?currentpage=1&mtypecode=1& Typecode=21', A '.server.voga360.com/nclient/sjson/category/categoriesdownload.htm?currentpage=1&mtypecode=2& Typecode=0', at '. server.voga360.com/nclient/sjson/category/categoriesrate.htm?currentpage=1&mtypecode=1&typecode=5 ', - '. server.voga360.com/nclient/sjson/category/categoriesrate.htm?currentpage=1&mtypecode=2&typecode=0 ', - '.server.voga360.com/nclient/sjson/category/categoriestrend.htm¤tpage=4&mtypecode=1& Typecode=19', - '.server.voga360.com/nclient/sjson/category/categoriestrend.htm¤tpage=4&mtypecode=2& Typecode=0', - '.server.voga360.com/nclient/sjson/category/categorieslatest.htm?currentpage=1&mtypecode=1& typecode=10', - '. Server.voga360.com/nclient/sjson/category/categorieslatest.htm?currentpage=1&mtypecode=2&typecode =0'] in - #country=[' GL ',] toUrla='/ http' + - the defCNT_MD (country,url_lst): * forUrlminchCountry: $ forUrlpinchUrl_lst:Panax Notoginseng Print(URLM,URLP) -urlr=urla+urlm+URLP the Print('Connect to%s\n'%urlr) +Fd=open ('T.log','W') Areq =urllib.request.Request (URLR) the +Response =Urllib.request.urlopen (req) -The_page =Response.read () $ Print(the_page,file=FD) $ - fd.close () -Fd=open ('T.log','R') thelines=Fd.readlines () -s=Lines[0]Wuyi Print('find MD5%d times\n\n\n\n'% S.count ('MD5')) the - WuCNT_MD (COUNTRY,URL_LST)
Looking for the MD5 field in the crawled stream file, this script also needs to add the processing when the error is returned, otherwise it will jump out of the stop, to be continue...//my the second day @sohu_cyou
Webserver Interface test Script