Python crawler gets pictures and downloads saved to local instances

Today's small series for you to share a Python crawler to get pictures and download saved in the local instance, has a good reference value, I hope to be helpful to everyone. Let's take a look!

1. Grab a picture of the fried egg online

2. The code is as follows

* * *Import Urllib.requestImport OSDefUrl_open(URL): req=urllib.request.request (URL) req.add_header (' User-agent ',' mozilla/5.0 (Windows NT 6.3; WOW64; rv:51.0) gecko/20100101 firefox/51.0 ') response=urllib.request.urlopen (URL) html=response.read ()return HTML#to get the num of page like1,2,3,4 ...DefGet_page(URL): Html=url_open (URL). Decode (' Utf-8 ') a=html.find (' Current-comment-page ') +23#add The offset th arrive at the [2356] B=html.find ('] ', a) print (HTML[A:B])Return html[a:b]find the URL of IMGsandReturn the URL of arrDefFind_imgs(URL): Html=url_open (URL). Decode (' Utf-8 ') img_addrs=[] A=html.find (' img src= ')While a!=-1:b=html.find ('. jpg ', a,a+255)# if False:return-1If b!=-1:img_addrs.append (' http: ' +html[a+9:b+4])else:b=a+9 A=html.find (' img src= ', b) print (Img_addrs)return Img_addrs print (' http: ' +each ' Save the IMGsDefSave_imgs(Folder,img_addrs):For eachIn Img_addrs:filename=each.split (‘/‘)[-1]#get the last member of Arr,that is the namewith Open (Filename, ' WB ') as f:img = Url_open (each) f.write (img) def  Download_mm (Folder= ' mm ', Pages=10 ): Os.mkdir (folder) Os.chdir (folder) Url= ' http://jandan.net/ooxx/' Page_num=int (get_page (URL)) for i in Range (pages): page_num-= i page_url = URL +  ' page-' + str (page_num) +  ' #comments ' Img_addrs=find_imgs (page_url) Save_imgs ( Folder,img_addrs) if __name__ =  ' __main__ ': download_mm () * * *< /span>               

Above this Python crawler to get pictures and download saved to the local instance is a small part of the whole content to share to everyone, I hope to give you a reference, but also hope that we support a lot of small series!

There is a need for Python Chinese software, a variety of books, videos of small partners! Can add QQ group: 838197940! Enter the group notes "information" can be obtained free of charge!

Python crawler gets pictures and downloads saved to local instances