Python crawler crawl to get fried egg net sister map

This article obtains and downloads the fried egg mesh from the simple Python crawler (not using frames, for entertainment only) to specify the page or all pictures, and download the picture to disk.

Import modules First: Urllib.request, RE, os

Import urllib.request Import Re Import OS

Urllib.request module for obtaining HTML page data

The RE module is used to parse and intercept HTML page picture URLs through regular expressions

OS Module for folder-related operations

Code is not much, directly posted, the code is interpreted in the comments:

defCrawl_jiandan (page, path):""":p Aram Page: Gets the specified paging data with a value of 0 or exceeds the maximum to crawl all data:p Aram path: File storage path, no directory to create directory"""    ifPage <0:return    #whether the path exists, does not exist, creates a directory    if  notos.path.exists (path): Os.mkdir (path)#Switch to Directoryos.chdir (path)#Fried Egg Net sister map homeURL ='http://jandan.net/ooxx/page-%d#comments'%page whiletrue:request=urllib.request.Request (URL) request.add_header ('user-agent',                           'mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:54.0) gecko/20100101 firefox/54.0') with Urllib.request.urlopen (request) as response:html= Response.read (). Decode ('Utf-8')            #print (HTML)Items = Re.findall (Re.compile (R'<div class= "text" ><span class= "Righttext" ><a'R'href= "http://.+" >[0-9]+</a></span><p><a'R'href= "//(. +\.jpg)" target= "_blank" class= "View_img_link" >'), HTML) Next_url= Re.findall (Re.compile (R'<a title= "older Comments" href= "('R'http://jandan.net/ooxx/page-[0-9]+ #comments) "'R'class= "Previous-comment-page" >'), HTML) forIteminchItems:filename= Item.split ('/') [-1]                #if the same file name already exists in the directory, the download does not exist                if  notos.path.exists (filename):Print(item) Urllib.request.urlretrieve ('/ http'+item, filename)if  notLen (next_url):return            Else: URL=Next_url[0]crawl_jiandan (1,'/volumes/ke/it source/python3/python crawler/fried Egg net sister figure')

To open this disk, the effect is as follows:

Only part of the image is shown here, interested can download all the girls in the fried egg net, just in the above function in the first parameter 0 can be

Note: This article is for informational and entertainment purposes only and the code is not rigorous enough.

Python crawler crawl to get fried egg net sister map