Python Reptile Soccer Teenager anime (image) download

?? Although the heat of the World Cup has faded, the World Cup has left countless memorable memories for the world's people, and I wonder if there is any Japanese team in your memory? The World Cup, the Japanese team's performance is a bright, hard to imagine, is such a team, more than 20 years or the level of our national football team, perhaps not as we replace him.
?? Football youngster (Captain Wing, キャプテン Wing) by the famous Japanese anime Gao Qiaoyang began to serial in 1981, from this anime accompanied with generation after generation of Japan, and even the world's children grow, but also in the intangible tangible to promote the progress of Japanese football. In this World Cup, in Japan and Belgium in the game, the Japanese fans held up the leader of the small wing of the picture is enough to prove the impact of this animation on Japanese football.
?? This article will show you how to use the Python crawler to download the football player's anime pictures.
?? First, we need to download the URL for: https://mhpic.samanlehua.com/comic/Z%2F%E8%B6%B3%E7%90%83%E5%B0%8F%E5%B0%86%E7%BF%BC%2F%E7%AC% AC01%E5%8D%B7%2F2.JPG-NORESIZE.WEBP, as follows:

We note that only the number of volumes and the number of anime images in this web site has changed, so we just need to find the total number of volumes and the images contained in each volume to complete this crawler.
?? However, it is a little bit to note that the crawler downloaded the image format for the WEBP format. WebP (pronunciation weppy, Project home page), is a picture file format that supports lossy compression and lossless compression, derived from the image Encoding format VP8. According to Google's tests, lossless compressed WebP has a 45% less file size than PNG files, and WebP can reduce the file size by 28% even after the PNG files are compressed by other compression tools.
?? We want to be able to convert images in WEBP format into PNG format. Therefore, we need to install the WEBP software on the Linux system in the following ways:

• Ubuntu:sudo Apt-get Install WEBP
• Centos:yum-y Install Libwebp-devel Libwebp-tools

After the installation, the following command will be able to speak the WEBP format of the picture converted to PNG format:

dwebp picture.webp -o picture.png

?? The whole reptile idea is finished, we use multi-threading to download pictures and image format conversion operations, the complete Python code is as follows (need to install WEBP, and save the directory needs to be set):

#-*-Coding:utf-8-*-import urllib.requestimport osimport timefrom concurrent.futures import ThreadPoolExecutor, wait, A Ll_completedglobal Count # download failed pictures count = 0# parameter: # dir: Picture saved directory # Juanshu_num: Number of Volumes # page: Pages # This function: Download a picture of a specific page of a specific volume to the specified save directory D EF Get_webp (dir, Juanshu_num, page): # stitching downloaded image URL Base_url = ' https://mhpic.samanlehua.com/comic/Z%2F ' name = URL Lib.parse.quote (' soccer youngster wing ') Juanshu = ' 0 ' +str (juanshu_num) if juanshu_num<10 else str (juanshu_num) Juanshu = Urllib.    Parse.quote ('%s volume '%juanshu) format = '. Jpg-noresize.webp ' url = base_url+name+ '%2f ' +juanshu+ '%2f ' +str (page) +format        # print (URL) try:urllib.request.urlretrieve (URL, '%s/%d.webp '% (dir, page)) # Download picture print (' Start conversion image format: ') Os.system (' dwebp%s/%d.webp-o%s/%d.png '% (dir, page, dir, page) # Converts the image from WEBP format to PNG format print (' Conversion picture format is complete. ') Os.system (' Rm-rf%s/%d.webp '% (dir, page)) # Remove picture from WEBP format except Exception as Err:print (ERR) # parameter: Juans Hu_num: Number of Volumes # Page_num: the volumeNumber of pictures # This function works: Download all pictures in a volume def download (Juanshu_num, page_num): # If the directory does not exist, create a new directory if not os.path.exists ('/home/tsubas A/volume%s '%juanshu_num): Os.mkdir ('/home/tsubasa/volume%s '%juanshu_num) dir = '/home/tsubasa/volume%s '%juanshu_num # download in each volume All pictures for page in range (1, page_num+1): Try:get_webp (dir, juanshu_num, page) except Urllib.er Ror. Httperror:print (' The picture does not exist or the network connection is wrong. ') COUNT + = 1def Main (): Start_time = Time.time () # Number of pictures per volume, altogether 21 volumes page_num_list = [175, 175, 165, 17 1, 169, 172, 170, 170, 168, 174, 171, 168, 168, 168, 176, 169, 171, 167, 166, 172, 172] # Set the number of threads    for 10 executor = Threadpoolexecutor (max_workers=10) # You can adjust the parameters of the Max_workers # submit () yourself: The first is a function, and then the passed-in parameter for the function, allowing multiple # concurrent Download Picture future_tasks = [Executor.submit (download, juanshu_num+1, page_num) for Juanshu_num,page    _num in Enumerate (page_num_list)] Wait (Future_tasks, return_when=all_completed) # Wait for all the tasks to endEnd_time = Time.time () print (' Picture download finished! ' Total time is%s seconds. '% (end_time-start_time)) print (' Download failed image count:%d '%count) main ()

Run the above program, silently wait for the program to complete, the output results are as follows:

?? Let's go to Linux and look at the pictures that are already in good condition:

Altogether downloaded 3577 pictures (not a download failed), with about 1521 seconds, the efficiency of the leverage, haha ~ ~

?? This article is about to end, and finally add a sentence: Chinese football, refueling Ah!

Note: I have now opened two public number: because Python (number: Python_math) and easy to learn the Python crawler (number: Easy_web_scrape), welcome to the attention OH ~ ~

Python Reptile Soccer Teenager anime (image) download