How to download Baidu music using Python multi-thread and queue

This article mainly introduces the method of downloading Baidu music by using Python multithreading and queuing. The example analyzes the implementation skills of Python multithreading and file downloading, which has some reference value, for more information about how to download Baidu music using Python multithreading and queue, see the example in this article. Share it with you for your reference. The details are as follows:

I have always wanted to download a script for music. Later I decided to take Baidu music for a knife. After several analyses, I finally produced a script for downloading Baidu music. Currently, only the first page is downloaded by default, kids shoes can be freely expanded.
Applicable to Windows and Linux platforms, relying on the BeautifulSoup library, mainly for HTML Parsing

#! /Usr/bin/python #-*-coding: UTF-8-*-''' batch download of a singer's songs in Baidu (currently, only the first page can be downloaded and expanded on its own) @ author: admin @ qq: 1243385033 ''' import threading, urllib2, OS, re, sysfrom bs4 import BeautifulSoupfrom Queue import Queue ''' target SINGER ''' SINGER = u'yadong ''' ''save path ''' SAVE _ FOLDER = 'f: /music/'# query urlsearch_url =" http://music.baidu.com/search/song?key=%s&s=1 "# Baidu Music Album urlsong_url =" http://box.zhangmen.baidu.com/x?op=12&count=1&mtype=1&title= "Class Downloader (threading. thread): def _ init _ (self, task): threading. thread. _ init _ (self) self. task = task def run (self): ''' overwrites the run method of the parent class ''' while True: url = self. task. get () self. download (url) self. task. task_done () def build_path (self, filename): join = OS. path. join parentPath = join (SAVE_FOLDER, SINGER) filename = filename + 'users' myPath = join (parentPath, filename) return myPath def download (self, url ): '''download file ''' sub_url = url. items () f_name = sub_url [0] [0] req_url = sub_url [0] [1] handle = urllib2.urlopen (req_url) # Save path save_path = self. build_path (f_name) with open (save_path, "wb") as handler: while True: chunk = handle. read (1024) if not chunk: break handler. write (chunk) msg = u "downloaded from % s" % req_url sys. stdout. write (msg) sys. stdout. flush () class HttpRequest: def _ init _ (self): self. task = [] self. reg_decode = re. compile ('
 
  
.*? CDATA \[(.*?) \]. *?
 ') Self. reg_encode = re. compile ('
 
  
.*? CDATA \[(.*?) \]. *?
 ') Self. init () self.tar get_url = search_url % urllib2.quote (self. encode2utf8 (SINGER) def encode2utf8 (self, source): if source and isinstance (source, (str, unicode): source = source. encode ("utf8") return source def mkDir (self, dir_name): if not OS. path. exists (dir_name): OS. mkdir (dir_name) def init (self): self. mkDir (SAVE_FOLDER) subPath = OS. path. join (SAVE_FOLDER, SINGER) self. mkDir (subPath) def http_request (self): global song_url ''' initiate the request ''' response=urllib2.urlopen(self.tar get_url) # obtain the header information content = response. read () response. close () # Use BeautifulSoup html = BeautifulSoup (content, from_encoding = "utf8") # extract the HTML Tag span_tag = html. find_all ('P', {"monkey": "song-list"}) [0]. find_all ('span ', class _ = 'song-title') # traverse List for a_tag in span_tag: song_name = unicode (a_tag.find_all ("a") [0]. get_text () song_url = song_url + urllib2.quote (self. encode2utf8 (song_name) song_url = song_url + '$' + urllib2.quote (self. encode2utf8 (SINGER) + '$ & url = & listenreelect = 0 &. r = 0.1696378872729838 'xmlfile = urllib2.urlopen (song_url) xml_content = xmlfile. read () xmlfile. close () url1 = re. findall (self. reg_encode, xml_content) url2 = re. findall (self. reg_decode, xml_content) if not url1 or not url2: continue url = url1 [0] [: url1 [0]. rindex ('/') + 1] + url2 [0] self. task. append ({song_name: url}) return self. taskdef start_download (urls): # create a Queue quene = Queue () # obtain the size of the list = len (urls) # enable the thread for _ in xrange (size ): t = Downloader (quene) t. setDaemon (True) t. start () # enter the queue for url in urls: quene. put (url) quene. join () if _ name __= = '_ main _': http = HttpRequest () urls = http. http_request () start_download (urls)

I hope this article will help you with Python programming.