Step by step analysis of Baidu Music playing address, using Python crawler bulk download

Source: Internet
Author: User

Baidu Music does not need to login also can download? Is it very exciting to hear the news?

Next we open Baidu Music, casually open a song, switch to the Baidu play page:

I use the Firfox browser, open Firebug first clear all the requests,


Now we re-refresh the next page, see this. mp3 address is the address of Baidu Music, we can directly copy to the Thunder download, but this approach is too elementary, if there are many songs, each copy, it is very troublesome ah. , and then we continue to analyze.


This link has a feature, that is music/1658513 what is this? You guessed right, this is the ID of each song, and then look at the following parameters Xcode this is a GUID, after comparison, each is different, this from where AH. We continue to analyze other requests:

Haha, or we find it, this songlink is the address of the music, but this request is how to come, we continue to look up, we see the POST request parameters, there is a songids this is the ID of each song,

At this point, everything went well and the address was copied, and then our Python appeared.


The environment used here is Python3.4, third-party library Beautifulsoup,requests, how to install, online has a lot, next on the code

Create a new XML file named Music.xml in the following format

<?XML version= "1.0"encoding= "Utf-8"?><Root> <URL>http://music.baidu.com/tag/Pure </URL> <pageSize>40</pageSize> <Saveplay>d:\\ Pure \\</Saveplay></Root>
URL is the category of Baidu Music address PageSize is the number of pages to download, Saveplay is the path to save

Next we build a PY file main code section:


Simulate browser requests to prevent blocking

headers={' User-agent ':' mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) gecko/20100101 firefox/44.0 ',' Referer ':' http://play.xml.baidu.com/',' Accept ':' text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 ',' accept-encoding ':' gzip, deflate, BR ',' Accept-language ':' zh-cn,zh;q=0.8,en-us;q=0.5,en;q=0.3 ',' Cache-control ':' max-age=0 ',' Connection ':' keep-alive '        } Methods for reading XML files

Gettagtext (TAG):        "Dom=xml.dom.minidom.parse (" Play.xml ")    node = Dom.getelementsbytagname (tag) [0]    node.childnodes:          (node. Text_node, NODE. Cdata_section_node):            rc=node.data    RC

Parsing HTML methods

defdownhtml (URL):Try: Saveplay=gettagtext ("Saveplay")Print("Ready to start Parsing page:"+url+"Please wait ...") Html=requests.get (URL,Headers=headers,Timeout= -) html.encoding="Utf-8"           Soup=beautifulsoup (Html.text,"Html.parser") Div_html=soup.find ("Div",Class_="Search-song-list song-list song-list-hook") Span_html=re.findall (' <span class= ' Music-icon-hook ' data-musicicon=\ '(.*?)\ '> ',Str(div_html)) forVinchSpan_html:data=json.loads (v) play=data["id"],data["SongTitle"] Queue.append (play) whileQueue:time.sleep (5) Music_tuple=queue.popleft () Playurl=music.replace ("$0$", music_tuple[0])Print(music_tuple[1]+". mp3 Enter the download channel and start waiting in line ...\ n") Resultjson=requests.get (Playurl,Headers=headers,Timeout= -) Data=resultjson.json ()if notdata[' Data ']:PassElse: v=data["Data"]["SongList"][0];Print("Downloading,"+v["Songname"]+". mp3 ...\ n") Time.sleep (2) Request.urlretrieve (v["Songlink"],saveplay+v["Songname"]+". mp3")Print(v["Songname"]+". mp3 download complete, download path:"+saveplay+v["Songname"]+". mp3")except:Pass

if__name__=="__main__": start = Time.time () pagesize=int(Gettagtext ("PageSize")) pageindex= -    Url=gettagtext ("url") playurls.append (URL)ifPagesize>1: forVinchRange(pageSize):ifV>0: purl=url+"? start="+Str(PageIndex) +"&size=25&third_type=0"                Playurls.append (purl) pageindex+= -     withPool (4) asP:p.map (Downhtml,playurls)Print("This download is shared:"+time.time ()-start)
Finally, let's take a look at the effect of our operation,

Step by step analysis of Baidu Music playing address, using Python crawler bulk download

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.