Python crawls Baidu Music.

Today under the challenge of Baidu Music Crawl, the first with chrome analysis of the requested link.

The key is the link

Http://play.baidu.com/data/music/songlink

Ask for the JSON that SongID can return to your music, so how do you get songid?

Open the Http://music.baidu.com/tag and find a label to go in. Then view the source page. The following fragments were found.

The data-songitem of the LI element in each song happens to contain the SID we need. OK, the goal is clear, first request the Music tab of Baidu Music, and then get SID. Then request

Http://play.baidu.com/data/music/songlink

And bring us the assembled SID. Finally, we get the information we need based on the return JSON.

The code is as follows (just learning Python, not writing too well):

#!/usr/bin/python#coding=utf-8__author__ =  ' Zhm ' import urllibimport urllib2import  refrom json import * #SONG_TAG_URL  =  ' http://music.baidu.com/tag/%E7%BB%8F%E5%85%B8%E8 %80%81%e6%ad%8c ' song_link_url =  ' Http://play.baidu.com/data/music/songlink ' def getcontent (URL, Pattern):     try:        f=urllib2.urlopen (URL)         result =  f.read ();         content = re.compile (Pattern, re. Dotall)         style = content.search (Result)          if style:             result = style.group (0)              return result        else:            return none     except exception ,e:        print eif   __name__== "__main__":     for i in range (0,1000,25):          #根据给出的百度音乐分类地址解析出songid          Result = getcontent (song_tag_url+ '? start= ' +unicode (i) + ' &size=25&third_type=0 ', ' <ul>.*? </ul> ')         sids = []         sidpattern = re.findall ("&quot;sid&quot;:.*?,&quot;", result)          for sid in sidPattern:             sids.append (re.sub (',&quot; ', ', re.sub (' &quot;sid&quot;: ', ', sid))         # print sids          #将songid构造成post请求参数         formdata = {  "Songids"  :  ",". Join (SIDs)}        data_encoded =  Urllib.urlencode (Formdata)         # print data_encoded         songlist = urllib2.urlopen (SONG_LINK_URL,data_encoded)         songlistjson = songlist.read ()          # print songListJson           #json   Turn dictionary         song_dict = jsondecoder (). Decode (Songlistjson)          #获取songList          song_data_dict =&nBsp;song_dict.get ("Data"). Get ("SongList")         for sond_data  in song_data_dict:            song_name  = sond_data.get (' Songname ')              Song_artistname = sond_data.get (' Artistname ')              song_format = sond_data.get (' format ')              song_link = sond_data.get (' Songlink ')              if song_name is None or                               song_artistName is None or                               song_format is None or                               song_link is None:                 continue             Print song_name+ '--' +song_artistname+ '. +song_format+u '       download link: ' +song_link              #下载方法此处就省略了.

Python crawls Baidu Music.