Multiple file extraction has: Only get the URL, or directly download, below is how to download the data, and show the progress. This section mainly introduces the Urlretrieve () function provided by the Urllib module. The Urlretrieve () method downloads remote data directly to a local, function model:
- Urlretrieve (URL, Filename-none, Reporthook=none, Data=none)
- Parameter filename Specifies the local path of the store
- The parameter reporthook is a callback function. This callback function is triggered when the connected server and the corresponding data block are transferred, and we can use this callback function to display the current progress.
- The parameter data refers to the post-to-server, which returns a two-element (filename,headers) tuple, filename means the local path, and the header indicates the server response header
The following example code
#coding:utf-8import urllibfrom lxml import etreeimport requestsdef Schedule(blocknum,blocksize,totalsize): ‘‘‘‘‘ blocknum:已经下载的数据块 blocksize:数据块的大小 totalsize:远程文件的大小 ‘‘‘ per = 100.0 * blocknum * blocksize / totalsize if per > 100 : per = 100 print ‘当前下载进度:%d‘%peruser_agent = ‘Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)‘headers={‘User-Agent‘:user_agent}r = requests.get(‘http://www.ivsky.com/tupian/ziranfengguang/‘,headers=headers)#使用lxml解析网页html = etree.HTML(r.text)img_urls = html.xpath(‘.//img/@src‘)#先找到所有的imgi=0for img_url in img_urls: urllib.urlretrieve(img_url,‘img‘+str(i)+‘.jpg‘,Schedule) i+=1
The schedule function contains 3 parameters: Blocknum: The data block that has been downloaded, blocksize: The size of the data block, TotalSize: The size of the remote file
Python Multimedia file extraction