Windows uses Python to invoke wget batch processing to download data

Source: Internet
Author: User
Tags glob

Wget is commonly used under the Linux/unix download HTTP/FTP data, the use is very convenient, in fact Wget is currently compiled, can also be used under Windows. Recently need to download a large number of remote sensing data, using Python to write a batch download program, using Urllib urlretrieve to download, data download can also, but the interface is not good-looking interactive. The data is improved based on your own Linux download.

1. wget installation under Windows:

Download the wget for Windows tool from the website (http://gnuwin32.sourceforge.net/packages/ wget.htm), after decompression wget.exe copy to System32, and then type wget from CMD, the installation success prompt is as follows:

2. python Batch script download

The large amount of remote sensing Aqua data downloaded is from the NASA OBPG website (http://oceandata.sci.gsfc.nasa.gov), which is saved to TXT based on the list of data after order, then the download URL and string are built, and then downloaded using Suprocess.call.

The code is as follows:

#http://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A2005067190000.L0_LAC.bz2#!/usr/bin/env python#Coding:utf-8ImportOSImportOs.pathImportsubprocessdefRETRIEVING_OBPG (filelist,outpath):" "Download Data" "F= Open (FileList,'R') Log= Open (Os.path.splitext (filelist) [0]+'_log.txt','W') Os.chdir (Outpath)Print(Os.curdir) forIinchF:Try: Each_item=Str (I.strip ()) cmd='wget http://oceandata.sci.gsfc.nasa.gov/cgi/getfile/'+Each_itemPrint(CMD)if  notOs.path.exists (outpath+each_item): status=subprocess.call (cmd)ifStatus! =0:log.write ('\nfailed:'+Each_item)ContinueLog.write ('\nsuccess:'+Each_item) Log.flush ()except: Log.write ('\nfailed:'+Each_item)Continuef.close () log.close ( )if __name__=='__main__':                ImportGlob Outpath='f:\\ satellite Data \\MODIS\\'         forFileListinchGlob.glob (R'f:\ satellite Data \modis\filelists\*m2s.txt'): RETRIEVING_OBPG (Filelist,outpath)Print('END')

Windows uses Python to invoke wget batch processing to download data

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.