Windows uses Python to invoke wget batch processing to download data

Wget is commonly used under the Linux/unix download HTTP/FTP data, the use is very convenient, in fact Wget is currently compiled, can also be used under Windows. Recently need to download a large number of remote sensing data, using Python to write a batch download program, using Urllib urlretrieve to download, data download can also, but the interface is not good-looking interactive. The data is improved based on your own Linux download.

1. wget installation under Windows:

Download the wget for Windows tool from the website (http://gnuwin32.sourceforge.net/packages/ wget.htm), after decompression wget.exe copy to System32, and then type wget from CMD, the installation success prompt is as follows:

2. python Batch script download

The large amount of remote sensing Aqua data downloaded is from the NASA OBPG website (http://oceandata.sci.gsfc.nasa.gov), which is saved to TXT based on the list of data after order, then the download URL and string are built, and then downloaded using Suprocess.call.

The code is as follows:

#http://oceandata.sci.gsfc.nasa.gov/cgi/getfile/A2005067190000.L0_LAC.bz2#!/usr/bin/env python#Coding:utf-8ImportOSImportOs.pathImportsubprocessdefRETRIEVING_OBPG (filelist,outpath):" "Download Data" "F= Open (FileList,'R') Log= Open (Os.path.splitext (filelist) [0]+'_log.txt','W') Os.chdir (Outpath)Print(Os.curdir) forIinchF:Try: Each_item=Str (I.strip ()) cmd='wget http://oceandata.sci.gsfc.nasa.gov/cgi/getfile/'+Each_itemPrint(CMD)if  notOs.path.exists (outpath+each_item): status=subprocess.call (cmd)ifStatus! =0:log.write ('\nfailed:'+Each_item)ContinueLog.write ('\nsuccess:'+Each_item) Log.flush ()except: Log.write ('\nfailed:'+Each_item)Continuef.close () log.close ( )if __name__=='__main__':                ImportGlob Outpath='f:\\ satellite Data \\MODIS\\'         forFileListinchGlob.glob (R'f:\ satellite Data \modis\filelists\*m2s.txt'): RETRIEVING_OBPG (Filelist,outpath)Print('END')

Windows uses Python to invoke wget batch processing to download data