Implement HTTP File queue download using Java

　　Preface

Many users may encounter this situation: a good resource is found on the website, but this resource is divided into many files for storage. If you want to save it locally, it is a disaster to save resources by clicking save. If resources are divided into hundreds or even tens of thousands.

When many resources on the Internet are divided into multiple files for storage, their file names have certain rules. For this reason, we can use programs to completely download this resource.

　　1. Basic Knowledge

On the Internet, if we want to download a Resource on the website, we will get a URL (Uniform Resource Locator), which is a description of server Resource location. The download process is always as follows:

• Step 1: The client initiates a connection request for a URL
• Step 2: The server parses the URL and returns an input stream of the specified resource to the customer.
• Step 3: the client receives the input stream and saves the content in the stream to a file

　　 2. Establish a network connection

Java provides APIs for URL access and a large number of stream operations. We can easily access resources on the network, the following code snippet completes access to the resources of a Website:

...... DestUrl = "http://www.ebook.com/java/network programming 001.zip "; Url = new URL (destUrl); httpUrl = (HttpURLConnection) url. openConnection (); // Connect to the specified network resource HttpUrl. connect (); // Obtain the network input stream Bis = new BufferedInputStream (httpUrl. getInputStream ()); ......

　　 3. proxy access

The method for accessing the Internet through a proxy server in Java is already a secret. Here we will not describe more here. The accessed Java code is as follows:

// Sets the Proxy Server System. getProperties (). put ("proxySet", "true "); System. getProperties (). put ("proxyHost", "10.154.134.110 "); System. getProperties (). put ("proxyPort", "8080 ");

　　 4. Save Network Resources

In the previous section, we have obtained the input stream of the specified network resource. Next, we need to read the content of the input stream and save it to the file. Sample Code:

...... Fos = new FileOutputStream (fileName ); If (this. DEBUG) System. out. println ("getting the content of link [" + destUrl + "]... \ n save it as file [" + fileName + "]"); // Save the file While (size = bis. read (buf ))! =-1) Fos. write (buf, 0, size );......

The sample code above saves the content of the network resource to the specified local file.

　　5. Code List

Import java. io. *; import java.net. *; import java. util. *;/*** <p> Title: personal-developed API </p> * <p> Description: store the specified HTTP network resources as files locally </p> * <p> Copyright: Copyright (c) 2004 </p> * <p> Company: newSky </p> * @ author MagicLiao * @ version 1.0 */public class HttpGet {public final static boolean DEBUG = true; // private static int BUFFER_SIZE = 8096 for debugging; // buffer size private Vector vDownLoad = new Vector (); // URL list private Vector vFileList = new Vector (); // list of saved file names after download/*** constructor */public HttpGet () {}/*** clear download list */public void resetList () {vDownLoad. clear (); vFileList. clear ();}/*** added the download list item ** @ param url String * @ param filename String */public void addItem (String url, String filename) {vDownLoad. add (url); vFileList. add (filename);}/*** download resources by list */public void downLoadByList () {String url = null; String Filename = null; // Save the resource for (int I = 0; I <vDownLoad. size (); I ++) {url = (String) vDownLoad. get (I); filename = (String) vFileList. get (I); try {saveToFile (url, filename);} catch (IOException err) {if (DEBUG) {System. out. println ("resource [" + url + "] Download failed !!! ") ;}} If (DEBUG) {System. out. println (" Download complete !!! ") ;}}/*** Save the HTTP resource as a file ** @ param destUrl String * @ param fileName String * @ throws Exception */public void saveToFile (String destUrl, string fileName) throws IOException {FileOutputStream fos = null; BufferedInputStream bis = null; HttpURLConnection httpUrl = null; URL url = null; byte [] buf = new byte [BUFFER_SIZE]; int size = 0; // link creation url = new URL (destUrl); httpUrl = (HttpURLConnection) u Rl. openConnection (); // connect to the specified resource httpUrl. connect (); // obtain the network input stream bis = new BufferedInputStream (httpUrl. getInputStream (); // create the file fos = new FileOutputStream (fileName); if (this. DEBUG) System. out. println ("retrieving the content of link [" + destUrl +... \ n save it as the file ["+ fileName +"] "); // save the file while (size = bis. read (buf ))! =-1) fos. write (buf, 0, size); fos. close (); bis. close (); httpUrl. disconnect ();}/*** set proxy Server ** @ param proxy String * @ param proxyPort String */public void setProxyServer (String proxy, String proxyPort) {// set the proxy server System. getProperties (). put ("proxySet", "true"); System. getProperties (). put ("proxyHost", proxy); System. getProperties (). put ("proxyPort", proxyPort);}/*** set the authentication username and password ** @ param uid String * @ param pwd String */public void setAuthenticator (String uid, string pwd) {Authenticator. setDefault (new MyAuthenticator (uid, pwd);}/*** main method (used for testing) ** @ param argv String [] */public static void main (String argv []) {HttpGet oInstance = new HttpGet (); try {// Add a download list (here you can write your own code to add a download list) oInstance. addItem (" http://www.ebook.com/java/ Network Programming 001.zip ","./network programming 1.zip "); oInstance. addItem (" http://www.ebook.com/java/ Network Programming 002.zip ","./network programming 2.zip "); oInstance. addItem (" http://www.ebook.com/java/ Network Programming 003.zip ","./network programming 3.zip "); oInstance. addItem (" http://www.ebook.com/java/ Network Programming 004.zip ","./network programming 4.zip "); oInstance. addItem (" http://www.ebook.com/java/ Network Programming 005.zip ","./network programming 5.zip "); oInstance. addItem (" http://www.ebook.com/java/ Network Programming 006.zip ","./network programming 6.zip "); oInstance. addItem (" http://www.ebook.com/java/ Network Programming 007.zip ",". /network programming 7.zip "); // starts to download oInstance. downLoadByList ();} catch (Exception err) {System. out. println (err. getMessage ());}}}

Download source code