Java Network Programming from entry to mastery (24): implements the HTTP breakpoint download Tool

Source code download: Click here to download

In the previous article, we discussed three fields related to resumable upload in the HTTP message header. One is the request message field Range, and the other two are the Response Message Field Accept-Ranges and Content-Range. Here, Accept-Ranges is used to determine whether the Web server supports resumable data transfer. Here, we want to demonstrate how to implement the resumable data transfer function. Assume that the Web server supports this function. Therefore, we only use Range and Content-Range to develop a resumable data transfer tool.

L what kind of resumable data transfer tool should be implemented?

This resumable download tool is a single-threaded download tool. It uses parameters to input a text file. The file format is as follows:

The http://www.ishare.cc/d/1174254-2/106.jpg d: ok1.jpg 8192
The http://www.ishare.cc/d/1174292-2/156.jpg d: ok2.jpg 12345
The http://www.ishare.cc/d/1174277-2/147.jpg d: ok3.jpg 3456

Each line of the text file is a download item, which is divided into three parts:

The URL of the Web resource to download.
The local file name to save.
The size of the downloaded buffer (in bytes ).
Use at least one space to separate the three parts. This download tool downloads these files one by one. After all these files are downloaded, the program exits.

L Working Principle of resumable Data Transfer

As the name implies, "resumable upload" means that after a part of a file is downloaded, the current network connection is interrupted due to reasons of the server or client. After the network connection is interrupted, you can establish a network connection to continue downloading the file.

To achieve resumable data transfer in a single thread, you must save two data records when the customer disconnects.

1. Number of downloaded bytes.

2. the URL of the downloaded file.

Once the network connection is re-established, you can use the two data to continue downloading files that have not been downloaded. In this download tool, the first type of data is the number of bytes that the file has downloaded, and the second type of data is saved in the downloaded file.

The number of downloaded bytes is detected when the download continues. If 3000 bytes have been downloaded, the Range field of the HTTP request message header is set to the following format:

Value Range: bytes = 3000-
The Content-Range field of the HTTP Response Message Header is set to the following format:

Content-Range: bytes 3000-10000/10001
L resumable download Tool

A resumable download program can be implemented in the following steps:

1. Enter the URL of the file to be downloaded and the local file name to be saved, and connect to the URL through the Socket class.

Server.

2. generate an HTTP request message on the client based on the URL of the downloaded file and the local file. When generating a request

Message time is in two situations:

(1) When this file is downloaded for the first time, the request message is generated normally, that is, the file does not contain Range.

Field request message.

(2) I have downloaded this file before. This is the next download. This enters the resumable upload program. In this case, the generated HTTP request message must contain the Range field. Because it is a single-threaded download, the size of a part of the downloaded files is the value of Range. Suppose the size of the current file is 1234 bytes, Set Range to the following values:

Value Range: bytes = 1234-
3. Send an HTTP request message to the server.

4. Receive the HTTP Response Message returned by the server.

5. process HTTP response messages. In this program, you need to obtain the total number of bytes of the downloaded file from the response message. For example

If this is the first download, that is, when the response message does not contain the Content-Range field, the total number of bytes is the value of the Content-Length field. If the response message does not contain the Content-Length field, the total number of bytes cannot be determined. This is why the download tool does not have the file size and download progress when downloading some files. If the response message contains the Content-Range field, the total number of bytes is k in Content-Range: bytes m-n/k. For example, the value of Content-Range is:

Content-Range: bytes 1000-5000/5001
The total number of bytes is 5001. Because the Range value type used in this program is to get all bytes starting from a certain byte, therefore, the Content-Range in the current response message always returns the number of bytes not downloaded. In the preceding example, the number of undownloaded bytes is 5000-1000 + 1 = 4001.

6. Start to download the file and calculate the download progress (in percent form ). If the file is not downloaded when the network connection is disconnected, re-execute the first step. If the file has been downloaded, exit the program.

The above six steps show that there are four main functions to be implemented:

1. generate an HTTP request message and send it to the server. This function is completed by the generateHttpRequest method.

2. Analyze the HTTP response message header. This function is completed by the analyzeHttpHeader method.

3. Get the actual size of the downloaded file. This function is completed by the getFileSize method.

4. download the file. This function is completed by the download method.

The above four methods are included in HttpDownload. java, the core class of the resumable data transfer tool. Before the implementation of the HttpDownload class is provided, an interface DownloadEvent interface is provided. From the interface name, it can be seen that it is used to handle events during the download process. The following is the implementation code of this interface:

Package download;

Public interface DownloadEvent
{
Void percent (long n); // download progress
Void state (String s); // status switching during connection
Void viewHttpHeaders (String s); // enumerate each Response Message Field
}

The code above shows that the DownloadEvent interface has three event methods. This interface will be implemented in future main functions to output relevant information to the console. The main framework code of the HttpDownload class is given below:

001 package download;
002
003 import java.net .*;
004 import java. io .*;
005 import java. util .*;
006
007 public class HttpDownload
008 {
009 private HashMap httpHeaders = new HashMap ();
010 private String stateCode;
011
012 // generateHttpRequest Method
013
014/* ananlyzeHttpHeader Method
015 *
016 * addHeaderToMap Method
017 *
018 * analyzeFirstLine Method
019 */
020
021 // getFileSize Method
022
023 // download Method
024
025/* getHeader Method
026 *
027 * getIntHeader Method
028 */
029}

The above code is only the Framework Code of the HttpDownload class, and the method is not directly implemented. We can see that rows 012nd, 014, 021, and 023 are the four main methods described above. The addHeaderToMap and analyzeFirstLine methods on the 016 and 018 rows are used in the analyzeHttpHeader method. The getHeader and getIntHeader methods of the 025 and 027 rows are used in the getFileSize and download methods. The implementation of the above eight methods will be provided later.

 

001 private void generateHttpRequest (OutputStream out, String host,
002 String path, long startPos) throws IOException
003 {
004 OutputStreamWriter writer = new OutputStreamWriter (out );
005 writer. write ("GET" + path + "HTTP/1.1 ");
006 writer. write ("Host:" + host + "");
007 writer. write ("Accept :*/*");
008 writer. write ("User-Agent: My First Http Download ");
If (startPos> 0) // if resumable upload is performed, add the Range field
010 writer. write ("Range: bytes =" + String. valueOf (startPos) + "-");
011 &