Download Software with Java Design

This article describes how to use Java to write and download software and implement multi-process download.

"Network ant", falshget, and many other multi-thread download software are essential tools for netizens. These tools can be used to quickly download large files from the server, these tools divide files on the server into several sections, each of which is downloaded separately and simultaneously. First, you must have a better understanding of the HTTP protocol. Second, you can use multi-threaded programming methods to implement the software.

Introduction to HTTP

HTTP is a hypertexttransferprotocol used at the network application layer. It has been widely used in WWW Global Information Services since 1990, for more information about the HTTP protocol, see rfc2518, rfc2616, and other documents on the Internet.

The old standard of HTTP protocol is HTTP/1.0. Currently, the most common standard is HTTP/1.1. HTTP/1.1 is an upgrade based on HTTP/1.0. It adds some features and is fully compatible with HTTP/1.0. HTTP/1.0 does not support resumable upload of files. If the server uses HTTP/1.0, any multi-thread download program of "Network ant" can only be downloaded in a single thread; fortunately, most of the current Web servers use HTTP/1.1. Therefore, we will introduce it based on HTTP/1.1.

Important HTTP commands

When an HTTP-based browser browses or downloads a webpage, the working principle is similar to the Client/Server mode: the browser sends an HTTP request line to the Web server. After receiving a valid request, the Web Server, return one status line or multiple response headers, one blank line, and related documents. Based on this principle, the download program must send requests to the server and obtain the server response status.

1. Send the GET Request command to the server

An HTTP request consists of one request line, an optional Number of request headers, a blank row, and some additional data in the Post case. The request line format is:

Request Method urihttp/version number

The GET command is a common document Request Method in the browser.

Geturihttp/1.1

Send the request line (row number 3) to the Web server. The Java code is as follows:

....

Clientsocket = newsocket (host, Port); // open the socket of the file server to be downloaded

Outstream = newprintstream (clientsocket. getoutputstream ());

....

Outstream. println ("get" + URI + "HTTP/1.1 ");

Outstream. println ("Host:" + host );

Outstream. println ("accept :*/*");

Outstream. println ("Referer :");

Outstream. println ();

....

Note: Row 4th provides the host name and port number in the URL. Line 5th indicates that the client receives all MIME types. Line 7th sends a blank line, indicating that the request line ends.

2. Get the server response status

After an HTTP request line is sent, the program can read the server's response status. The HTTP response status line includes the HTTP Status Code and some HTTP response headers.

1) HTTP status code

The HTTP status code format is a digital representation of HTTP/version information. An example of a status code is as follows:

HTTP/1.0200ok // indicates that the server supports the HTTP/1.0 Protocol and is successful.

HTTP/1.20. OK // indicates that the server supports the HTTP/1.1 protocol and is successful.

HTTP/1.0404 notfound // indicates that the server supports the HTTP/1.0 protocol.

If you read a string like "HTTP/1.20. OK" in the middle of the program, it indicates that the file to be downloaded exists and the server supports resumable data transfer. You can use multiple threads to download the file. If you read a string such as "HTTP/1.0200ok", it indicates that the file to be downloaded exists, but the server does not support resumable data transfer. You can only download the file in a single thread.

.....

While (line = instream. Readline ())! = NULL) // read the server response status to line

........

If (line. substring (1.1). Equals ("HTTP/1.") // determine whether HTTP/is supported

{If (line. charat (7) = '0 ')

{

System. Out. println ("serverusehttp/1.0 ");

Threadcount = 1;

}

If (! (Line. substring (200). Equals ("") // checks whether the request is successful.

{System. Out. println ("error:" + line );

Returnfalse;

}

}

2) read important response headers and obtain the file length of the document to be downloaded.

If the HTTP status code table shows that the access is successful, the server will send back some header lines. We are most concerned with the Content-Length line. For example, if the server returns "Content-Length: 1000 ", it indicates that the length of the request file is 1000 bytes. Therefore, the length of the file can be obtained by reading this line of information:

....

If (line. substring (0, 15). Equals ("Content-Length :"))

{Filelength = long. parselong (line. substring (15). Trim ());

System. Out. println ("filelength:" + filelength );

}

......

Send a resumable upload request to the server

As mentioned above, if the server supports HTTP/1.1, send a GET request to the server again:

.....

Outstream. println ("get" + URI + "HTTP/1.1 ");

Outstream. println ("Host:" + host );

Outstream. println ("accept :*/*");

Outstream. println ("range: bytes =" + (fileblocklength) * thisthreadid + "-");

Outstream. println ();

.....

Line 4th is the key, and "range: bytes =" is the new content in HTTP/1.1. Each time an HTTP/1.0 file is transferred, it starts from the file header, that is, it starts at 0 bytes. "range: bytes = XXXX "indicates that the server is required to transmit the file from the XXXX byte. This is what we call resumable data transfer!

Split files and multi-thread download

Using multi-thread programming technology, start multiple threads at the same time, calculate the file split location based on the number of threads, send several different download breakpoints to the server, simultaneously accept data and write files, you can download multiple threads.

.....

RAF = newrandomaccessfile (file, "RW"); // open the file in Random Access Mode

.....

Synchronized (RAF) // write the data obtained by each thread to the file in synchronous Mode

{Raf. Seek (thisthreadid * (filelength/threadcount) + K * buflength );

Raf. Write (readbytes );

......

}