HTTP multi-Threaded download

Rationale, each thread starts downloading from a different location in the file, and finally merges the complete data.

Benefits of using multi-threaded downloads
Fast download speed. Why is it. Well understood, I used to be a thread to download on the server. That is, the corresponding on the server, there is a my download thread exists.
I'm sure it's not just me. In the download, the server must have multiple download threads at the same time, downloading server resources. Concurrent execution is not possible for CPUs.
The CPU will fairly divide the time slices for these threads, take turns, a-line Cheng milliseconds, B-line Cheng milliseconds ...
Assuming that this technique is used, it means that my download application can be downloaded simultaneously (theoretically) using any number of threads on the server side.
Assuming that the number of threads is 50, the application will be more than 50 times times more likely to be cared for by the server CPU.
But it will always be limited by the speed of the local network.

The length of data that each thread is responsible for downloading can be calculated by dividing the total length of downloaded data by the total number of threads participating in the download. But consider the case that cannot be divisible.
Assuming that there are 5 threads involved in the download, the formula should be:
int block = Total data length% number of threads = = 0? 10/3:10/3+1; (not divisible, then add one)

and database paging query types. Each thread needs to know where to start downloading from the data and where to download it.
First, with an ID for each thread, the ID starts from zero and is 0 1 2 3 ...
Start location: The thread ID is multiplied by the length of data each thread is responsible for downloading.
End Position: The previous position of the next thread's start position.
Such as:
int startposition = thread id * Length of data downloaded per thread
int endposition = (thread ID + 1) * The length of data downloaded per thread-1;

The Range header of the HTTP protocol can specify where to start the download from where the file is downloaded and where to end the download. Unit is 1byte
range:bytes=2097152-4194304 means to download from the 2M location of the file and end at 4M
If Range specifies the number of bytes to be read to 5104389 of the file, the downloaded file itself has only 4,104,389 lengths. Then the download operation will automatically stop at 4104389.
Therefore, no extra invalid data is downloaded.

Another challenge is how to write data to a local file sequentially. Because threads are executed synchronously, they write data to the local destination file at the same time.
The data that threads write between threads does not follow the order in which the data is downloaded. The final local download file will be distorted if you follow the normal OutputStream write method.
So we're going to use the following class:
Java.io.RandomAccessFile
Because this class implements both DataOutput and Datainput methods. Enable them to have both write and read capabilities.
This class seems to have something like a file pointer that can be arbitrarily executed at any point in the file to begin reading and writing.
Therefore, instances of this class support read and write to random access files.

For example:

Java Code Collection Code
File File = new file ("1.txt");
Randomaccessfile accessfile = new Randomaccessfile (file, "RWD");
Accessfile.setlength (1024);

Although, after executing this code, we have not yet made the target file "1.txt" Write any data. But if you look at the size at this point, it's already 1kb. 
This is the size that we set ourselves. This operation is similar to storing a large byte array for this file. This array will support this file to the specified size. 
Waiting to be filled. 
In this case, the advantage is that we can randomly access a portion of the file system through the index. 
For example, maybe this file size is 500 so, my business needs may need to start writing data from 300 bits for the first time and write to 350. 
The second time, I started writing data from 50 and wrote it until 100. 
In short, I am not a "one-time" "in order" to finish this file. 

 Then, Randomaccessfile can support this operation. API void SetLength (Long newlength) sets the length of this file.  (Set the expected size of the file) void Seek (long pos) sets the File-pointer offset, measured from the beginning of this file, at which 
      The next read or write occurs. 
  Assuming that this method passes 1028 for this parameter, it indicates that it will be written from the 1028-bit position of the file. void Write (byte[] b, int off, int len) writes Len bytes from the specified byte array starting at offset off to thi 
  S file. Write (byte[] b) writes B.length bytes from the specified byte array to this file, starting at the current file Poin 
  ter. void writeUTF (String str) writes a string to the file using modified UTF-8 encoding in a machine-independent manner. 

  String ReadLine () Reads the next line of text from this file. 
 Experiment Code:

Java Code Collection Code
public static void Main (string[] args) throws Exception {

File File = new file ("1.txt");

Randomaccessfile accessfile = new Randomaccessfile (file, "RWD");

/* Set file to 3 byte size */

Accessfile.setlength (3);

/* write ' 2 ' to the second position */

Accessfile.seek (1);

Accessfile.write ("2". GetBytes ());

/* write ' 1 ' to the first position */

Accessfile.seek (0); Accessfile.write ("1". GetBytes ());

/* write ' 3 ' to the third position */

Accessfile.seek (2);

Accessfile.write ("3". GetBytes ()); Accessfile.close ();

Expected content of the file is: 123

}

The above experiments were successful, although we wrote the strings in the order of "2", "1", "3&quot, but because the file offset was set in the relationship, The final data saved by the file is: 123 Another question, after writing the three data, the file size is already 3 bytes in size. 

is already full of data written, so what happens when we continue to put data in it. 
/* Write the data to the fourth byte in the out-of-size position */Accessfile.seek (3); Accessfile.write ("400". 

GetBytes ()); 
The above code, regardless of the file pointer offset specified by the Seek method, and the data deposited, have exceeded the initial 3-byte size set for the file. According to my guess, at least "Accessfile.seek (3)" position will be thrown " arrayindexoutofboundsexception" 
An exception that indicates that the subscript is out of bounds. Instead, perform "accessfile.write alone ("400". GetBytes ()) " Should be able to succeed. 
Because this requirement is reasonable, there should be a mechanism to implement it. The result of the experiment is that both codes are successful. 

It seems that the file implies a large array of bytes that can automatically brace up. 
However, it is important to note that you must ensure that each location of the file size you set has legitimate data, at least not null. 
    For example:/* write ' 3 ' */Accessfile.seek (2) to the third position; Accessfile.write ("3". 

    GetBytes ()); 
    Accessfile.seek (5); Accessfile.write ("400". 
GetBytes ()); Then the combination of the previous code, the final result is: 123 Port 400 in the blank of the two locations appear garbled. 

That's for granted. 
Also, suppose we specify 100 lengths for a file: accessfile.setlength (100); And, in fact, we only set values for the first five locations. So take it for granted that the data saved by the file will eventually be prefixedOn 95 garbled characters. 
 

The preparatory work should be very adequate. Next the code.

Java Code Collection Code
Import Java.io.File;
Import java.io.IOException;
Import Java.io.InputStream;
Import Java.io.RandomAccessFile;
Import java.net.HttpURLConnection;
Import Java.net.URL;
/**
* Multi-threaded file download
*/
public class Multhreaddownload {
/* URL to download */
Private URL DownloadURL;
/* Local file for saving */
Private File LocalFile;
/* Data length not downloaded by thread */
private int block;
public static void Main (string[] args) {
/* can be any legitimate download address on the network */
String Downpath = "Http://192.168.1.102:8080/myvideoweb/down.avi";
Multhreaddownload threaddownload = new Multhreaddownload ();
/* Open 10 threads download Download */
try {
Threaddownload.download (Downpath, 10);
} catch (Exception e) {
E.printstacktrace ();
}
}
/**
* Multi-threaded file download
*
* @param path Download address
* @param number of threadcount threads
*/
public void Download (String path, int threadcount) throws Exception {
DownloadURL = new URL (path);
HttpURLConnection conn = (httpurlconnection) downloadurl
. OpenConnection ();
/* Set GET Request method */
Conn.setrequestmethod ("GET");
/* Set response time timeout to 5 seconds */
Conn.setconnecttimeout (5 * 1000);
/* Get local file name */
String filename = parsefilename (path);
/* Get the total size of the downloaded file */
int datalen = Conn.getcontentlength ();
if (Datalen < 0) {
System.out.println ("Failed to get Data");
Return
}
/* Create a local destination file and set its size to the total size of the ready-to-download file */
LocalFile = new File (filename);
Randomaccessfile accessfile = new Randomaccessfile (LocalFile, "RWD");
/* At this time, in fact, the local directory, has created a size for the total size of the download file * * *
Accessfile.setlength (Datalen);
Accessfile.close ();
/* Calculate the size of data to download per thread */
block = datalen% ThreadCount = = 0? Datalen/threadcount:datalen/threadcount + 1;
/* Start thread download file */
for (int i = 0; i < ThreadCount; i++) {
New Downloadthread (i). Start ();
}
}
/**
* Parse File
*/
private string Parsefilename (string path) {
Return path.substring (Path.lastindexof ("/") + 1);
}
/**
* Inner class: File Download Thread class
*/
Private Final class Downloadthread extends Thread {
/* Thread ID */
private int ThreadID;
/* Where to start the download */
private int startposition;
/* End the download location */
private int endposition;
/**
* Create a new download thread
* @param threadid Thread ID
*/
Public downloadthread (int threadid) {
This.threadid = ThreadID;
StartPosition = ThreadID * block;
Endposition = (ThreadID + 1) * BLOCK-1;
}
@Override
public void Run () {
SYSTEM.OUT.PRINTLN ("Thread '" + ThreadID + "' Start Download:");

        Randomaccessfile accessfile = null; The try {/* setting starts writing data from where the local file is written, and "RWD" means that the file has read/write delete permissions. */accessfile = new Randomaccessfile (local    
            File, "RWD");    
            Accessfile.seek (startposition);    
            HttpURLConnection conn = (httpurlconnection) downloadurl.openconnection ();    
            Conn.setrequestmethod ("GET");    
            Conn.setreadtimeout (5 * 1000);    
                    /* To set the Range property for HTTP, you can specify the scope of the server's return data */conn.setrequestproperty ("range", "bytes=" + startposition + "-"    
            + endposition);    

            /* Write data to local file */WriteTo (ACCESSFILE, conn);    
        SYSTEM.OUT.PRINTLN ("thread" + ThreadID + "finish download");    
        } catch (IOException e) {e.printstacktrace ();     
                } finally {try {if (accessfile! = null) {accessfile.close (); }} catch (IOexception ex) {ex.printstacktrace (); }}}/** * Write download data to local file */private void WriteTo (Randomaccessfile ACCESSF    
        Ile, HttpURLConnection conn) {InputStream is = null;    
            try {is = Conn.getinputstream ();    
            byte[] buffer = new byte[1024];    
            int len =-1;    
            while (len = is.read (buffer))! =-1) {accessfile.write (buffer, 0, Len);    
        }} catch (IOException e) {e.printstacktrace ();    
                } finally {try {if (is = = null) {is.close ();    
            }} catch (Exception ex) {ex.printstacktrace ();    
 }    
        }    
    }    
}

}