Use java to implement http multi-thread breakpoint download (2)

I don't think many people will use the download tool. Some time ago it was boring. It took me some time to write a simple http multi-thread download program in java. It was just boring to write, I only implemented a few simple functions and did not write the interface. Today is just a boring day, so I wrote an article. I thought it was good to give a round of applause and don't need to spray it if it was not good, thank you!
The http download tool I implemented is very simple, that is, a multi-thread and a breakpoint recovery. Of course, download is essential. SoSort out the work to be done first.:
1. Connect to the resource server, obtain resource information, and create a file
2. Splitting resources and multi-thread download
3. breakpoint recovery
4. download speed statistics
The first thing to do is to connect resources and obtain Resource Information. Here I use the URLConnection provided by JavaSE to connect resources. The Code is as follows:

Copy codeThe Code is as follows: String urlStr = "http://www.sourcelink.com/download/xxx#; // resource address, which can be written at will
URL url = new URL (urlStr); // create a URL
URLConnection con = url. openConnection (); // establish a connection
ContentLen = con. getContentLength (); // obtain the resource Length
File file = new File (filename); // create a download File based on filename, which will be the final downloaded file.

It's easy. That's right. The first step is done. The second step is to split resources and implement multithreading. In the previous step, we have obtained the resource length contentLen. How can we split the resource based on this? If we want to run ten threads, we will first take contentLen to 10 to get the size of each thread, and then create ten threads respectively, each thread is responsible for writing one of them, this requires the RandomAccessFile class, which provides random access to the file and allows you to specify the write operation to a location in the file. The Code is as follows:Copy codeCode: long subLen = contentLen/threadQut; // obtain the size of each block
// Create ten threads and start the threads
For (int I = 0; I <threadQut; I ++ ){
DLThread thread = new DLThread (this, I + 1, subLen * I, subLen * (I + 1)-1); // create a thread
DlThreads [I] = thread;
QSEngine.pool.exe cute (dlThreads [I]); // submit the thread to the thread pool for management.
}

The DLThread class is used here. Let's take a look at the definition of the constructor of this class:
Public DLThread (DLTask dlTask, int id, long startPos, long endPos)
The first parameter is a DLTask. This class represents a download task, which stores information about the download task, including the download Resource Name and local file name. The second parameter is a thread id. If there are 10 threads, the id ranges from 1 to 10. The third parameter startPos indicates where the thread starts writing data from the file, the last endPos parameter indicates the end of writing.
Let's take a look at how to download a thread after it is started. See the run method:Copy codeThe Code is as follows: public void run (){
System. out. println ("Thread" + id + "start ");
BufferedInputStream bis = null; // create a buff
RandomAccessFile fos = null;
Byte [] buf = new byte [BUFFER_SIZE]; // buffer size
URLConnection con = null;
Try {
Con = url. openConnection (); // create a connection. A connection is created for each thread.
Con. setAllowUserInteraction (true );
If (isNewThread ){
Con. setRequestProperty ("Range", "bytes =" + startPos + "-" + endPos); // you can specify the Range from startPos to endPos to obtain resource data.
Fos = new RandomAccessFile (file, "rw"); // create a RandomAccessFile
Fos. seek (startPos); // starts from startPos
} Else {
Con. setRequestProperty ("Range", "bytes =" + curPos + "-" + endPos );
Fos = new RandomAccessFile (dlTask. getFile (), "rw ");
Fos. seek (curPos );
}
// The following section writes data to the object. curPos is the unknown data currently written. It determines whether it is smaller than endPos,
// If the value of endPos is exceeded, the thread has been executed.
Bis = new BufferedInputStream (con. getInputStream ());
While (curPos <endPos ){
Int len = bis. read (buf, 0, BUFFER_SIZE );
If (len =-1 ){
Break;
}
Fos. write (buf, 0, len );
CurPos = curPos + len;
If (curPos> endPos ){
ReadByte + = len-(curPos-endPos) + 1; // obtain the number of bytes correctly read
} Else {
ReadByte + = len;
}
}
System. out. println ("Thread" + id + "has been downloaded. ");
This. finished = true;
Bis. close ();
Fos. close ();
} Catch (IOException ex ){
Ex. printStackTrace ();
Throw new RuntimeException (ex );
}
}

The code above is to write the file Model Based on startPos and endPos. Each thread has its own independent resource block, from startPos to endPos. The above method is the core of thread download. After multithreading is done, the next step is to implement the breakpoint recovery function. In fact, the breakpoint recovery is nothing more than to record which unknown each thread has completed, here, I use curPos for record. As you can see in the code above, I will record the curPos of each thread, and then when the thread restarts, consider curPos as startPos, while endPost remains unchanged. You have not noticed the following code in the run method:Copy codeThe Code is as follows: if (isNewThread) {// determines whether a breakpoint is set. if it is set to true, it indicates a new download thread instead of breakpoint recovery.
Con. setRequestProperty ("Range", "bytes =" + startPos + "-" + endPos); // you can specify the Range from startPos to endPos to obtain resource data.
Fos = new RandomAccessFile (file, "rw"); // create a RandomAccessFile
Fos. seek (startPos); // starts from startPos
} Else {
Con. setRequestProperty ("Range", "bytes =" + curPos + "-" + endPos); // use curPos to replace startPos. The rest are the same as the new one.
Fos = new RandomAccessFile (dlTask. getFile (), "rw ");
Fos. seek (curPos );
}

The above is the method of breakpoint recovery. It is no different from creating a new thread, but it is different from startPos. Everything is the same, but it is not enough because if the program is closed, how can this information be saved? For example, file name, curPos for each thread, etc. When you use the software to download, I believe that there will be two temporary files in the directory when the software is not downloaded, one of them is used to save the download task information. Without such information, the program does not know how to restore the download progress. How can I implement it here? I am a lazy person, and I don't want to create another file to save the information, and then I want to read the information to create an object. This is too troublesome. So I thought of java to provide a serialization mechanism, my idea is to serialize the entire DLTask object to the hard disk. As mentioned above, the DLTask class is used to save the information of each task, so when I need to restore the data, deserialization of this object makes it easy to implement the breakpoint function. Let's take a look at the information saved by this object:Copy codeThe Code is as follows: public class DLTask extends Thread implements Serializable {
Private static final long serialVersionUID = 126148287461276024L;
Private final static int MAX_DLTHREAD_QUT = 10; // maximum number of download threads
/***//**
* The suffix of the temporary file to be downloaded. After the download is complete, the file will be automatically deleted.
*/
Public final static String FILE_POSTFIX = ". tmp ";
Private URL;
Private File file;
Private String filename;
Private int id;
Private int Level;
Private int threadQut; // Number of download threads, which can be customized by the user
Private int contentLen; // download object Length
Private long completedTot; // The total number of downloads completed.
Private int costTime; // The download time count to record the download time.
Private String curPercent; // download percentage
Private boolean isNewTask; // whether to create a download task, which may be a resumable upload task.
Private DLThread [] dlThreads; // the thread that saves the current task
Transient private DLListener listener; // listener of the current task, which is used to instantly obtain the relevant download information

In the code above, this object implements the Serializable interface, stores all information about the task, and contains each thread object dlThreads, which makes it easy to restore the breakpoint, let me re-write a file to save the information, and then create an object based on the information during recovery, it is my life. Here we create a method for breakpoint recovery:Copy codeThe Code is as follows: private void resumeTask (){
Listener = new DLListener (this );
File = new File (filename );
For (int I = 0; I <threadQut; I ++ ){
DlThreads [I]. setDlTask (this );
QSEngine.pool.exe cute (dlThreads [I]);
}
QSEngine.pool.exe cute (listener );
}

In fact, it reduces the code for first connecting to resources and then Splitting resources, because the information has been saved under the DLTask object.
Seeing the code above, I don't know if you noticed that there is a DLListener object that is actually used to listen to the information of the entire task. Here I am mainly used for two purposes, one is to regularly serialize the DLTask and save the task information for breakpoint recovery. The other is to collect statistics on the download rate and the average duration. Let's take a look at its code. This class is also a separate thread:Copy codeThe Code is as follows: public void run (){
Int I = 0;
BigDecimal completeTot = null; // percentage of completions
Long start = System. currentTimeMillis (); // the current time, used to record the start statistical time
Long end = start;
While (! DlTask. isComplete () {// whether the entire task is completed. if the task is not completed, continue the cycle.
I ++;
String percent = dlTask. getCurPercent (); // get the current completion percentage
CompleteTot = new BigDecimal (dlTask. getCompletedTot (); // obtain the total number of completed bytes.
// Obtain the current time and compare it with the start time. If not, divide the total number of current completions by the time used to obtain an average download speed.
End = System. currentTimeMillis ();
If (end-start! = 0 ){
BigDecimal pos = new BigDecimal (end-start)/1000) * 1024 );
System. out. println ("Speed :"
+ CompleteTot
. Divide (pos, 0, BigDecimal. ROUND_HALF_EVEN)
+ "K/s" + percent + "% completed .");
}
Recoder. record (); // record task information to the hard disk
Try {
Sleep (3000 );
} Catch (InterruptedException ex ){
Ex. printStackTrace ();
Throw new RuntimeException (ex );
}
}
// The following figure shows the information of the entire download task.
Int costTime = + (int) (System. currentTimeMillis ()-start)/1000 );
DlTask. setCostTime (costTime );
String time = QSDownUtils. changeSecToHMS (costTime );
DlTask. getFile (). renameTo (new File (dlTask. getFilename ()));
System. out. println ("Download finished." + time );
}

In this method, the recoder. record () method is called to serialize the task object. Other code is used for statistics. For details, see the annotations. The code of this method is as follows:Copy codeThe Code is as follows: public void record (){
ObjectOutputStream out = null;
Try {
Out = new ObjectOutputStream (new FileOutputStream (dlTask. getFilename () + ". tsk "));
Out. writeObject (dlTask );
Out. close ();
} Catch (IOException ex ){
Ex. printStackTrace ();
Throw new RuntimeException (ex );
} Finally {
Try {
Out. close ();
} Catch (IOException ex ){
Ex. printStackTrace ();
Throw new RuntimeException (ex );
}
}
}

At this point, the general code has been completed, but the above Code is part of the snippet, just as a reference for everyone to see, and due to my limited level, the Code has not been considered too much in many places, and has not been optimized. It is just entertaining itself, so it may be poorly written in many places, and this program lacks many functions, even the interface does not exist, so the code of the entire program will not be uploaded, so as to avoid shame. I hope to help anyone who is interested.