Download tools I think few people will not use it, the previous period of time is more boring, spent a little time in Java to write a simple HTTP multithreaded download program, is purely boring to write, only to achieve a few simple functions, but also did not write the interface, today is also a boring day, take to write an article, to swim, Feel good to give a applause, not good also don't spray, thank you!
My implementation of this HTTP download tool function is very simple, is a multi-threaded and a breakpoint recovery, of course, download is essential. Then probably tidy up the thing to do first :
1, connect the resource server, get the resource information, create the file
2, splitting resources, multithreading download
3. Breakpoint Recovery function
4, download rate statistics
This is probably the point, then the first thing to do is to connect resources and obtain resource information, I use the javase from the URLConnection for resource connection, the approximate code is as follows:
Copy Code code as follows:
String urlstr = "http://www.sourcelink.com/download/xxx"; Resource address, casually written.
URL url = new URL (urlstr); Create a URL
URLConnection con = url.openconnection (); Establish a connection
Contentlen = Con.getcontentlength (); Get resource length
File File = new file (filename); Create a download file based on filename, and it will be the file we downloaded.
Very simple, yes it is so simple, the first step is done, then next to do a second step, splitting resources, to achieve multithreading. In the previous step we have obtained the length of the resource Contentlen, so how to divide the resources according to this? If we were going to run 10 threads, we would first put Contentlen at 10, get the size of each chunk, and then create 10 threads per thread, each one of which is responsible for writing, which requires the use of the class of Randomaccessfile, which provides random access to files , you can specify a write operation to a location in the file, roughly the following code:
Copy Code code as follows:
Long Sublen = Contentlen/threadqut; Get the size of each block
Create 10 threads and start the thread
for (int i = 0; i < Threadqut; i++) {
Dlthread thread = new Dlthread (this, i + 1, Sublen * I, Sublen * (i + 1)-1); Creating Threads
Dlthreads[i] = thread;
QSEngine.pool.execute (Dlthreads[i]); Handing threads to the thread pool for management
}
Using the Dlthread class here, let's take a look at the definition of how this class is constructed:
Public Dlthread (dltask dltask, int id, long startpos, long Endpos)
The first parameter is a dltask, this class represents a download task, which mainly holds the information of the download task, including the download resource name, local filename and so on. The second parameter is an ID that marks the thread, and if there are 10 threads, then the ID is from 1 to 10, and the third parameter startpos represents where the thread starts writing from the file, and the last parameter Endpos where the representative writes it ends.
Let's take a look at how to download a thread after it has started, see the Run method:
Copy Code code as follows:
public void Run () {
SYSTEM.OUT.PRINTLN ("thread" + ID + "start");
Bufferedinputstream bis = null; Create a Buff
Randomaccessfile fos = null;
byte[] buf = new Byte[buffer_size]; Buffer size
URLConnection con = null;
try {
con = url.openconnection (); Create a connection, where a connection is created for each thread
Con.setallowuserinteraction (TRUE);
if (isnewthread) {
Con.setrequestproperty ("Range", "bytes=" + startpos + "-" + endpos);/set the scope for obtaining resource data, from Startpos to Endpos
FOS = new Randomaccessfile (file, "RW"); Create Randomaccessfile
Fos.seek (startpos); Starting from Startpos
} else {
Con.setrequestproperty ("Range", "bytes=" + CurPos + "-" + endpos);
FOS = new Randomaccessfile (Dltask.getfile (), "RW");
Fos.seek (CurPos);
}
The following paragraph writes data to a file, CurPos is unknown to the current write, and here determines whether it is less than Endpos,
If the endpos is exceeded, the thread has finished executing
bis = new Bufferedinputstream (Con.getinputstream ());
while (CurPos < Endpos) {
int len = bis.read (buf, 0, buffer_size);
if (len = = 1) {
Break
}
Fos.write (buf, 0, Len);
CurPos = CurPos + len;
if (CurPos > Endpos) {
ReadByte = Len-(curpos-endpos) + 1; Gets the number of bytes read correctly
} else {
ReadByte = Len;
}
}
SYSTEM.OUT.PRINTLN (thread + ID +) has been downloaded. ");
This.finished = true;
Bis.close ();
Fos.close ();
catch (IOException ex) {
Ex.printstacktrace ();
throw new RuntimeException (ex);
}
}
The above code is based on startpos and Endpos file model write operations, each thread has its own independent resource block, from Startpos to Endpos. The above way is the thread download the core, multithreading after the next is to achieve the function of breakpoint recovery, in fact, the breakpoint recovery is nothing more than the record of each thread completed to which unknown, where I was using CurPos record, everyone in the above code should be able to see, I will record the curpos of each thread, and then when the thread restarts, take CurPos as Startpos, and Endpost is the same, and you have not noticed that there is a code in the Run method:
Copy Code code as follows:
if (isnewthread) {//To determine whether a breakpoint, if true, represents a new download thread, rather than a breakpoint recovery
Con.setrequestproperty ("Range", "bytes=" + startpos + "-" + endpos);/set the scope for obtaining resource data, from Startpos to Endpos
FOS = new Randomaccessfile (file, "RW"); Create Randomaccessfile
Fos.seek (startpos); Starting from Startpos
} else {
Con.setrequestproperty ("Range", "bytes=" + CurPos + "-" + endpos);//Use CurPos instead of startpos, others are the same as the newly created one.
FOS = new Randomaccessfile (Dltask.getfile (), "RW");
Fos.seek (CurPos);
}
The above is the practice of breakpoint recovery, and a new thread is no different, but startpos is not the same, all the same, but only this is not enough, because if the program is closed, how this information is saved? For example, the filename, the curpos of each thread, and so on, when you use the download software, you will find that when the software is not finished downloading, there will be two temporary files in the directory, and one of them is used to save the information of the download task, if there is no such information, the program is not know how to restore the download progress. And how do I achieve this? I am lazy, and do not want to create a file to save information, and then I will read the information to create objects, it is too troublesome, so I thought of Java provides serialization mechanism, my idea is to directly serialize the entire Dltask object to the hard disk, It says Dltask This class is used to keep information about each task, so I just want to deserialize the object when I need to recover, and it's easy to implement the breakpoint function, so let's take a look at the information that this object holds:
Copy Code code as follows:
public class Dltask extends Thread implements Serializable {
Private static final long serialversionuid = 126148287461276024L;
Private final static int max_dlthread_qut = 10; Maximum number of download threads
/** *//**
* Download the temporary file suffix, the download will be automatically deleted after completion
*/
Public final static String File_postfix = ". tmp";
Private URL URL;
private file file;
private String filename;
private int id;
private int level;
private int threadqut; Number of download threads, user customizable
private int Contentlen; Download file length
Private long Completedtot; Current Total Download complete
private int costtime; Download time count, recording the time spent downloading
Private String curpercent; Download percent
Private Boolean isnewtask; Whether to create a new download task, possibly a breakpoint renewal task
Private dlthread[] dlthreads; The thread that saved the current task
transient private Dllistener listener; The listener for the current task, for instant access to relevant download information
As the code above, this object implements the serializable interface, saves all information about the task, and includes each thread object Dlthreads, so it's easy to restore the breakpoint, let me write a file to save this information, And then create an object based on this information at the time of recovery, it's killing me. This creates a method to use for breakpoint recovery:
Copy Code code as follows:
private void Resumetask () {
Listener = new Dllistener (this);
File = new file (filename);
for (int i = 0; i < Threadqut; i++) {
Dlthreads[i].setdltask (this);
QSEngine.pool.execute (Dlthreads[i]);
}
QSEngine.pool.execute (listener);
}
It actually reduces the code that first connects the resources and then splits the resources, because the information is already stored under the Dltask object.
See the above code, do not know that you have noticed an object Dllistener No, this object is actually used to monitor the entire task information, where I mainly for two purposes, one is timed to dltask serialization, save task information, for breakpoint recovery, One is to download the rate of statistics, on average how long to do a statistic. Let's take a look at its code and this class is also a separate thread:
Copy Code code as follows:
public void Run () {
int i = 0;
BigDecimal completetot = null; Percentage of completion
Long start = System.currenttimemillis (); Current time, for recording start statistics time
Long end = start;
while (!dltask.iscomplete ()) {//whether the entire task is completed, continue looping without completing
i++;
String percent = dltask.getcurpercent (); Get the current percent complete
Completetot = new BigDecimal (Dltask.getcompletedtot ()); Gets the total number of bytes currently completed
Get the current time, then compare the start time, and if not, get an average download speed by dividing the total number of current finishes by the time used
End = System.currenttimemillis ();
if (end-start!= 0) {
BigDecimal pos = new BigDecimal ((End-start)/1000) * 1024);
System.out.println ("Speed:"
+ Completetot
. Divide (POS, 0, Bigdecimal.round_half_even)
+ "K/S" + percent + "% completed.");
}
Recoder.record (); Log task information to the hard disk
try {
Sleep (3000);
catch (Interruptedexception ex) {
Ex.printstacktrace ();
throw new RuntimeException (ex);
}
}
Here is the information for printing the entire download task after the download is complete
int costtime =+ (int) (System.currenttimemillis ()-start)/1000);
Dltask.setcosttime (Costtime);
String time = Qsdownutils.changesectohms (costtime);
Dltask.getfile (). Renameto (New File (Dltask.getfilename ()));
System.out.println ("Download finished." + time);
}
The invocation of the Recoder.record () method in this method is used to serialize the task object, and the other code is used for statistical information, which can be read in the comments, and the code for the method is as follows:
Copy Code code as follows:
public void record () {
ObjectOutputStream out = null;
try {
out = new ObjectOutputStream (New FileOutputStream (Dltask.getfilename () + ". Tsk"));
Out.writeobject (Dltask);
Out.close ();
catch (IOException ex) {
Ex.printstacktrace ();
throw new RuntimeException (ex);
finally {
try {
Out.close ();
catch (IOException ex) {
Ex.printstacktrace ();
throw new RuntimeException (ex);
}
}
}
Here, the approximate code is complete, however, the above code is part of the fragment, just as a reference for everyone to see, and because of my limited level, code many places have not been too much consideration, not optimized, just entertain themselves, so there may be a lot of places are written very rotten, This program also lacks many functions, even the interface is not, so the entire program code will not upload, lest disgrace, hehe. I hope to help my friends who are interested in it.
