Multi-threaded downloader (excluding the database section)

Multi-threaded downloader (excluding the database section)

1. Write in front:

Although the program framework has been set up in the demo, but due to the author's time, the core part is only completed: Multi-threaded Download part, other database, service notification, pause part has not been added to the project.

2, relevant knowledge points:

(1) Java threads and how to stop threads

(2) Java randomaccessfile file operation

(3) configuration of the httpurlconnection related range field

(4) Sqlite synchronous Operation

2. Core idea:

(1) Through httpurlconnection to determine whether the server support power-down continued to pass:

<1> No , directly open the normal multi-threaded download (when the network is broken, etc. will be re-downloaded)

<2> is a Common multi-threaded download, but each thread contains its own download progress information so that the network is disconnected or the user pauses to start downloading again to re-open the download. The author in the different download size intelligent allocation of different number of threads to download resources, by setting the buffer size to improve the download speed.

3, the core technology:

(1) Configuration of HttpURLConnection

(2) Randomaccessfile random file read and buffer settings

(3) thread pause and start

4. Analysis results:

(1) divide the function into three parts: Downloader (Unified external interface), memory (internal storage implementation), notification Service (User interaction section).

(2) Project structure:

5. Core code:

Package Com.jx.downloader;import Java.io.bufferedinputstream;import Java.io.file;import java.io.IOException;import Java.io.inputstream;import Java.io.randomaccessfile;import Java.net.httpurlconnection;import Java.net.malformedurlexception;import Java.net.url;import Java.util.arraylist;import Android.content.Context; Import Android.os.environment;import Android.text.textutils;import Android.util.log;import Com.jx.dbhelper.downloadrecorddb;import com.jx.model.downloadmodel;/** * Custom Downloader: 1, according to the content size of the download of the intelligent allocation of the number of download threads, Each thread carries its own thread's download information via downloader (download start, end point, thread name) * 2, if the server supports power-down to turn on the database *, when the download is in a paused state (causes the download pause may be a manual pause or poor network), Automatically saves the download information to the database, automatically clears the database information when the download is canceled, * when the download is restarted, the read content is re-downloaded (typically the local storage variable has not been reclaimed and is not re-read from the database) 2, content downloaded using randomaccessfile storage * * @ Author J_x March 19, 2016 09:58:00 */public class Jx_downloader {/** * 0~3m download range default on 1 threads */private final static long Below_thre E_m_size = 5 * 1024x768 * 1024;/** * 3~6m The download range is turned on by default 2 threads */private final static Long below_six_m_size = 6 * 1024 * 1024;/** * 10 The ~18m download range is enabled by default for 4 threads */privAte final static long below_eighteen_m_size = 18 * 1024 * 1024;/** * thread paused flag */private static volatile Boolean Isstopdow  nloading = false;/** * The ID of the downloader, thought can create multiple downloader */private long downloaderid;/** * need to download the resource link */private String resourceurl;/** * Download the link to the URL Protocol object */private URL resourceurl;/** * The total number of bytes required to download the resource */private long resourcesize;/** * Required total quantity of download threads */private int totaltreadnum;/** * Whether the server supports breakpoint continuation function */private boolean issupportloadingandsaving;/** * Download the database for the resource store */private            DOWNLOADRECORDDB downlaoddb;/** * File storage directory */private String savafilename;/** * Constructor no subclass so no default constructor is provided * * @param context * Context * @param DownloadURL * The link to be downloaded * @param savefilename * Download file name with suffix */public jx_downlo Ader (context context, String downloadurl,string savefilename) throws Malformedurlexception {//TODO auto-generated Constructor Stubresourceurl = Downloadurl;this.resourceurl = new URL (resourceurl); this.downlaoddb = new DOWNLOADRECORDDB (context);//If the user does not set the download suffix, provide the default storage folder if (TextutiLs.isempty (Savefilename)) {this.savafilename = "jx_downloader.txt";} else {this.savafilename = Savefilename;}} /** * Open Download task, external to provide convenient download method */public void Startdownload () {New Thread (new Runnable () {@Overridepublic void Run () {//TODO Auto -generated method Stubtry {readydownload ();} catch (IOException e) {//TODO auto-generated catch blocklog.e ("Debug", "Down Loading fail!!! ");}}). Start ();} /** * Core Downloader (handled with a similar façade mode method with strict execution order required) * * @throws IOException * */private void Readydownload () throws IOException {// Determine if the current connection server supports a breakpoint download//this.issupportloadingandsaving = false;///** does not support a friendly prompt, only multi-threaded download Task *///if (! issupportloadingandsaving) {//LOG.E ("Debug", "Server don t support Pause_save download!"); /} else {////** Supported: 1, open database store individual thread progress *///}/* multithreaded download process *///1, calculate the resource size to download httpurlconnection httpurlconnection = Settingreq Uesthttp (null); if (httpurlconnection.getresponsecode () = =) {//TODO: Need to further optimize network LOG.E ("Debug", "coonected Sucessfully ");} Resourcesize = Httpurlconnection.getcontentlength ();if (resourcesize <= 0) {log.e ("Debug", "Unkown file length and return"), return;} else {log.e ("Debug", "File length:" + Resourcesize + "bytes");} 2, according to user settings or resource size smart settings Download the total number of threads int tempthreadnum = Settotaltreadnum (resourcesize);//3, according to the number of threads allocated, to "divide" the resources, and set headerarraylist<downloadmodel> Downloadarray = new arraylist<downloadmodel> (); issupportloadingandsaving) {//read from database Downloadarray = Downlaoddb.getallinfo ();} else {//read from local method Downloadarray = Initresouresize (Tempthreadnum, resourcesize);} 4. Divide the thread for download for (int i = 0; i < downloadarray.size (); ++i) {Downloadmodel Tempmodel = Downloadarray.get (i); Myrunnable runnable = new myrunnable (Tempmodel); Thread thread = new Thread (runnable); Thread.Start ();}} /** * Configure HTTP information, ready to start download data * * @throws ioexception */private httpurlconnection settingrequesthttp (Downloadmodel model) throw s IOException {httpurlconnection coon = (httpurlconnection) this.resourceURL.openConnection (); coon.setconnecttimeout (3 * +); Coon.setrequestmethod ("GET"); coon. Setrequestproperty ("Accept", "Image/gif, Image/jpeg, Image/pjpeg, Image/pjpeg, Application/x-shockwave-flash, Application/xaml+xml, Application/vnd.ms-xpsdocument, APPLICATION/X-MS-XBAP, Application/x-ms-application, Application/vnd.ms-excel, Application/vnd.ms-powerpoint, Application/msword, */* "); Coon.setrequestproperty (" Accept-language "," ZH-CN "), Coon.setrequestproperty (" Referer ", ResourceUrl); Coon.setrequestproperty (" Charset "," UTF-8 "); Coon.setrequestproperty (" User-agent "," mozilla/4.0 "(compatible; MSIE 8.0; Windows NT 5.2; trident/4.0;. NET CLR 1.1.4322;. NET CLR 2.0.50727;. NET CLR 3.0.04506.30;. NET CLR 3.0.4506.2152;. NET CLR 3.5.30729); c Oon.setrequestproperty ("Connection", "keep-alive");//The core code of the continuation of the breakpoint, setting the download interval, note that bytes= has eaten the loss if (model! = NULL) { Coon.setrequestproperty ("Range", "bytes=" + model.getdownloadedlength () + "-" + model.getdownloadlengthsum ());} return coon;} /** * Multi-threaded Download Shared Run method * * @author Administrator * */private class Myrunnable implements Runnable {private DownloadmodeL Rmodel;public myrunnable (Downloadmodel model) {This.rmodel = model;} @Overridepublic void Run () {try {log.i ("Debug", Rmodel.getthreadname ()); HttpURLConnection coon = settingrequesthttp (Rmodel), if (coon.getresponsecode () = =) {LOG.E ("Debug", "coonected Sucessfully ");} Resourcesize = Coon.getcontentlength (); if (resourcesize <= 0) {log.e ("Debug", "Unkown file Length and return"); return; } else {Readresourceandsave (coon, Rmodel);}} catch (IOException e) {e.printstacktrace ();}}} /** * Read network data from the Internet * * @throws ioexception */private void Readresourceandsave (HttpURLConnection coon, Downloadmodel model ) throws IOException {//Create folder File Savdir = Environment.getexternalstoragedirectory (); File File = null;if (Environment.getexternalstoragestate (). Equals (environment.media_mounted)) {if (!savdir.exists ()) {Savdir.mkdirs ();} File = new file (Savdir, this.savafilename);} Randomaccessfile rfile = new Randomaccessfile (file, "RWD");//jump to the starting position of the current thread's read and write to the file Rfile.seek (Model.getdownloadedlength ( )); InputStream Finput = Coon.getinputstream (), if (finput! = null) {LOG.E ("Debug", "Finput=>" + finput.tostring ());} else {log.e ("debug "," Finput is null and return ");} LOG.E ("Debug", "rfile.getfilepointer=>" + rfile.getfilepointer ()); Bufferedinputstream binput = new Bufferedinputstream (finput);//set 3KB buffer, speed up the download speed, here later changed to by character or the whole line read way to optimize int size = 3 * 1024; byte[] readbytes = new Byte[size];int Readnum = 0;long sum = 0;//when there is data readable and the thread is not paused long Needreadnum = Model.getdownloadlength SUM ()-model.getdownloadedlength (), while ((Readnum = Binput.read (readbytes, 0, size)) > 0&& sum < NEEDREADN Um &&!isstopdownloading) {sum + = readnum;//If the size of the request is about to exceed the total demand quantity, the corrected read content is guaranteed not to read more if (sum + size) > Needread Num) {size = (int) (needreadnum-sum);} Write to File Rfile.write (readbytes, 0, readnum);} LOG.E ("Debug", Model.getthreadname () + "Finish downloading:" + rfile.length () + "-bytes"); Rfile.close (); Finput.close () ;} /** * Calculate the resource size to download * * @param URL * Resource Content link to download * @return The number of bytes returned by the resource */private Arraylist<downloadmodel> initresouresize (int tnum,long contentlength) {Long templastsize = contentLength;// The remaining download content size arraylist<downloadmodel> Downloadarray = new arraylist<downloadmodel> (); Long commonsize = contentlength/tnum;for (int i = 0; i < Tnum; ++i) {Downloadmodel Downloadmodel = new Downloadmodel ();//The download has not started, so the downloaded The size is 0byte, and his length represents the download starting point of the next thread downloadmodel.setdownloadedlength (i * commonsize); Long loadending = templastsize;// Download size of the last download thread = Bus path-Other thread download size and if (i = = tNum-1) {loadending = ContentLength;} else {loadending = commonsize * i + common Size;} Templastsize = Contentlength-commonsize;downloadmodel.setdownloadlengthsum (loadending); Downloadmodel.setthreadname ("Jx_download_thread" + i);//If a breakpoint is supported for continuation, it is stored in the database, otherwise it is temporarily stored in ArrayList if ( issupportloadingandsaving) {Downlaoddb.insert (Downloadmodel);} else {downloadarray.add (Downloadmodel);}} return Downloadarray;} /** * Download the total number of threads according to the smart settings of the resource size * * @param totaltreadnum * Set the amount of downloaded data */private int settotaltreadnum (lOng contentlength) {//If the user does not set the number of downloaded threads then intelligently sets the IF (0 = = This.totaltreadnum) {if (contentlength <= 0) {log.i ("Debug", "bottom") according to the size of the The contents of the download are too small! "); return 0;} else if (ContentLength < below_three_m_size) {this.totaltreadnum = 1;} else if (ContentLength < below_six_m_size) {T His.totaltreadnum = 2;} else if (ContentLength < below_eighteen_m_size) {this.totaltreadnum = 4;} else {this.totaltreadnum = 5;}} return this.totaltreadnum;} /** * Determine if the current connection server supports a breakpoint download * * @return default does not support power-down */public boolean issupportloadingandsaving () {return Issupportloadingan dsaving;} Public long Getdownloaderid () {return downloaderid;} public void Setdownloaderid (long downloaderid) {This.downloaderid = Downloaderid;} public int Gettotaltreadnum () {return totaltreadnum;} Public String Getresourceurl () {return resourceurl;} Public long Getresourcesize () {return resourcesize;}}

Downloader uses:

Package Com.jx.main;import Java.net.malformedurlexception;import Android.app.activity;import android.os.Bundle; Import Android.view.view;import Android.view.view.onclicklistener;import Android.widget.button;import Com.example.mutilthreaddownloader.r;import com.jx.downloader.jx_downloader;/** * March 19, 2016 09:41:00 * * @author J_X    Multithreaded test class */public class Mainactivity extends Activity {jx_downloader Downloader; Button btndownlaod; @Overrideprotected void OnCreate (Bundle savedinstancestate) {super.oncreate (savedinstancestate); Setcontentview (r.layout.activity_main); btndownlaod = (Button) Findviewbyid (r.id.btn_start_downlaoding); Btndownlaod.setonclicklistener (New Onclicklistener () {@Overridepublic void OnClick (View v) {//TODO auto-generated Method Stubtry {downloader = new Jx_downloader (mainactivity.this, "****.apk" "****.apk"); Downloader.startdownload ();} catch (Malformedurlexception E1) {//TODO auto-generated catch Blocke1.printstacktrace ();}});}}

Summary:

The speed of the downloader, out of the external factors speed, the server transfer rate, the internal factors are mainly the number of threads and buffers can affect the download speed, but the author in the Meizu Cool millet virtual machine on the same WiFi same number of threads and the same size buffer, download speed download speed is very slow, still need to verify the specific reasons.

Multi-threaded downloader (excluding the database section)