[ASP.net documentation] use C # to implement multi-threaded file transmission over HTTP

Many people have experienced the use of network ant financial or network Express software to download Internet Files. The use of these software can greatly speed up the transmission of files on the Internet and reduce the time for file transmission. Why are these software so powerful? The main reason is that these software uses multi‑thread download and resumable data transfer. If we compile a program like this and can quickly download files on the internet, it must be a pleasure. Next I will explain how to use the C # language to compile a program that supports multi-threaded File Download. You will see how easy it is to use the C # language to compile network applications, you can also learn about the powerful network functions in C.

First, let's introduce the HTTP protocol, which is short for Hpyer Text Transfer Protocal. It is the most important network protocol on the modern Internet. Hypertext Transfer Protocol is located at the TCP/IP protocol application layer, is a simple and fast C/S-oriented protocol. The HTTP process is divided into four steps: connection, request, response, and disconnection. C # language provides good support for the HTTP protocol. NET Class Library provides the WebRequest and WebResponse classes, both of which are included in the System. in the. Net namespace, using these two classes can implement many advanced network functions. In this article, multi-threaded file downloads are implemented using these two classes. Both WebRequest and WebResponse are abstract base classes. Therefore, they cannot be used directly as objects in a program and must be inherited. In actual use, you can select appropriate subclasses Based on the URI prefix in the URI parameter, for Uris such as HTTP, The HttpWebRequest and HttpWebResponse classes can be used to process HTTP Communication between client programs and WEB servers.

The HttpWebRequest class provides many advanced functions for accessing files on WEB Servers through HTTP. The HttpWebRequest class supports attributes and methods defined in WebRequest. HttpWebRequest exposes the values of public HTTP headers sent to Internet resources as attributes, which are set by methods or systems, common HTTP headers set by attributes or methods are: Accept, set by Accept, connect, set by Connection and KeepAlive attributes, and Content-Length, set by ContentLength, content-Type, set by the ContentType attribute, range, set by the AddRange method. in actual use, the header information is correctly set and then transmitted to the WEB server. The WEB server responds as required.

The HttpWebResponse class inherits from the WebResponse class and specifically handles HTTP responses returned from the WEB server. This class implements many methods and has many attributes that can fully process received Internet information. In the HttpWebResponse class, for most common HTTP header fields, there are independent attributes corresponding to them. Through these attributes, programmers can conveniently access the information in the header field of the HTTP received message, in this example, the HttpWebResponse class attribute is: ContentLength, which is the length of the received content.

With the above understanding, let's take a look at the usage of these two classes. To create an HttpWebRequest object, do not directly use the HTTP WebRequest constructor, but use WebRequest. the Create method initializes an HttpWebRequest instance, for example:

HttpWebRequest hwr = (HttpWebRequest) WebRequest. Create (http://www.163.com /);

After this object is created, you can set the content of many HTTP header fields through the HttpWebRequest attribute, such as hwr. AddRange (,); set the receiving object range to-bytes.

When the HttpWebReques object uses the GetResponse () method, an HttpWebResponse object is returned. To propose HTTP Response Message Information, you must use the GetResponseStream () method of HttpWebResponse to return a Stream object, you can read the packets returned by HTTP. For example, you must first define a Strean object public System. IO. stream ns; then ns = hwr. getResponse (). getResponseStream (); To create a Stream object. With the above preparation knowledge, we start to design our multi-threaded Internet File Download program. First, open Visual Studio.. Net integrated development environment, select "file", "new", and "project", then select "Visual C # Project", and select "Windows application" in the list box on the right of the wizard ", enter the project name, for example, httpftp, and then click OK. The wizard automatically generates a Windows application project. First, open the window designer design application window and add the following controls:

A listBox1 three text labels label1-label3 three text boxes textBox1-textBox3 a start receiving button button1 designed window such:

The control definition code is:

Public System. Windows. Forms. ListBox listBox1;
Private System. Windows. Forms. Label label1;
Private System. Windows. Forms. TextBox textBox1
Private System. Windows. Forms. Button button1;
Private System. Windows. Forms. Label label2;
Private System. Windows. Forms. TextBox textBox2;
Private System. Windows. Forms. Label label3;
Private System. Windows. Forms. TextBox textBox3;
Private System. Windows. Forms. Label label4;
Private System. Windows. Forms. TextBox textBox4;

Open the Form1 code editor and add the following namespace:

Using System. Net; // Network Function
Using System. IO; // stream support
Using System. Threading; // thread support

Add the following program variables:

Public bool [] threadw; // indicates the end of each thread.
Public string [] filenamew; // file name of the file received by each thread
Public int [] filestartw; // start position of the file received by each thread
Public int [] filesizew; // the size of the file received by each thread
Public string strurl; // URL of the accepted File
Public bool hb; // file merging mark
Public int thread; // number of processes

Define an HttpFile class to manage the receiving thread. The Code is as follows:

Public class HttpFile
{
Public Form1 formm;
Public int threadh; // thread code
Public string filename; // file name
Public string strUrl; // the URL of the received File
Public FileStream fs;
Public HttpWebRequest request;
Public System. IO. Stream ns;
Public byte [] nbytes; // receiving buffer
Public int nreadsize; // number of received bytes
Public HttpFile (Form1 form, int thread) // Constructor
{
Formm = form;
Threadh = thread;
}
~ HttpFile () // destructor
{
Formm. Dispose ();
}
Public void receive () // receiving thread
{
Filename = formm. filenamew [threadh];
StrUrl = formm. strurl;
Ns = null;
Nbytes = new byte [2, 512];
Nreadsize = 0;
Formm. listBox1. Items. Add ("Thread" + threadh. ToString () + "start receiving ");
Fs = new FileStream (filename, System. IO. FileMode. Create );
Try
{
Request = (HttpWebRequest) HttpWebRequest. Create (strUrl );
// Receiving start position and length
Request. AddRange (formm. filestartw [threadh],
Formm. filestartw [threadh] + formm. filesizew [threadh]);
Ns = request. GetResponse (). GetResponseStream (); // get the receiving stream
Nreadsize = ns. Read (nbytes, 0,512 );
While (nreadsize> 0)
{
Fs. Write (nbytes, 0, nreadsize );
Nreadsize = ns. Read (nbytes, 0,512 );
Formm. listBox1. Items. Add ("Thread" + threadh. ToString () + "receiving ");
}
Fs. Close ();
Ns. Close ();
}
Catch (Exception er)
{
MessageBox. Show (er. Message );
Fs. Close ();
}
Formm. listBox1. Items. Add ("process" + threadh. ToString () + "received! ");
Formm. threadw [threadh] = true;
}
}

This class is in the same namespace as the Form1 class, but not included in the Form1 class. The following defines the Event Response Function of the "start to receive" button control:

Private void button#click (object sender, System. EventArgs e)
{
DateTime dt = DateTime. Now; // start receiving time
TextBox1.Text = dt. ToString ();
Strurl = textBox2.Text. Trim (). ToString ();
HttpWebRequest request;
Long filesize = 0;
Try
{
Request = (HttpWebRequest) HttpWebRequest. Create (strurl );
Filesize = request. GetResponse (). ContentLength; // get the length of the target file
Request. Abort ();
}
Catch (Exception er)
{
MessageBox. Show (er. Message );
}
// Number of receiving threads
Thread = Convert. ToInt32 (textBox4.Text. Trim (). ToString (), 10 );
// Initialize the Array Based on the number of threads
Threadw = new bool [thread];
Filenamew = new string [thread];
Filestartw = new int [thread];
Filesizew = new int [thread];

// Calculate the size of the file that each thread should receive
Int filethread = (int) filesize/thread; // average allocation
Int filethreade = filethread + (int) filesize % thread; // the remaining part is completed by the last thread.
// Assign values to Arrays
For (int I = 0; I <thread; I ++)
{
Threadw [I] = false; // The initial value of each thread state is false.
Filenamew [I] = I. ToString () + ". dat"; // The temporary file name of the file received by each thread
If (I <thread-1)
{
Filestartw [I] = filethread * I; // The starting point for each thread to receive files
Filesizew [I] = filethread-1; // The length of the file received by each thread
}
Else
{
Filestartw [I] = filethread * I;
Filesizew [I] = filethreade-1;
}
}
// Define the thread array and start the receiving thread
Thread [] threadk = new Thread [thread];
HttpFile [] httpfile = new HttpFile [thread];
For (int j = 0; j <thread; j ++)
{
Httpfile [j] = new HttpFile (this, j );
Threadk [j] = new Thread (new ThreadStart (httpfile [j]. receive ));
Threadk [j]. Start ();
}
// Start the file thread to merge received by each thread
Thread hbth = new Thread (new ThreadStart (hbfile ));
Hbth. Start ();
}

The thread hbfile of the merged file is defined in the Form1 class and defined as follows:

Using System;
Using System. Collections. Generic;
Using System. ComponentModel;
Using System. Data;
Using System. Drawing;
Using System. Linq;
Using System. Text;
Using System. Windows. Forms;

Namespace HttpDownload
{
Public partial class Form1: Form
{
Public int threadNum; // Process
Public bool [] threadStatus; // indicates the end of each thread.
Public string [] fileNames; // file name of the file received by each thread
Public int [] fileStartPos; // start position of the file received by each thread
Public int [] fileSize; // the size of the file received by each thread
Public string url; // URL of the accepted File
Public bool isMerge; // file merging flag

Public Form1 ()
{
InitializeComponent ();
}
/// <Summary>
///
/// </Summary>
/// <Param name = "sender"> </param>
/// <Param name = "e"> </param>
Private void btn_rec_Click (object sender, EventArgs e)
{
Url = this.txt _ url. Text. Trim (). ToString ();

System. Net. HttpWebRequest request;
Long fileSizeAll = 0;

Request = (System. Net. HttpWebRequest) System. Net. HttpWebRequest. Create (url );
FileSizeAll = request. GetResponse (). ContentLength;
Request. Abort ();
ThreadNum = int.Parse(this.txt _ threadNum. Text. Trim (). ToString ());
Init (fileSizeAll );
For (int I = 0; I <threadNum; I ++)
{
This. lst_infos.Items.Add ("Thread" + I + ":" +
"Thread status:" + threadStatus [I]. toString () + "-start position:" + fileStartPos [I]. toString () + "-size:" + fileSize [I]. toString ());
}
This. lst_infos.Items.Add ("total file size:" + fileSizeAll );

// Define and start the thread Array
System. Threading. Thread [] threads = new System. Threading. Thread [threadNum];
HttpFile [] httpDownloads = new HttpFile [threadNum];
For (int I = 0; I <threadNum; I ++)
{
HttpDownloads [I] = new HttpFile (this, I );
Threads [I] = new System. Threading. Thread (new System. Threading. ThreadStart (httpDownloads [I]. receive ));
Threads [I]. Start ();
}
System. Threading. Thread merge = new System. Threading. Thread (new System. Threading. ThreadStart (MergeFile ));
Merge. Start ();
This.txt _ overTime. Text = DateTime. Now. ToString ();
}
/// <Summary>
/// Initialization
/// </Summary>
/// <Remarks>
/// Average file size allocated to each thread. The remaining part is completed by the last thread.
/// </Remarks>
/// <Param name = "filesize"> </param>
Private void Init (long filesize)
{
ThreadStatus = new bool [threadNum];
FileNames = new string [threadNum];
FileStartPos = new int [threadNum];
FileSize = new int [threadNum];
Int filethread = (int) filesize/threadNum;
Int filethreade = filethread + (int) filesize % threadNum;
For (int I = 0; I <threadNum; I ++)
{
ThreadStatus [I] = false;
FileNames [I] = I. ToString () + ". dat ";
If (I <threadNum-1)
{
FileStartPos [I] = filethread * I;
FileSize [I] = filethread-1;
}
Else
{
FileStartPos [I] = filethread * I;
FileSize [I] = filethreade-1;
}
}
}
/// <Summary>
/// Merge files
/// </Summary>
Public void MergeFile ()
{
While (true)
{
IsMerge = true;
For (int I = 0; I <threadNum; I ++)
{
If (threadStatus [I] = false) // if a thread is not terminated, wait
{
IsMerge = false;
System. Threading. Thread. Sleep (100 );
Break;
}
}
If (isMerge = true) // otherwise, stop waiting
{
Break;
}
}

System. IO. FileStream fs;
System. IO. FileStream fstemp;
Int readfile;
Byte [] bytes = new byte [1, 512];
Fs = new System. IO. FileStream (txt_localFile.Text.Trim (). ToString (), System. IO. FileMode. Create );
For (int k = 0; k <threadNum; k ++)
{
Fstemp = new System. IO. FileStream (fileNames [k], System. IO. FileMode. Open );
While (true)
{
Readfile = fstemp. Read (bytes, 0,512 );
If (readfile> 0)
{
Fs. Write (bytes, 0, readfile );
}
Else
{
Break;
}
}
Fstemp. Close ();
}
Fs. Close ();
System. Windows. Forms. MessageBox. Show ("Received !!! ");
}
}
} At this point, a program that downloads files with multiple threads is finished. Note that when you enter a local file name, enter the following format: "c: \ test \ httpftp \ bin \ d.htm ", because the character after" \ "is an escape character in C #, the larger the number of threads is, the better, generally, you can run the program on five threads in Visual Studio.. Net 2002 development environment and Windows xp operating system.