Implement multi-thread FTP download

Many software programs can download multiple threads, such as NetAnts and jetcar. In fact, the principle of multi-thread download is not complicated. The two main technologies are multithreading and breakpoint download. Multiple Threads are enabled in the program. Each thread uses breakpoint download to download different parts of the file separately. After the download is completed, merge the parts. Multi-threaded programming has been introduced in many books. I will not repeat it here. For resumable download, The cinternetfile class provided in MFC can implement HTTP breakpoint download, but cannot implement FTP breakpoint download. Therefore, we had to start with the commands in the FTP protocol and write an FTP class to implement multi-thread download. I have compiled the cmultiftp class (in Win2000 + iis5. 0) the test is successful.

FTP command details, you can obtain from the http://info.internet.isi.edu/in-notes/rfc/, here to introduce to you with multithreading download related to several command extremely format:

User <username>: the user name used to log on to FTP. If the execution succeeds, 220 is returned;

Pass <password>: password. 230 is returned after successful execution;

Rest <POS>: Specifies the start position of the file download. If the file is successfully downloaded, 350 is returned;

Size <FILENAME>: the file size. If the file is successfully executed, 213 is returned;

PASV: Set up a data connection and obtain the port number used by the FTP server to download the file. If the file is successfully downloaded, 227 is returned;

Type: specifies the type of the downloaded file. If the parameter is "I", the parameter is a binary file, and "A" is a character file, 200 is returned after successful execution;

RETR <FILENAME>: download the file. If the file is successfully downloaded, 125 is returned;

Among these commands, the rest, RETR, and size commands are the most critical. We will give you more details later. In addition, when you execute the FTP command, the FTP server will return a code to the client, the code for successful command execution is provided above. Sends commands to the server as strings, such as: Send (socket, "rest 100 ",...) (Note: add it after the command ).

Before introducing multi-threaded download, we will first introduce the process of connecting to the FTP server and downloading files from the FTP server. It is easy to connect to the FTP server. Create a socket, specify the server address and port number, connect to the server, and send the user and PASS commands to it. If the server returns 230, the login is successful, in addition, the server establishes a control connection with the customer.

The process of downloading files on the FTP server is complicated. First, the client needs to establish a data connection with the server. The port or PASV command can be used to establish a data connection. The PORT command must specify a port number for download. The PASV command is assigned a port number by the server, the client can extract the port number from the server's returned information. The returned information format is:

(Server IP address, port number), my program will use the PASV command. Then, send the RETR command to the server to download the file, or first send a REST command to specify the download path. Then, create a new socket to connect to the specified port of the data connection, and download the file data from this socket. After the download is complete, close the socket.

Now go to the essence of this article to implement multi-threaded download. After the login operation is completed, the "Rest 100" command is sent to test whether the server supports resumable download. If a successful code is returned, multi-threaded download can be implemented. Then, the "size" command is sent ", obtain the file size. Based on the file size, divide the file into several parts, write down the offset address of each part, and use it as a parameter for each thread to download. In the download thread, first accept the parameters (file name, offset address, save address, etc.) that the main thread sends to him, then send the "PASV" command to establish a data connection, create a socket to connect to the new port, and then send the "type I" command to the binary file based on the file type, and send the "Type A" command to the text file; then, run the "Rest <file offset address>" command to notify the server to change the start address of the file to be downloaded. Finally, run the "retr <File Name>" command to download the file. After downloading, merge the part code into a file.

The problem here is how the main thread knows that each download thread has been executed. Windows provides several thread mutex technologies, such as criticalsection and mutex. For details about them, refer to various programming books. Here I recommend the criticalsection technology. You can create a Global Counter in the program, and create a global criticalsection variable in front of the file download. During the download process, after the file is downloaded, The Global criticalsection variable is locked, the counter is added, and the global criticalsection variable is released. In the main thread, you can set up a timer to regularly check the counter value, or ask the download thread to call a function of the main thread after the download is complete. In this way, the main thread can detect that the file has been downloaded and the file can be merged at any time.

The multi-thread download program is designed like this, and it is not difficult at all. It seems that it is best to master some computer technologies, especially network technologies, from the implementation principle, master the most essential part, inspire your own inspiration, and write out an excellent software. Your level will not be greatly improved if you stay on the basis of using other components and function libraries.