Complete Solution for transferring large files based on RMI Service

Complete Solution for transferring large files based on RMI Service

Data Transmission Based on the RMI service is divided into two types of operations: upload and download. Note that there are three main technical points. First, the data transmitted in the RMI service must be serializable. Second, there should be a progress reminder Mechanism during transmission of large files. This is very important for transmission of large files because the transmission time of large files is usually long, the transmission progress of large files must be notified in real time. Third, it is unrealistic to read large files to byte [] at one time for large files, because this will be subject to the available memory of JVM, memory overflow may occur.

The solution based on the RMI service to transmit large files is mainly to solve the problems in the above three aspects. The following sections describe how to upload and download large files.

1. Upload large files based on the RMI Service

1.1. Design Ideas

Uploading a large file is divided into two phases: the CS handshake phase and the file content upload and update phase ".

The CS handshake stage is also called the basic information exchange stage between the client and the server. The client must determine the local file to be uploaded and the target path to be uploaded to the server. Read local files, get the file length, and pass the local file length, local file path, and target path of the server to the server through the interface method. Then, the server generates unique FileKey information, return to the client. This FileKey is the unique identifier of the data transmission channel between the client and the server. These information data types are basic Java data types, so they are all serializable. This interface is called a "handshake interface". Through CS handshake, the server can determine three things: Which file is to be transmitted, the size of the file, and where the file will be stored, these three are the foundation for completing file upload.

Then there is the "File Content upload and update phase ". After the client opens the file, it reads a certain amount of file content each time, for example, each read (1024*1024*2) byte data, this batch of data is transmitted to the server through the upload and update interface. The server writes the data to the output stream and checks whether the file content has been transmitted. If the data has been transmitted, the persistent flush operation is performed to generate a file on the server. After the client transfers a batch of data, the "Total Amount of transmitted data" value is updated and compared with the total file length, in this way, you can get the file upload progress and notify the user in real time. To sum up, the client cyclically reads the local file content and transmits it to the server until the file content is read and uploaded.

1.2. Functional Design

1.2.1 Management of status information related to File Upload

In the process of uploading large files, the most important thing on the server is the precise management of the status information related to the file upload process, such as the total file length, the total number of uploaded bytes, and the file storage path. In addition, ensure that the data is updated in real time and cannot be lost throughout the upload process, and clear the information in a timely manner after the file is uploaded, so as to prevent the server from accumulating excessive invalid status data.

In view of this, we have designed a class RFileUploadTransfer to implement the above functions. The Code is as follows:

/**
* Description: State Information encapsulation class during file upload. <Br>
* Copyright: Copyright (c) 2016 <br>
* Company: smart grid Institute of Henan Electric Power Research Institute <br>
* @ Author shangbingbing compiled on
* @ Version 1.0
*/
Public class RFileUploadTransfer implements Serializable {
Private static final long serialVersionUID = 1L;
Private String fileKey;
// Client file path
Private String srcFilePath;
// The path of the target file uploaded by the server
Private String destFilePath;
// File Size
Private int fileLength = 0;
// Total number of transmitted bytes
Private int transferByteCount = 0;
// Whether the file has been fully written to the server disk
Private boolean isSaveFile = false;
Private OutputStream out = null;

Public RFileUploadTransfer (String srcFilePath, int srcFileLength, String destFilePath ){
This. fileKey = UUID. randomUUID (). toString ();
This. srcFilePath = srcFilePath;
This. fileLength = srcFileLength;
This. destFilePath = destFilePath;

File localFile = new File (this. destFilePath );
If (localFile. getParentFile (). exists () = false ){
LocalFile. getParentFile (). mkdirs ();
}
Try {
This. out = new FileOutputStream (localFile );
} Catch (FileNotFoundException e ){
E. printStackTrace ();
}
}

Public String getFileKey (){
Return fileKey;
}
Public String getSrcFilePath (){
Return srcFilePath;
}
Public String getDestFilePath (){
Return destFilePath;
}
Public boolean isSaveFile (){
Return isSaveFile;
}

Public void addContentBytes (byte [] bytes ){
Try {
If (bytes = null | bytes. length = 0 ){
Return;
}
If (this. transferByteCount + bytes. length> this. fileLength ){
// If the data length has been transferred + the Data Length in this batch> the file length, it indicates that this batch of data is the last batch of data;
// This batch of data may contain free bytes, so you need to filter them out.
Byte [] contents = new byte [this. fileLength-this. transferByteCount];
For (int I = 0; I <contents. length; I ++ ){
Contents [I] = bytes [I];
}
This. transferByteCount = this. fileLength;
This. out. write (contents );
} Else {
// This batch of data is not the last batch of data, and the file has not been transferred.
This. transferByteCount + = bytes. length;
This. out. write (bytes );
}
If (this. transferByteCount> = this. fileLength ){
This. out. flush ();
This. isSaveFile = true;
If (this. out! = Null ){
Try {
This. out. close ();
} Catch (IOException e ){
E. printStackTrace ();
}
}
}
} Catch (Exception ex ){
Ex. printStackTrace ();
}
}
}

Then, construct a thread-safe set in the RMI service interface method implementation class to store and manage the transmission process of each large file. The Code is as follows:

/**
* Upload File status monitor
*/
Private Hashtable <String, RFileUploadTransfer> uploadFileStatusMonitor = new Hashtable <String, RFileUploadTransfer> ();

1.2.2 CS handshake Interface Design

The CS handshake interface is named startUploadFile. The main function is to transmit the basic information of the exchanged file and construct the state control object during file upload. Its code in the interface implementation class is as follows:

@ Override
Public String startUploadFile (String localFilePath, int localFileLength, String remoteFilePath) throws RemoteException {
RFileUploadTransfer fileTransfer = new RFileUploadTransfer (localFilePath, localFileLength, remoteFilePath );
If (this. uploadFileStatusMonitor. containsKey (fileTransfer. getFileKey ())){
This. uploadFileStatusMonitor. remove (fileTransfer. getFileKey ());
}
This. uploadFileStatusMonitor. put (fileTransfer. getFileKey (), fileTransfer );
Return fileTransfer. getFileKey ();
}

1.2.3 file content upload and update Interface Design
The file content upload and update interface is named updateUploadProgress. The main function is to receive the file content byte [] transmitted by the client. Its code in the interface implementation class is as follows:

@ Override
Public boolean updateUploadProgress (String fileKey, byte [] contents) throws RemoteException {
If (this. uploadFileStatusMonitor. containsKey (fileKey )){
RFileUploadTransfer fileTransfer = this. uploadFileStatusMonitor. get (fileKey );
FileTransfer. addContentBytes (contents );
If (fileTransfer. isSaveFile ()){
This. uploadFileStatusMonitor. remove (fileKey );
}
}
Return true;
}

1.2.4 client Design

The main function of the client is to open a local file, read the file content byte [] in batches, call the RMI Interface Method for transmission, and remind the transmission progress. The following is the test code developed by the author using swing and the real-time progress reminder using JProgressBar.

ProgressBar. setMinimum (0 );
ProgressBar. setMaximum (100 );
InputStream is = null;
Try {
File srcFile = new File (localFilePath );
Int fileSize = (int) srcFile. length ();
String fileKey = getFileManageService (). startUploadFile (localFilePath, fileSize, remoteFilePath );

Byte [] buffer = new byte [1024*1024*2];
Int offset = 0;
Int numRead = 0;
Is = new FileInputStream (srcFile );
While (-1! = (NumRead = is. read (buffer ))){
Offset + = numRead;
GetFileManageService (). updateUploadProgress (fileKey, buffer );
Double finishPercent = (offset * 1.0/fileSize) * 100;
ProgressBar. setValue (int) finishPercent );
}
If (offset! = FileSize ){
Throw new IOException ("file cannot be fully read" + localFilePath );
} Else {
ProgressBar. setValue (100 );
}
} Catch (Exception ex ){
Ex. printStackTrace ();
} Finally {
Try {
If (is! = Null ){
Is. close ();
}
} Catch (IOException e ){
E. printStackTrace ();
}
}

The instance interface is as follows:

For more details, please continue to read the highlights on the next page:

1 2 Next Page