Using Visual C # To process large volumes of Files

In scientific research and enterprise data processing, a large number of file operations are sometimes involved. When processing a large amount of data, you sometimes need to perform multiple-to-one or one-to-multiple file operations, that is, information interaction between multiple data files and one data file, Microsoft's. NET Development Tool C # provides unprecedented high development efficiency and also reflects file operations. Based on the. NET platform and C # development tools, the author has implemented the following functions:

1. Merge multiple text files into one file;

2. A Workbook file is divided into multiple files.

　　I. Programming task description:

1. read data from multiple files to generate a text file

The author encountered this programming requirement when dealing with logging data. There are a number of existing logging data and each well has a file. To further process the existing needs, he will read the data to a new file, in actual processing, it includes computing, filtering, and formatting.

Multiple source files are in the source file path, and the final generated target file exists in the target file path.

The example data is stored in the sourcefile Folder: 1.bln,2.bln,3.bln,4,bln,5 bln,6.bln. the generated total.txt files are stored in the aimfile subfolder.

2. read data from a file and generate multiple text files

The example source file is: xcsj.xls is stored in the aimfile folder, and its data content is:

Well name	Horizon	Subdivision level Number	Top layer depth	Layer Thickness
13-10	S3 4	1	3263.5	1.5
13-10	S3 5	2	3311.6	1.6
... ... ...

The data of multiple wells is in a table. In actual work, you need to select the data of each well to generate an independent file with the suffix. xc, which exists in the path of the target file. The format of the generated file is:

Well name	Horizon	Subdivision level Number	Top layer depth	Layer Thickness
13-10	S3 4	1	3263.5	1.5
13-10	S3 5	2	3311.6	1.6
... ... ...

The generated destination file 13-10.xc is also stored in the aimfile folder.

　　 2. Program Interface

Generate a form. Add the following controls on the form:

1. Three buttons: Merge files, split files, and exit;

2. A list box is used to display Program Execution

3. Two text boxes are used to enter the source file path and target file path respectively;

4. a progress bar to indicate the progress of program execution;

The program interface is as follows:

　　　　　　　　　　



　　 Iii. Main source code:

1. namespace

Using System; using System. Drawing; Using System. Collections; Using System. ComponentModel; Using System. Windows. Forms; Using System. Data; Using System. IO; // Namespaceusing required for using StreamWriter and StreamReader System. Data. OleDb; Using System. Data. SqlClient; // The Namespace required to read the workbook File

2. File merge button RESPONSE METHOD

String winDir1; // Source file path String winDir2; // Target file path String temp1; Int ii; WinDir1 = textBox1.Text. Trim (); WinDir2 = textBox2.Text. Trim () + "\ total.txt "; // Obtain the list of all files in the winDir1 path String [] dirs = Directory. GetFiles (winDir1 ); // Create or open the target file StreamWriter writer = new StreamWriter (winDir2 ); Writer. WriteLine ("this is the Summary File Header "); Writer. WriteLine ("this is the second line of the Summary File "); Ii = 0; Foreach (string dir in dirs) { StreamReader reader = new StreamReader (dir ); Try { Reader. ReadLine (); Do { Temp1 = reader. ReadLine (); Temp1 = temp1.Trim (); Writer. WriteLine (temp1 ); } While (reader. Peek ()! =-1 ); Ii = ii + 1; } Catch (Exception err) { MessageBox. Show (err. Message ); } Finally { Reader. Close (); } } AddListItem ("merge operation ended "); AddListItem ("total operations in this operation" + ii. ToString () + "merge individual files into total.txt "); AddListItem ("total.txt file storage location" + textBox2.Text ); Writer. Close ();

3. File splitting button RESPONSE METHOD

// Define variables String winDir; // File path String filename; // Result file name String [] filename2 = new string [300]; // Save the generated file name String temp1, temp2, temp3, temp4; String strCon; // Connection string Int ii; // Count the number of generated result files Ii = 1; WinDir = textBox2.Text. Trim () + @ "xcsj.xls ;"; // Create a data link StrCon = "Provider = Microsoft. Jet. OLEDB.4.0; Data Source = "+ winDir +" Extended Properties = Excel 8.0 "; OleDbConnection myConn = new OleDbConnection (strCon ); String strCom = "SELECT * FROM [Sheet1 $]"; MyConn. Open (); // Open the data link to obtain a dataset OleDbDataAdapter myCommand = new OleDbDataAdapter (strCom, myConn ); // Create DataSet object DataSet myDataSet = new DataSet (); // Get your own DataSet object MyDataSet Try { MyCommand. Fill (myDataSet, "[Sheet1 $]"); } Catch (Exception err) { MessageBox. Show (err. Message ); } // Read the well name to the Array Based on the well number in the first column in the workbook Temp1 = myDataSet. Tables [0]. Rows [0] [0]. ToString (). Trim (); Filename2 [ii] = temp1; Foreach (DataRow row in myDataSet. Tables [0]. Rows) { If (temp1! = Row [0]. ToString (). Trim ()) { Ii = ii + 1; Filename2 [ii] = row [0]. ToString (); Temp1 = row [0]. ToString (). Trim (); } } // Progress bar Initialization ProgressBar1.Step = 1; ProgressBar1.Minimum = 1; ProgressBar1.Maximum = ii; WinDir = textBox2.Text. Trim (); For (int k = 1; k <= ii; k ++) { Filename = @ winDir + "\" + filename2 [k]. Trim () + ". xc "; StreamWriter writer = new StreamWriter (filename ); Writer. WriteLine ("layer subdivided layer # top depth layer "); Foreach (DataRow row in myDataSet. Tables [0]. Rows) { Temp1 = row [1]. ToString (). Trim (); Temp2 = row [2]. ToString (). Trim (); Temp3 = row [3]. ToString (). Trim (); Temp4 = row [4]. ToString (). Trim (); Writer. WriteLine (temp1 + "" + temp2 + "" + temp3 + "" + temp4 ); } Writer. Close (); ProgressBar1.PerformStep (); } MyConn. Close (); AddListItem ("operation result "); AddListItem ("co-occurrence into" + ii. ToString () + "file "); AddListItem (@ "File Location: F: wangweiworkh12xc path "); ListBox1.Refresh ();

4. An upper list box is added for the displayed execution result. To facilitate the operation of the list box, you need to add a method for the form class:

Private void addListItem (string value) { This. listBox1.Items. Add (value ); }

　　 V. Key Points Analysis

1. Use StreamWriter and StreamReader for File Access

This is a pair of classes derived from Stream for byte input and output. The namespace is system. io.

2. Reading Electronic Table Data

This program provides a method to connect to an e-Table file, and the necessary namespace should be added. Note that the connection string should not be written incorrectly.

3. Use the GetFiles method of the Directory class

The program can automatically read the file list to a standard array for further use. There are multiple methods to select files. This method reflects the efficiency of C # And is recommended to you.

Note: The source program is included in this article. You must enter the correct file path during running. If. net is not installed on your machine, you can directly run the compiled execution file. Csdata.exe under the indebug folder.