Small Tool for sorting records in text files

I wrote a small tool for a practical problem in my work a few days ago, but it should be said that "the sparrow is small and dirty." It also involves several processes of design, coding, and testing, and involves some interesting knowledge, share it with you.

First, describe the problem to be solved:

Each month, a department in the company downloads a large amount of data on a specific day (usually at the beginning of the month). The data is obtained from the database and a text file is generated, it is then used for subsequent processing, similar to EDI'sProgram. However, subsequent programs require that all records be sorted, and the current program for downloading data is doing this, Directly Writing SQL statements with order, then generate data directly. However, the program runs for a long time, because sorting in the memory of the server consumes a lot of resources. This is not the main problem. The main problem is that data downloading has a great impact on system operations in other departments, and even causes system pauses and no response.

Therefore, when downloading data, I consider not sorting the data. Instead, I take out the sorting work and use a dedicated applet to implement it, in this way, even if performance declines, the machine running the mini-tool program will not be affected by other users in the company. (It was later found that the performance of the local machine was not significantly affected ).

Following this idea, all I need to do is to develop a small tool dedicated to sorting the data. To improve work efficiency, I used VS 2005 for development, although most of the projects in the company use Java.

The first step is how to sort data in text files. In fact, it is very simple. I just used this text file as a data source and configured it in ODBC.

Then, you can configure access to this data source in the Server Manager of VS 2005.

However, there are two problems:

First, for fields such as ID card numbers, the system will use floating point numbers by default. In this way, we can see that the records are in the form of scientific notation. In fact, what I want is the text type.

This problem can be solved by configuring the schema of the data source,

You can specify each field as the text type to facilitate export.

However, the second problem is that the file name must be specified, and the names of the files to be sorted are different each time.

To solve this problem, copy the files to be sorted to this temporary directory and specify a fixed file name, then configure the field type for the fixed file name in this directory. After sorting this file, a sorted text file is generated and copied to the specified target file.

In this way, the technical problems are basically solved. Finally, we have a design for the interface. Let's take a look at the interface first:

Here I have set a tip. After the source file is selected, I will automatically add _ sorted to the end based on the name to serve as the output file name, which is convenient for users.

 

Finally, the following are the keyCode:

 

    //  Display the selected file name in the source file input box  
 Openfiledialog opensourcefile  =     New  Openfiledialog ();
Initializefileopendialog (opensourcefile );

  If  (Opensourcefile. showdialog ()  =  Dialogresult. OK)
{
String filename  =  Opensourcefile. filename;

Txtsourcefile. Text  =  Filename;

 //  If the file name is not blank, the name of the target file is automatically generated.  
    If  (  !  Filename. Trim (). Equals (string. Empty ))
{
  Int  Extendnameposition  =  Filename. indexof (  "  .  " );
String targetfilename  =  Filename. substring (  0  , Extendnameposition)  +     "  _ Sorted  "     +     "  .  "    +  Filename. substring (extendnameposition  +     1  );
Txttargetfile. Text  =  Targetfilename;
}
}
 

 

 

 

This is the sorting operation and corresponding file operations, where the dataaccess class library of Microsoft Enterprise Library 3.1 is used. The read data is directly processed in idatareader, which is much faster than dataset.

   Private     Void  Btnsort_click (  Object  Sender, eventargs E)
{
  If  (Txtsourcefile. Text. Trim (). Equals (string. Empty ))
{
MessageBox. Show (  "  Select the source file first.  "  );
  Return  ;
}

  If (Txttargetfile. Text. Trim (). Equals (string. Empty ))
{
MessageBox. Show (  "  Specify the target file name first.  "  );
  Return  ;
}

  //  First, copy the source data to the specified folder.
  //  Then, use the data in the folder as the data source for operations.  
  Currentjobinfo. Text  =    "  Copying the file to a Temporary Folder ......  "  ;

Copyfiletotemppath ();

  //  Sort data in temporary folders  
  Currentjobinfo. Text  =     "  Sorting ......  "  ;

Database DB  = Databasefactory. createdatabase (  "  Datastore  "  );

String strsql  =  Createsql ();

Dbcommand  =  DB. getsqlstringcommand (strsql );

  //  Prepare the output file  
  String sortedfilename  = Getsortedfilename ();
Fileinfo sortedfile  =     New  Fileinfo (temp_data_path  +  Sortedfilename );

Streamwriter  =  Sortedfile. createtext ();

  //  Write data to a file  
    Using  (Idatareader Dr =  DB. executereader (dbcommand ))
{
  While  (Dr. Read ())
{
Stringbuilder sbrecord  =     New  Stringbuilder ();

  For  (  Int  I  =     0 ; I  <  Dr. fieldcount  -     1  ; I  ++  )
{
Sbrecord. append (Dr [I]. tostring ()  +     "  ,  "  );
}
Sbrecord. append (Dr [DR. fieldcount -     1  ]. Tostring ());

Writer. writeline (sbrecord. tostring ());
}
}

Writer. Close ();

  //  Output the sorted file to the target file.  
  File. Delete (txttargetfile. Text );
File. Copy (temp_data_path  +  Sortedfilename, txttargetfile. Text );

Currentjobinfo. Text  =    "  Done!  "  ;
}
 

 

I hope that the solution to this problem will help you solve similar problems, and I hope you can discuss it together ,:)