. Net for data extraction, conversion, and loading

See an article in the morningArticle:

Http://www.codeproject.com/Articles/34556/Write-ETL-jobs-in-pure-C

 

There are two files:User name file (ID name ),Addresses file (ID address ):

 
ID name
 

 
1 Bob

 

 
Id address1 123 Main st.2 42 everywhich way
If you want
 

 
ID name address can be obtained in a query in SQL. In the process of File Processing, what should I do based on C?CodeThe solution is provided.

First, read the content of the two files:
 

 
Fileengine is based on Filehelpers
 

 Public   Class Usernameread: abstractoperation { Public Usernameread ( String Filepath ){This . Filepath = filepath ;} String Filepath = Null ; Public   Override Ienumerable <row> execute (ienumerable <row> rows ){ Using (Fileengine file = fluentfile. For <usernamerecord> (). From (filepath )){ Foreach ( Object OBJ In File ){ Yield   Return Row. fromobject (OBJ );}}}} Public   Class Useraddressread: abstractoperation { Public Useraddressread ( String Filepath ){ This . Filepath = filepath ;} String Filepath = Null ; Public   Override Ienumerable <row> execute (ienumerable <row> rows ){ Using (Fileengine file = fluentfile. For <useraddressrecord> (). From (filepath )){ Foreach ( Object OBJ In File ){ Yield   Return Row. fromobject (OBJ );}}}}

Create the relationship between the two files and create a new file

 Public   Class Joinuserrecords: joinoperation { Protected   Override   Void Setupjoinconditions () {innerjoin. Left ("  ID" ). Right ( "  ID" );} Protected   Override Row mergerows (row leftrow, row rightrow) {row = New Row (); row. Copy (leftrow ); //  Copy over all properties not in the user records Row [ "  Address" ] = Rightrow [ "  Address" ];Return Row ;}}


How to output the created structure:

 Public   Class Userfullwrite: abstractoperation { Public Userfullwrite ( String Filepath ){ This . Filepath = filepath ;} String Filepath = Null ; Public   Override Ienumerable <row> execute (ienumerable <row> rows) {fluentfile engine = fluentfile. For <userfullrecord> (); Engine. headertext = "  ID \ tname \ taddress" ; Using (Fileengine file = engine. To (filepath )){ Foreach (Row row In Rows) {file. Write (row. toobject <userfullrecord> ()); //  Pass through rows if needed for another later operation                  Yield   Return Row ;}}}}

Call method:

 
Public ClassMainprocess: etlprocess {Protected Override VoidInitialize () {register (NewJoinuserrecords (). Left (NewUsernameread (settings. Default. namesfile). Right (NewUseraddressread (settings. Default. addressesfile); Register (NewUserfullwrite (settings. Default. outputfile ));}}

Summary: structured files are usually better processed, but they are not easy to process.