A Java ETL Tool project: jmyetl is launched on Google Code.

In the past, we used the underlying C-API of each database as wrapping to realize the function of data import and export between several heterogeneous databases. However, the code is complex and it is inconvenient to open source.

 

In the afternoon, a simple data extraction program was written in Java to port the MySQL database to Sybase ASE. Put it open-source, put it on: http://code.google.com/p/jmyetl/ top. I originally named myetl, and someone applied for it on sf.net. Then I added a J to it. Java version.

If you are interested, try it out.

 

Currently, it can be a user's full-database data import, or you can choose to import several tables.

It is set to version 0.9.0 because a preliminary frame has just been implemented.

 

I plan to write another ETL job for the database later, and then refactor and combine it.

 

Usage:

The current directory contains an xml configuration file etl_config.xml in the following format:

<? XML version = "1.0" encoding = "UTF-8"?> <Br/> <root> <br/> <source> <br/> <type> mysql </type> <br/> <post> localhost </post> <br /> <port> 3306 </port> <br/> <user> test </user> <br/> <PWD> password1 </pwd> <br/> <DB> test </DB> <br/> <charset> GBK </charset> <br/> <! -- Table is an optional element, if not specified, all the tables will be transfered. --> <br/> <! -- <Br/> <Table> T </table> <br/> <Table> tblob </table> <br/> --> <br/> </source> </P> <p> <target> <br/> <type> ASE </type> <br/> <post> Sean-laptop </post> <br/> <port> 5000 </port> <br/> <user> test </user> <br/> <PWD> password1 </pwd> <br/> <dB> demo </DB> <br/> <charset> eucgb </charset> <br/> <! -- Table element will be ignored in the target element. --> <br/> </Target> <br/> </root> <br/>

 

Directly run myetl. BAT to import data. If the comment of the <Table> block is removed, the whole database is imported.

Note character set options. Without this configuration, garbled characters may easily occur.

 

 

The following restrictions apply:

Do not use database superusers to configure connections because system tables are not filtered in the program currently.

 

Future Plan:

Sybase ASE --> Sybase ASE

Sybase asa --> Sybase ASE

Sybase ASE --> Sybase ASA

 

Oracle --> Sybase ASE

DB2 --> Sybase ASE

SQL Server --> Sybase ASE

 

Reverse import, which will be done later.

 

Existing features:

1. Maintain almost the same table structure and type compatibility

2. the ETL. log file records the detailed import process and possible error information.

3. The target table will have the same primary key structure as the source table

4. The target table creates the same identity information as the source table.

5. the character set consistency configuration will keep the character set in the source table to the target table without garbled characters.

 

Functions to be improved: (the following functions are detailed and are applicable for different purposes. If you only extract data, you do not need)

1. Add the same foreign key constraints as the source table (this is a little complicated and will increase the import time)

2. Add the same index function as the source table, that is, the created index is also displayed on the target table.

 

In fact, the principle previously implemented by C ++ wrapper is similar to that of Java. Compared with JDBC, it can manipulate more underlying APIs and obtain richer information, the amount of code may be large.