A Java ETL Tool project: jmyetl is launched on Google Code.

Source: Internet
Author: User
Tags sybase

In the past, we used the underlying C-API of each database as wrapping to realize the function of data import and export between several heterogeneous databases. However, the code is complex and it is inconvenient to open source.

 

In the afternoon, a simple data extraction program was written in Java to port the MySQL database to Sybase ASE. Put it open-source, put it on: http://code.google.com/p/jmyetl/ top. I originally named myetl, and someone applied for it on sf.net. Then I added a J to it. Java version.

If you are interested, try it out.

 

Currently, it can be a user's full-database data import, or you can choose to import several tables.

It is set to version 0.9.0 because a preliminary frame has just been implemented.

 

I plan to write another ETL job for the database later, and then refactor and combine it.

 

Usage:

The current directory contains an xml configuration file etl_config.xml in the following format:

<? XML version = "1.0" encoding = "UTF-8"?> <Br/> <root> <br/> <source> <br/> <type> mysql </type> <br/> <post> localhost </post> <br /> <port> 3306 </port> <br/> <user> test </user> <br/> <PWD> password1 </pwd> <br/> <DB> test </DB> <br/> <charset> GBK </charset> <br/> <! -- Table is an optional element, if not specified, all the tables will be transfered. --> <br/> <! -- <Br/> <Table> T </table> <br/> <Table> tblob </table> <br/> --> <br/> </source> </P> <p> <target> <br/> <type> ASE </type> <br/> <post> Sean-laptop </post> <br/> <port> 5000 </port> <br/> <user> test </user> <br/> <PWD> password1 </pwd> <br/> <dB> demo </DB> <br/> <charset> eucgb </charset> <br/> <! -- Table element will be ignored in the target element. --> <br/> </Target> <br/> </root> <br/>

 

Directly run myetl. BAT to import data. If the comment of the <Table> block is removed, the whole database is imported.

Note character set options. Without this configuration, garbled characters may easily occur.

 

 

The following restrictions apply:

Do not use database superusers to configure connections because system tables are not filtered in the program currently.

 

Future Plan:

Sybase ASE --> Sybase ASE

Sybase asa --> Sybase ASE

Sybase ASE --> Sybase ASA

 

Oracle --> Sybase ASE

DB2 --> Sybase ASE

SQL Server --> Sybase ASE

 

Reverse import, which will be done later.

 

Existing features:

1. Maintain almost the same table structure and type compatibility

2. the ETL. log file records the detailed import process and possible error information.

3. The target table will have the same primary key structure as the source table

4. The target table creates the same identity information as the source table.

5. the character set consistency configuration will keep the character set in the source table to the target table without garbled characters.

 

Functions to be improved: (the following functions are detailed and are applicable for different purposes. If you only extract data, you do not need)

1. Add the same foreign key constraints as the source table (this is a little complicated and will increase the import time)

2. Add the same index function as the source table, that is, the created index is also displayed on the target table.

 

In fact, the principle previously implemented by C ++ wrapper is similar to that of Java. Compared with JDBC, it can manipulate more underlying APIs and obtain richer information, the amount of code may be large.

 

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.