Database Migration Tool DATAX-MIGRATION__ Database

Source: Internet
Author: User
Tags postgresql git clone
datax-migration

datax-migration is yxt (Yunxuetang) Full Database migration Tool based on Alibaba datax 3.0. Support Database Migration among Mysql, Oracle, SQL Server, POSTGRESQL. and support where condition when migration.
datax-migration is the Cloud Academy open source based on the Alibaba Datax 3.0 Database Migration Tool. Supports migration to Mysql,oracle,sqlserver, PostgreSQL, supports migration with where query conditions, and generates migrated data reports. What is Datax?

Datax is a widely used offline data synchronization tool/platform within the Alibaba Group, including MySQL, Oracle, SQL Server, Postgre, HDFS, Hive, ADS, HBase, OTS, ODPs and other heterogeneous data sources of efficient data synchronization functions.
datax is widely used in Alibaba Group, taking on the off-line synchronization business of all large data, and has been running stably for 6 years. Currently completes synchronous 8w multi-channel operation Daily, transmits the data quantity more than 300TB daily.


For more detailed information please see here: https://github.com/alibaba/DataX/wiki/DataX-Introduction Why do you need datax-migration

Datax focuses on data synchronization, which uses scripts and configurable methods to synchronize Tanku data easily with a single script task. But we need a more intelligent or automated way to synchronize the entire database, so we packaged the datax to make it easier to migrate the entire database. the function of Datax-migration

datax-migration can generate a separate datax JSON configuration for these database tables based on the query criteria of the user's metabase tables, then start Datax's script to start the data migration of these tables and generate the corresponding cvs** reports . When the number of tables is too large, you can configure the segmentation strategy to divide multiple threads to make the migration faster and migrate the data.

Support Data Channels

Datax currently has a more comprehensive plug-in system, the main RDBMS database, NOSQL, large data computing systems have been connected to the current support data as shown below, please click: datax Data Source Reference Guide

type Data Source Reader (Read) Writer (write)
RDBMS-relational database Mysql
Oracle
Sql server
Postgresql
Da Meng
Data storage of Aliyun warehouse ODPs
ADS
Oss
OCS
NoSQL data storage Ots
Hbase0.94
Hbase1.1
Mongodb
Unstructured data storage Txtfile
Ftp
Hdfs

Datax-migration at present, the direct use of relational database is basically no problem, as for the non-relational database also need to verify. Quick Start

1. Download [datax can run bin download address] (http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz)
2. Git Clone https://github.com/Jawf/datax-migration.git
3 mvn clean install
4 copy Target/datax-migration.jar and Target/datax-migration_lib to Datax home directory.
5. Open the Datax-migration.jar, edit the Config.properties, config migration db information source/target URL, dbname, US er, password, etc.
6. Open the Datax-migration.jar, edit the Job/jobtemplate.json accordingly, default it migration from MYSQLREADER->MYSQ Lwriter
7. Java-jar Datax-migration.jar

Datax can run the bin download address Special Function

Java-jar Datax-migration.jar JSON #Only generate JSON files config for all tables.
Java-jar datax-migration.jar #Only generate CVS to view migration status.
Java-jar Datax-migration.jar Run #Only run the migration and generate the CVS reports. Before run this, need generate JSON config files to all tables.
More information view GitHub

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.