Database Migration Tool DATAX-MIGRATION__ Database

datax-migration

datax-migration is yxt (Yunxuetang) Full Database migration Tool based on Alibaba datax 3.0. Support Database Migration among Mysql, Oracle, SQL Server, POSTGRESQL. and support where condition when migration.
datax-migration is the Cloud Academy open source based on the Alibaba Datax 3.0 Database Migration Tool. Supports migration to Mysql,oracle,sqlserver, PostgreSQL, supports migration with where query conditions, and generates migrated data reports. What is Datax?

Datax is a widely used offline data synchronization tool/platform within the Alibaba Group, including MySQL, Oracle, SQL Server, Postgre, HDFS, Hive, ADS, HBase, OTS, ODPs and other heterogeneous data sources of efficient data synchronization functions.
datax is widely used in Alibaba Group, taking on the off-line synchronization business of all large data, and has been running stably for 6 years. Currently completes synchronous 8w multi-channel operation Daily, transmits the data quantity more than 300TB daily.

For more detailed information please see here: https://github.com/alibaba/DataX/wiki/DataX-Introduction Why do you need datax-migration

Datax focuses on data synchronization, which uses scripts and configurable methods to synchronize Tanku data easily with a single script task. But we need a more intelligent or automated way to synchronize the entire database, so we packaged the datax to make it easier to migrate the entire database. the function of Datax-migration

datax-migration can generate a separate datax JSON configuration for these database tables based on the query criteria of the user's metabase tables, then start Datax's script to start the data migration of these tables and generate the corresponding cvs** reports . When the number of tables is too large, you can configure the segmentation strategy to divide multiple threads to make the migration faster and migrate the data.

Support Data Channels

Datax currently has a more comprehensive plug-in system, the main RDBMS database, NOSQL, large data computing systems have been connected to the current support data as shown below, please click: datax Data Source Reference Guide

type	Data Source	Reader (Read)	Writer (write)
RDBMS-relational database	Mysql	√	√
	Oracle	√	√
	Sql server	√	√
	Postgresql	√	√
	Da Meng	√	√
Data storage of Aliyun warehouse	ODPs	√	√
	ADS		√
	Oss	√	√
	OCS	√	√
NoSQL data storage	Ots	√	√
	Hbase0.94	√	√
	Hbase1.1	√	√
	Mongodb	√	√
Unstructured data storage	Txtfile	√	√
	Ftp	√	√
	Hdfs	√	√

Datax-migration at present, the direct use of relational database is basically no problem, as for the non-relational database also need to verify. Quick Start

1. Download [datax can run bin download address] (http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz)
2. Git Clone https://github.com/Jawf/datax-migration.git
3 mvn clean install
4 copy Target/datax-migration.jar and Target/datax-migration_lib to Datax home directory.
5. Open the Datax-migration.jar, edit the Config.properties, config migration db information source/target URL, dbname, US er, password, etc.
6. Open the Datax-migration.jar, edit the Job/jobtemplate.json accordingly, default it migration from MYSQLREADER->MYSQ Lwriter
7. Java-jar Datax-migration.jar

Datax can run the bin download address Special Function

Java-jar Datax-migration.jar JSON #Only generate JSON files config for all tables.
Java-jar datax-migration.jar #Only generate CVS to view migration status.
Java-jar Datax-migration.jar Run #Only run the migration and generate the CVS reports. Before run this, need generate JSON config files to all tables.

More information view GitHub