SQOOP2 provides a relational database with HDFs (including hive,hbase) to transmit data to each other.
Available Commands:
CodeGen Generate code to interact with database records
create-hive-table Import a table definition into Hive
Eval Evaluate a SQL statement and display the results
Export Export an HDFS directory to a database table
Help List available Commands
Import Import a table from a database to HDFS
Import-all-tables Import tables from a database to HDFS
Import-mainframe Import datasets from a mainframe server to HDFS
Job Work with saved jobs
list-databases List available databases on a server
List-tables List available tables in a database
Merge Merge Results of Incremental imports
Metastore Run a standalone Sqoop metastore
version Display Version Information
The specific usage is as follows:
List-databases
Sqoop list-databases–connect jdbc:mysql://ip:port/dbname–username ${db_user_name}–password ${DB_PASSWORD}
List-tables
Sqoop list-tables–connect jdbc:mysql://ip:port/dbname–username ${db_user_name}–password ${DB_PASSWORD}
Eval
Sqoop eval–connect jdbc:mysql://ip:port/dbname–username ${db_user_name}–password ${db_password}-e ' select Col_1,col_ 2,col_3 from Tbla limit 20 '
Import
Sqoop import–connect jdbc:mysql://ip:port/dbname–username ${db_user_name}–password ${db_password}–hbase-table ${ Hbase_tbl_name}–hbase-row-key ${field_select_as_your_rowkey}–column-family ${cf_name}–hbase-create-table
A) import MySQL to HBase
Sqoop import–connect jdbc:mysql://ip:port/dbname–username ${db_user_name}–password ${db_password}–hbase-table ${ Hbase_tbl_name}–hbase-row-key ${field_select_as_your_rowkey}–column-family ${hbase_cf_name}–hbase-create-table– Table ${hbase_tbl_name}
b) Import MySQL to HBase (concurrency, fragment read, cast to original data type conversion, query clause, WHERE clause)
Sqoop import–connect jdbc:mysql://ip:port/dbname–username ${db_user_name}–password ${db_password}–hbase-table ${ Hbase_tbl_name}–hbase-row-key ${field_select_as_your_rowkey}–column-family ${hbase_cf_name}–hbase-create-table-m ${parallel_number}–split-by id–query ' Select Id,col2,cast (COL3 as signed) as COL3 from ${rdbms_tbl_name} where $CONDITIO NS '
Note:
- before the option option, the hyphen is actually replaced with a double hyphen character.