Deployment Installation # Sqoop is a tool for transferring data from Hadoop and relational databases to each other, and can lead data from a relational database (e.g. MySQL, Oracle, Postgres, etc.) into the HDFs of Hadoop. HDFs data can also be directed into a relational database.
# Deploy Sqoop to 13.33, reference documentation: Sqoop installation configuration and demo http://www.micmiu.com/bigdata/sqoop/sqoop-setup-and-demo/
# Sqoop only need to deploy one copy, currently deployed in 13.33, and hadoopmaster a machine.
"MySQL to HDFs" # Test sync MySQL data to hdfs:sqoop import--connect jdbc:mysql://192.168.12.210/zhangsp--username hive--password 123456--table test20141105--warehouse-dir/user/sqoop/12_210--split-by col1
Common parameter Description: # The path imported to HDFs by default is:/user/username/tablename/(files), such as my current user is Hadoop, then the actual path is:/user/hadoop/demo_blog/( files). # If you want to customize the path you need to add parameters:--warehouse-dir, for example:--warehouse-dir/user/sqoop/12_210# No primary key table imports need to add parameters--split-by xxx or-M 1
Oracle to HDFS-View Oracle Data Sheet sqoop list-tables--connect jdbc:oracle:thin:@192.168.12.222:1521:edu--username edu-- password=jyedu
--Sync to an HDFs file Sqoop import--append--connect jdbc:oracle:thin:@192.168.12.222:1521:edu--username edu--password jyedu --target-dir/user/sqoop/12_222_edu/t_ssync_database--num-mappers 1--table t_ssync_database--columns DB_ID,DB_NAME , db_description,db_type,status,create_date--fields-terminated-by ' \ t '
--Sync to multiple HDFs files Sqoop import--append--connect jdbc:oracle:thin:@192.168.12.222:1521:edu--username edu--password jyedu --target-dir/user/sqoop/12_222_edu/t_ssync_database_distribute--num-mappers 5--table t_ssync_database--columns DB _id,db_name,db_description,db_type,status,create_date--fields-terminated-by ' \ t '--split-by DB_ID
"Oracle to HBase" Sqoop oracle2hbasehttp://f.dataguru.cn/forum.php?mod=viewthread&tid=126765
Sqoop import--connect jdbc:oracle:thin:@192.168.12.222:1521:edu--username edu--password jyedu--table t_ssync_ Database--hbase-create-table--hbase-table h_t_ssync_database--column-family cf_database--hbase-row-key db_id-m 1-- Columns Db_id,db_name,db_description,db_type,status,create_date
"Oracle to Hive"
Sqoop import--hive-import--hive-overwrite--hive-table hive_t_ssync_database--connect jdbc:oracle:thin:@ 192.168.12.222:1521:edu--username edu--password jyedu-m 2--split-by db_id--columns db_id,db_name,db_description,db_ Type,status,create_date--table T_ssync_database
Sqoop tool Use (ii)-Import data from Oracle to hivehttp://www.xingxingge.com/hadoop/148.html
Installation and testing of Sqoop
