Installation and introduction of Chao Wu teacher course--sqoop

Sqoop is used to import and export data.
(1) Import data from databases such as MySQL, Oracle, etc. into HDFs, Hive, HBase
(2) Export data from HDFs, Hive and hbase to databases such as MySQL, Oracle, etc.

One: Installation steps
1. Unzip, set environment variable, slightly
2. Put the Mysql.............jar in the $sqoop_home/lib

Second: Copy the table from MySQL to hdfs/hive
Sqoop # #sqoop命令
Import # #表示导入
--connect jdbc:mysql://ip:3306/sqoop # #告诉jdbc, link mysql URL
--username Root # #连接mysql的用户名
--password Admin # #连接mysql的密码
--table MYSQL1 # #从mysql导出的表名称
--fields-terminated-by ' \ t ' # #指定输出文件中的行的字段分隔符
-M 1 # #复制过程使用1个map作业
--hive-import # #把mysql表数据复制到hive空间中. If you do not use this option, it means that you are copying to HDFs

Three: Copy table data from hive to MySQL
Sqoop
Export # #表示数据从hive复制到mysql中
--connect Jdbc:mysql://ip:3306/sqoop
--username Root
--password Admin
--table MYSQL2 # #mysql中的表, the table name that will be imported
--export-dir '/USER/ROOT/WAREHOUSE/MYSQL1 ' # #hive中被导出的文件目录
--fields-terminated-by ' \ t ' # #hive中被导出的文件字段的分隔符
Note: MYSQL2 must exist

Four: Perform the action specifically
1. Turn data from MySQLImport to HDFs(Default is/user/<username>)
Sqoop Import--connect jdbc:mysql://hadoop:3306/hive--username root--password 465213a--table tbls--fields-terminated -by ' \ t '--null-string ' * * '-M 1--append--hive-import

Sqoop Import--connect jdbc:mysql://hadoop:3306/hive--username root--password admin--table tbls--fields-terminated-b Y ' \ t '--null-string ' * * '-M 1--append--hive-import--check-column ' tbl_id '--incremental append--last-value 6

2. Export data from HDFs to MySQL
Sqoop export--connect jdbc:mysql://hadoop0:3306/hive--username root--password admin--table IDs--fields-terminated-b Y ' \ t '--export-dir '/ids '

3. Set as job, run job
Sqoop Job--create myjob--Import--connect jdbc:mysql://hadoop0:3306/hive--username root--password admin--table tbls --fields-terminated-by ' \ t '--null-string ' * * '-M 1--append--hive-import

4. Import the exported transactions in the Mapper task Unit .

Installation and introduction of Chao Wu teacher course--sqoop