Sqoop安裝配置教程

1.安裝部署
（1）、下載地址：http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.5.2.tar.gz
解壓到／home/neil/Downloads/hadoop-2.7.3/sqoop-1.4.6
（2）、拷貝MySQL的jdbc驅動包mysql-connector-java-5.1.31-bin.jar到sqoop/lib目錄下。
（3）、配置環境變數 sqoop

export SQOOP_HOME=／home/neil/Downloads/hadoop-2.7.3/sqoop-1.4.6
export PATH= PATH: PATH:SQOOP_HOME/bin
（4）、複製sqoop/conf/sqoop-env-template.sh為sqoop-env.sh
添加相關的配置 Setpath to where bin/hadoop is available

exportHADOOP_COMMON_HOME=／home/neil/Downloads/hadoop-2.7.3 Setpath to where hadoop-*-core.jar isavailable

exportHADOOP_MAPRED_HOME=／home/neil/Downloads/hadoop-2.7.3 setthe path to where bin/hbase isavailable

exportHBASE_HOME=/opt/cdh5/hbase-1.1.8 Setthe path to where bin/hive is available

exportHIVE_HOME= ／home/neil/Downloads/hadoop-2.7.3/hive-1.2.1 Setthe path for where zookeper config diris

exportZOOCFGDIR= ／home/neil/Downloads/hadoop-2.7.3/zookeeper-3.4.5
（5）測試Sqoop version
發現有警告
修改$SQOOP_HOME/bin/configure-sqoop
注釋掉HCatalog，Accumulo檢查(除非你準備使用HCatalog，Accumulo等HADOOP上的組件) Moved to be a runtime check in sqoop. if [ ! -d “${HCAT_HOME}” ]; then echo “Warning: $HCAT_HOME does not exist! HCatalog jobs willfail.” echo ‘Please set $HCAT_HOME to the root of your HCatalog installation.’ fi if [ ! -d “${ACCUMULO_HOME}” ];then echo “Warning: $ACCUMULO_HOME does not exist! Accumulo imports willfail.” echo ‘Please set $ACCUMULO_HOME to the root of your Accumuloinstallation.’ fi

再次執行sqoop version
你也可以查看某一個命令的使用說明：
sqoopimport–help sqoopimport –help sqoophelp import

sqoop import 的一個樣本如下：
登入MYSQL
1.mysql -u root -p

2.建立資料庫hadoopguide;
create database hadoopguide;

use hadoopguide;

3.建立表
create table widgets(id int not null primary key auto_increment,widget_name varchar(64) not null,price decimal(10,2),design_date date,version int,design_comment varchar(100));

4.插入幾條記錄
insert into widgets values(null,’sprocket’,0.25,’2010-02-10’,1,’connects two gizmos’);
insert into widgets values(null,’gizmo’,4.00,’2009-02-10’,1,null);
insert into widgets values(null,’sprocket’,99.99,’2011-02-10’,1,’Our flagship product’);

5.退出
quit

6.使用sqoop匯入這個表資料到HDFS

$ sqoop import –connect jdbc:mysql://localhost:3306/hadoopguide –username root –password neil –table widgets

你還可以使用 –options-file 來傳入一個檔案，使用這種方式可以重用一些配置參數：
$ sqoop –options-file /users/homer/work/import.txt –table TEST
/users/homer/work/import.txt 檔案內容如下：
import
–connect
jdbc:mysql://192.168.56.121:3306/metastore
–username
hiveuser
–password
redhat