Hadoop cluster installation and configuration--sqoop installation

1. Sqoop installed on Hadoop.client

2. Duplicate a copy of sqoop-env-template.sh, named sqoop-env.sh

3. Modify the contents of sqoop-env.sh:

Export Hadoop_common_home=/home/hadoopuser/hadoop

Export Hadoop_mapred_home=/home/hadoopuser/hadoop/lib

Export Hive_home=/home/hadoopuser/hive

4. Duplicate a copy of Sqoop-site-template.xml, named Sqoop-site.xml

5. If you do not use the HBase database, you will need to modify the Bin/configure-sqoop file and use the # sign for all of the hbase content.

6. To connect to the SQL Server database, you need to download Sqljdbc.jar and Sqljdbc4.jar and copy the two files to the Sqoop/lib folder

Download Address: http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=21599

Download the tar.gz installation package

7. Common commands:

(1) Import

A. Creating a hive table based on the import table

Bin/sqoop create-hive-table--connect ' jdbc:sqlserver://ip;username= user name; password= password; database= database name '--table table name-- Table name for hive-table Hive

B. Importing data and associating to hive tables

Bin/sqoop import--connect ' jdbc:sqlserver://ip;username= user name; password= password; database= database name '--table table name--hive-import-- Table name for hive-table Hive

(2) Exporting Bin/sqoop export--connect ' jdbc:sqlserver://ip;username= username; password= password; database= database name '--table table name--export-dir /user/hive/warehouse/table name--input-fields-terminated-by ' \001 '