The chapter of Hadoop Learning: Sqoop installation Configuration

I. Introduction of Sqoop

Sqoop is a tool for transferring data from Hadoop (Hive, HBase), and relational databases, to importing data from a relational database (such as MySQL, Oracle, Postgres, etc.) into Hadoop's HDFs. You can also import HDFs data into a relational database.

Sqoop is now an Apache top-level project, and the current version is 1.4.4 and Sqoop2 1.99.3, this article takes the 1.4.4 version as an example to explain the basic installation configuration and simple application demonstration.

Version is:

Sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz

1. Environment variable Configuration

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/7F/5A/wKiom1cbBH2g6tFkAAAWLuZDIkU875.png "title=" 1.png " alt= "Wkiom1cbbh2g6tfkaaawluzdiku875.png"/>

2.Sqoop parameter Configuration

# Set hadoop-specific Environment variables here. #Set path to where Bin/hadoop is Available#export hadoop_common_home= #Set Path to where Hadoop-*-core.jar are available#export hadoop_mapred_home= #set the path to where Bin/hbase is Available#expo RT hbase_home= #Set the path to where Bin/hive is Available#export hive_home=

3. Drive Jar Package

The following test shows MySQL as an example, you need to copy the MySQL corresponding driver lib file to the <sqoop_home>/lib directory.

Test data in 4.Mysql

create table  ' Demo_blog '   (   ' id '  int (one)  not null auto_increment,    ' blog '  varchar (+)  NOT NULL,  PRIMARY KEY  (' id '))  engine= myisam  default charset=utf8; create table  ' Demo_log '   (   ' operator '  varchar (+)  NOT NULL,    ' log '  varchar ( not null)  ENGINE=MyISAM  DEFAULT CHARSET=utf8; Insert test Data:insert into demo_blog  (id, blog)  values  (1,  "micmiu.com"); insert  into demo_blog  (Id, blog)  values  (2,  "ctosun.com");insert into  demo_blog  (Id, blog)  values  (3,  "baby.micmiu.com"); Insert into demo_log   (Operator, log)  values  ("Micmiu",  "create");insert into demo_log  ( Operator, log)  values  ("Micmiu",  "Update"); Insert into demo_log  (operator, log)  values  ("Michael",  "edit");insert into demo_log  (operator ,  log)  values  ("Michael",  "delete");

Two. Sqoop command operation

1.Sqoop basic Commands

(1) List the databases in MySQL

Sqoop list-databases--connect jdbc:mysql://master-hadoop:3306--username root--password rootroot

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M01/7F/58/wKioL1cbDYmzKJysAABByXqto-c254.png "title=" 1.png " alt= "Wkiol1cbdymzkjysaabbyxqto-c254.png"/>

(2) list All tables in the test database

Sqoop list-databases--connect jdbc:mysql://master-hadoop:3306--username root--password rootroot

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M02/7F/58/wKioL1cbDmSh9LETAAA9jzzdQ9o736.png "title=" 2.png " alt= "Wkiol1cbdmsh9letaaa9jzzdq9o736.png"/>

(3) Import the HDFs file from MySQL

Sqoop Import--connect jdbc:mysql://master-hadoop:3306/test--username root--password rootroot--table demo_log--split -by operator--target-dir/usr/sqoop/other

This article is from the "If you bloom, the breeze came in" blog, please be sure to keep this source http://iqdutao.blog.51cto.com/2597934/1767005

The chapter of Hadoop Learning: Sqoop installation Configuration