Sqoop installation and deployment (note)

Sqoop is a tool that extracts data from relational databases to hadoop. You can also import hive, pig, and other query results to a relational database for storage.
Because the author deploys the hadoop version is 2.2.0, so sqoop version is: sqoop-1.99.3-bin-hadoop200
1. Download sqoop wget http://mirrors.cnnic.cn/apache/sqoop/1.99.3/sqoop-1.99.3-bin-hadoop200.tar.gz
2. Unzip the sqoop Installation File tar-zxvf sqoop-1.99.3-bin-hadoop200.tar.gz
3. Configure sqoop Environment Variables
Export sqoop_home =/home/likehua/sqoop
Export Path = $ sqoop_home/bin: $ path
Modify sqoop -- server references to hadoop shared jar
VI server/CONF/Catalina. Properties
Find the common. loader line and change/usr/lib/hadoop/lib/*. jar to your hadoop jar package directory.
For example:/home/hadoop/share/hadoop/yarn/lib/*. jar,
/Home/hadoop/share/hadoop/yarn/*. jar,
/Home/hadoop/share/hadoop/HDFS/*. jar,
/Home/hadoop/share/hadoop/HDFS/lib/*. jar,
/Home/hadoop/share/hadoop/mapreduce/*. jar,
/Home/hadoop/share/hadoop/mapreduce/lib/*. jar,
/Home/hadoop/share/hadoop/common/lib/*. jar,
/Home/hadoop/share/hadoop/common/*. Jar
VI server/CONF/sqoop. Properties
Find: mapreduce. configuration. directory, and modify the value to your hadoop configuration file directory.
For example:/home/likehua/hadoop/etc/hadoop
Replace @ [email protected] And @ [email protected]:
0, $ S/@ [email protected]/logs/g
0, $ S/@ [email protected]/base/g

Find your database JDBC driver and copy it to the sqoop/lib directory. If it does not exist, create it.

VI/etc/profile

Add the following content:

Export sqoop_home =/home/hadoop/sqoop

Export Path = $ path: $ sqoop_home/bin

Export catalina_base = $ sqoop_home/Server

Export logdir = $ sqoop_home/logs/
 

Start [[email protected] sqoop] #./bin/sqoop. Sh Server start
Sqoop Home Directory:/home/likehua/sqoop
Setting sqoop_http_port: 12000
Setting sqoop_admin_port: 12001
Using catalina_opts:
Adding to catalina_opts:-dsqoop. http. Port = 12000-dsqoop. admin. Port = 12001
Using catalina_base:/home/likehua/sqoop/Server
Using catalina_home:/home/likehua/sqoop/Server
Using catalina_tmpdir:/home/likehua/sqoop/Server/temp
Using jre_home:/usr/local/jdk1.7.0
Using classpath:/home/likehua/sqoop/Server/bin/Bootstrap. Jar
(Sqoop Server is a service program running on Tomcat)
Disable sqoop server:./bin/sqoop. Sh server stop
Use the sqoop client:
[[Email protected] sqoop] # bin/sqoop. Sh Client
Sqoop Home Directory:/home/likehua/sqoop
Sqoop shell: Type 'help' or '\ H' for help.

Sqoop: 000> show version -- all # display version: Show version -- all display connector: Show connector -- all create connection: Create connection -- CID 1

Client Version:
Sqoop 1.99.3 revision 2404394150301df16a94716a3034e31b03e27b0b
Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
Server version:
Sqoop 1.99.3 revision 2404394150301df16a94716a3034e31b03e27b0b
Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
Protocol version:
[1]
Sqoop: 000>