Install Sqoop Configuration

Sqoop is a tool used to transfer data from Hadoop and relational databases. It can be used to transfer data from a relational database (such as MySQL, Oracle, and S)

Sqoop is a tool used to transfer data from Hadoop and relational databases. It can be used to transfer data from a relational database (such as MySQL, Oracle, and S)

Sqoop is a tool used to transfer data between Hadoop and relational databases. It can import data from a relational database (such as MySQL, Oracle, and S) to HDFS of Hadoop, you can also import HDFS data to a relational database.

Sqoop User Guide address:

1: tar zxvf sqoop-1.1.0.tar.gz

2: Modify the profile/home/hadoopuser/sqoop-1.1.0/conf/sqoop-site.xml

Generally, you only need to modify the following items:

Sqoop. metastore. client. enable. autoconnect
Sqoop. metastore. client. autoconnect. url
Sqoop. metastore. client. autoconnect. username
Sqoop. metastore. client. autoconnect. password
Sqoop. metastore. server. location
Sqoop. metastore. server. port

3:

Bin/sqoop help

Bin/sqoop help import

4:

[Hadoopuser @ master sqoop-1.1.0] $ bin/sqoop import -- connect jdbc: mysql: // localhost/ppc -- table data_ip -- username kwps-P

Enter password:

11/02/18 10:51:58 ERROR sqoop. Sqoop: Got exception running Sqoop: java. lang. RuntimeException: cocould not find appropriate Hadoop shim for 0.20.2

Java. lang. RuntimeException: cocould not find appropriate Hadoop shim for 0.20.2

At com. cloudera. sqoop. shims. ShimLoader. loadShim (ShimLoader. java: 190)

At com. cloudera. sqoop. shims. ShimLoader. getHadoopShim (ShimLoader. java: 109)

At com. cloudera. sqoop. tool. BaseSqoopTool. init (BaseSqoopTool. java: 173)

At com. cloudera. sqoop. tool. ImportTool. init (ImportTool. java: 81)

At com. cloudera. sqoop. tool. ImportTool. run (ImportTool. java: 411)

At com. cloudera. sqoop. Sqoop. run (Sqoop. java: 134)

At org. apache. hadoop. util. ToolRunner. run (ToolRunner. java: 65)

At org. apache. hadoop. util. ToolRunner. run (ToolRunner. java: 79)

At com. cloudera. sqoop. Sqoop. runSqoop (Sqoop. java: 170)

At com. cloudera. sqoop. Sqoop. runTool (Sqoop. java: 196)

At com. cloudera. sqoop. Sqoop. main (Sqoop. java: 205)

Solution:

By default:

./Hadoop-0.20.2/conf/hadoop-env.sh

# Extra Java runtime options. Empty by default.
# Export HADOOP_OPTS =-server

Change:

Export HADOOP_OPTS = "-Djava.net. preferspon4stack = true-Dsqoop. shim. jar. dir =/home/hadoopuser/sqoop-1.1.0/shims"

Note:

Sqoop is currently unavailable in Apache version Hadoop 0.20.2.

Currently, only CDH 3 beta 2 is supported. So if you want to use it, you have to upgrade it to CDH 3 beta 2.

"Sqoop does not run with Apache Hadoop 0.20.2. the only supported platform is CDH 3 beta 2. it requires features of MapReduce not available in the Apache 0.20.2 release of Hadoop. you shoshould upgrade to CDH 3 beta 2 if you want to run Sqoop 1.0.0."

This problem has been marked as a Major Bug by Cloudera. I hope it can be solved as soon as possible.

,