Install and configure sqoop in Ubuntu

You need to use sqoop to import data from the original MySQL database to hbase. The steps and problem records for installing and configuring sqoop are as follows:

1. The project uses hadoop version 1.0.3, so the corresponding sqoop is sqoop-1.4.3.bin _ hadoop-1.0.0, MySQL JDBC is mysql-connector-java-5.1.24

2. Decompress sqoop and configure the environment variables:

Add the following in/etc/profile:

Export $ sqoop_home =/home/hadoop/sqoop-1.4.3.

Export $ Path = $ sqoop_home/bin: $ path

After the configuration is complete, you need to log out or restart

3. Unzip MySQL and put the mysql-connector-java-5.1.24-bin.jar in $ sqoop_home/lib. The configuration is complete.

However, the problem is as follows:

1. Run sqoop in the command line and the prompt is:
Error: cocould not find or load main class org. Apache. sqoop. sqoop

Here sqoop unzip the sqoop-1.4.3.jar under the root directory to add to the hadoop-1.0.3/lib.

2. Run sqoop list-tables -- connect JDBC: mysql: // 192.168.10.111: 3306/database-username 'root'-P to indicate a MySQL error:

13/07/02 10:09:53 info manager. mysqlmanager: preparing to use a MySQL streaming resultset.
13/07/02 10:09:53 error sqoop. sqoop: Got exception running sqoop: Java. Lang. runtimeexception: cocould not load dB Driver Class: COM. MySQL. JDBC. Driver
Java. Lang. runtimeexception: cocould not load dB Driver Class: COM. MySQL. JDBC. Driver
At org. Apache. sqoop. Manager. sqlmanager. makeconnection (sqlmanager. Java: 716)
At org. Apache. sqoop. Manager. genericjdbcmanager. getconnection (genericjdbcmanager. Java: 52)
At org. Apache. sqoop. Manager. catalogquerymanager. listtables (catalogquerymanager. Java: 101)
At org. Apache. sqoop. tool. listtablestool. Run (listtablestool. Java: 49)
At org. Apache. sqoop. sqoop. Run (sqoop. Java: 145)
At org. Apache. hadoop. util. toolrunner. Run (toolrunner. Java: 65)
At org. Apache. sqoop. sqoop. runsqoop (sqoop. Java: 181)
At org. Apache. sqoop. sqoop. runtool (sqoop. Java: 220)
At org. Apache. sqoop. sqoop. runtool (sqoop. Java: 229)
At org. Apache. sqoop. sqoop. Main (sqoop. Java: 238)

All solutions found on the internet say that they did not put the MySQL jar package under $ sqoop_home/lib, But I did put it in. Then I saw a point that hadoop could not find MySQL, and I put the MySQL jar package into the hadoop-1.0.3/lib, and the operation was successful.

No such solution was found on the internet for these two problems. It is unclear if the configuration of my own hadoop is incorrect, which leads to the failure of running the conventional configuration method. After adding sqoop and MySQL jar packages to hadoop/lib, I deleted the MySQL jar package that I put in sqoop-1.4.3/lib, and it can also run normally.