Sqoop-1.99.7 Installation Deployment

sqoop Installation Deployment

Official documents: http://sqoop.apache.org/

Download Address

[Hadoop@slavenode8 hadoop]$ wget http://apache.fayea.com/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz

Setting environment variables

[Hadoop@slavenode8 sqoop-1.99.7]$ VI ~/.bash_profile

Export sqoop_home=/opt/hadoop/sqoop-1.99.7

Export path= $PATH: $SQOOP _home/bin

Export sqoop_server_extra_lib= $SQOOP _home/extra

Export catalina_base= $SQOOP _home/server

Export logdir= $SQOOP _home/logs/

Install Sqoop

[Hadoop@slavenode8 hadoop]$ tar-zxf sqoop-1.99.7-bin-hadoop200.tar.gz

[Hadoop@slavenode8 hadoop]$ MV sqoop-1.99.7-bin-hadoop200 sqoop-1.99.7

[Hadoop@slavenode8 hadoop]$ CD sqoop-1.99.7

Configuration parameter file

[Hadoop@slavenode2 ~]$ Vi/opt/hadoop/hadoop-2.7.2/etc/hadoop/container-executor.cfg

yarn.nodemanager.linux-container-executor.group= #configured Value of Yarn.nodemanager.linux-container-executor.group

banned.users= #comma separated list of users who can not run applications

Min.user.id=1000#prevent other Super-users

Allowed.system.users=hadoop

VI Core-site.xml

<property>

<name>hadoop.proxyuser.hadoop.hosts</name>

<value>*</value>

</property>

<property>

<name>hadoop.proxyuser.hadoop.groups</name> Run Sqoop user name

<value>*</value>

</property>

[Hadoop@masternode2 sqoop-1.99.7]$ VI conf/sqoop.properties

Org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/opt/hadoop/hadoop-2.7.2/etc/hado

Op

Org.apache.sqoop.security.authentication.type=simple

Org.apache.sqoop.security.authentication.handler= Org.apache.sqoop.security.authentication.SimpleAuthenticationHandler

Org.apache.sqoop.security.authentication.anonymous=true

and replaces @logdir@ and @basedir@:
0,$ s/@LOGDIR @/logs/g
0,$ s/@BASEDIR @/base/g

Create a table of contents

Mkdir/opt/hadoop/sqoop-1.99.7/extra

Mkdir/opt/hadoop/sqoop-1.99.7/logs

MySQL driver jar files are copied to this directory.

CP Mysql-connector-java-5.1.36-bin.jar/opt/hadoop/sqoop-1.99.7/extra

Verify that the configuration is valid

Verify using the Sqoop2-tool tool in Bin:

[Hadoop@masternode2 sqoop-1.99.7]$ Bin/sqoop2-tool Verify

Setting conf dir:/opt/hadoop/sqoop-1.99.7/bin/. /conf

Sqoop Home directory:/opt/hadoop/sqoop-1.99.7

Sqoop Tool Executor:

version:1.99.7

revision:435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb

Compiled on Tue June 16:08:27 PDT 2016 by Abefine

Running Tool:class Org.apache.sqoop.tools.tool.VerifyTool

0 [main] INFO org.apache.sqoop.core.sqoopserver-initializing sqoop server.

7 [main] INFO org.apache.sqoop.core.propertiesconfigurationprovider-starting config file poller thread

Verification was successful.

Tool class Org.apache.sqoop.tools.tool.VerifyTool has finished correctly.

To turn on the server:

[Hadoop@masternode2 sqoop-1.99.7]$ bin/sqoop2-server start

Setting conf dir:/opt/hadoop/sqoop-1.99.7/bin/. /conf

Sqoop Home directory:/opt/hadoop/sqoop-1.99.7

Starting the SQOOP2 server ...

0 [main] INFO org.apache.sqoop.core.sqoopserver-initializing sqoop server.

7 [main] INFO org.apache.sqoop.core.propertiesconfigurationprovider-starting config file poller thread

SQOOP2 server started.