Using Sqoop2 to import MySQL data to HDFS

On SQOOP2 server:
MySQL JDBC Driver:
CP mysql-connector-java-5.1.33/mysql-connector-java-5.1.33-bin.jar/var/lib/sqoop2/

Oracle JDBC Driver:
CP ojdbc6.jar/var/lib/sqoop2/

Microsoft SQL Server JDBC Driver:
CP sqljdbc_4.0/enu/sqljdbc4.jar/var/lib/sqoop2/

Cd/var/lib/sqoop2
Chown sqoop2:sqoop2 Mysql-connector-java-5.1.33-bin.jar Ojdbc6.jar Sqljdbc4.jar
chmod 755 Mysql-connector-java-5.1.33-bin.jar Ojdbc6.jar Sqljdbc4.jar

Restart SQOOP2 Service


On SQOOP2 client:
MySQL JDBC Driver:
CP mysql-connector-java-5.1.33/mysql-connector-java-5.1.33-bin.jar/var/lib/sqoop2/

Oracle JDBC Driver:
CP ojdbc6.jar/var/lib/sqoop2/

Microsoft SQL Server JDBC Driver:
CP sqljdbc_4.0/enu/sqljdbc4.jar/var/lib/sqoop2/

Cd/var/lib/sqoop2
Chown sqoop2:sqoop2 Mysql-connector-java-5.1.33-bin.jar Ojdbc6.jar Sqljdbc4.jar
chmod 755 Mysql-connector-java-5.1.33-bin.jar Ojdbc6.jar Sqljdbc4.jar


Sqoop2(You can also use HUE Web UI > Sqoop transfer)
Sqoop:000> Set Server--host Sqoop2-server-ip
Sqoop:000> Show Connector--all

sqoop:000> Create connection--cid 1
Creating connection for connector with ID 1
Please fill following values to create new Connection object
Name:mysql-conection

Connection Configuration

JDBC Driver Class:Com.mysql.jdbc.Driver
JDBC Connection String:Jdbc:mysql://remote-mysql-server/dbname
Username:username
Password:*******
JDBC Connection Properties:
There is currently 0 values in the map:
entry#

Security Related configuration options

Max Connections:Ten
New connection is successfully created with validation status FINE and persistent ID 1

sqoop:000> Create job--xid 1--type Import
Creating job for connection with ID 1
Please fill following the values to create new Job Object
Name:Export-job1

Database Configuration

Schema Name:dbname
Table Name:tablename
Table SQL Statement:
Table column names:
Partition Column Name:
Nulls in partition column:
Boundary query:

Output Configuration

Storage Type:
0:hdfs
Choose:0
Output format:
0:text_file
1:sequence_file
Choose:0
Compression format:
0:none
1:default
2:deflate
3:gzip
4:bzip2
5:lzo
6:lz4
7:snappy
Choose:0
Output directory:/user/sqoop2/output

Throttling Resources

Extractors:
Loaders:
New job is successfully created with validation status FINE and persistent ID 1

sqoop:000> Start Job--jid 1

sqoop:000> Status Job--jid 1



JDBC Driver Class:com.mysql.jdbc.Driver
JDBC Connection String:jdbc:mysql://remote-mysql-server/dbname


JDBC Driver Class:com.microsoft.sqlserver.jdbc.SQLServerDriver
JDBC Connection String:jdbc:sqlserver://remote-mysql-server/dbname


JDBC Driver Class:oracle.jdbc.driver.OracleDriver
JDBC Connection String:

This article is from the "ilovecat (personal note)" blog, so be sure to keep this source http://hj192837.blog.51cto.com/655995/1570316

Using Sqoop2 to import MySQL data to HDFS