Sqoop Tool Introduction (HDFS and relational database for data import and export)

Data Sheet

First class: Data in the database is imported into HDFs

#数据库驱动jar包用mysql-connector-java-5.1. to-bin, otherwise there may be an error!./sqoop Import--connect Jdbc:mysql://localhost:3306/erpdb--username root--password 123456--table tbl_dep--columns ' uuid, name, Tele ': <<!Output: part-m-00000:1, President of the Office,8888 　　　　　　 2, purchasing department,6668 　　　　　　 3, Sales department,6888 Part-m-00001:4, Transport Center,3434 　　　　　　 5, Library tube Center,5666 　　　　　　 6Human resources1234 Part-m-00002:7, Finance Department,9999 Part-m-00003: -,222,222!

#指定输出路径, specifies the data delimiter./sqoop Import--connect Jdbc:mysql://localhost:3306/erpdb--username root--password 123456--table tbl_dep--target-dir '/sqoop/td '-- Fields-terminated-by ' \ t ': <<!Output: part-m-00000:1President's Office8888 　　　　　　 2Purchasing Department6668　　　　　　 3Sales Department6888 Part-m-00001:4Transportation Center3434 　　　　　　 5The center of Library and Tube5666 　　　　　　 6Human resources1234 Part-m-00002:7Finance Department9999 Part-m-00003: -    222    222!

#指定Map数量-m./sqoop Import--connect Jdbc:mysql://localhost:3306/erpdb--username root--password 123456--table tbl_dep--target-dir '/sqoop/td1 '-- Fields-terminated-by ' t '-M 2: <<!Output: part-m-00000:1President's Office8888 　　　　　　 2Purchasing Department6668 　　　　　　 3Sales Department6888 　　　　　　 4Transportation Center3434 　　　　　　 5The center of Library and Tube5666 　　　　　　 6Human resources1234 Part-m-00001:7Finance Department9999 　　　　　　  -    222    222!

#增加where条件, note: The condition must be quoted./sqoop Import--connect Jdbc:mysql://localhost:3306/erpdb--username root--password 123456--table tbl_dep--where ' uuid>3 '--target-dir '/SQOOP/TD2 ': <<!Output: part-m-00000:4Transportation Center3434 　　　　　　 5The center of Library and Tube5666 　　　　　　 6Human resources1234 Part-m-00001:7Finance Department9999 Part-m-00002: part-m-00003: -    222    222!

#增加query语句 (You can wrap the statement using \)./sqoop Import--connect Jdbc:mysql://localhost:3306/erpdb--username root--password 123456--query ' SELECT * from TBL_DEP where UUID > 2 and $CONDITION S '--split-by tbl_dep.uuid--target-dir '/sqoop/td3 ': <<!Output: part-m-00000:3Sales Department6888 　　　　　　 4Transportation Center3434 　　　　　　 5The center of Library and Tube5666 Part-m-00001:6Human resources1234 　　　　　　 7Finance Department9999 Part-m-00002: part-m-00003: -    222    222Note: If you use--query This command, you need to be aware of the parameters in the where, and $CONDITIONS This parameter must be added, and there is a single quotation mark and double quotation marks, if--query is used after the double quotation marks, then need to precede the $conditions \ $CONDITIONS, if you set the number of maps to 1-m1, without adding--Split-by ${tablename.column}, otherwise you need to add!

Type II: Export data from HDFs to the database (do not forget to specify delimiters)

./sqoop Export--connect jdbc:mysql://localhost:3306/erpdb--username root--password 123456--export-dir '/sqoop/td3 '--table aaa-m 1--fields-terminated-by ', '

Sqoop Tool Introduction (HDFS and relational database for data import and export)