Sqoop exporting data from a relational library to hive

[Author]: Kwu

Sqoop export data from the relational library to Hive,sqoop supports the number of conditions in the query relational library to the Hive Data Warehouse, and the fields do not need to match the fields in the Hive table.

Specific implementation of the script:

#!/bin/sh  # Upload logs to HDFs  today= ' date--date= ' 0 days ago ' +%y-%m-%d ' sqoop import--connect JDBC:MYSQL://10. 130.2.6:3306/bdc_test--username Lvwenjuan--password Abcd1234  --table sh_compsite_index--where "V1= ' ${today} '"- -columns "v1,v2,v3,v4,v5,v6,v7"--fields-terminated-by ' \001 '--hive-import--hive-drop-import-delims--hive-table Stage.sh_comp_index-m 1year= ' Date +%y '    hive-e '  insert overwrite table Ods.sh_comp_index partition (year= ' ${ Year} ') Select Distincttrans_date,           open_price, Ceil_price  , Close_price, Bottom_price, Volume_of_business, Trans_amountfromstage.sh_comp_indexwhere year (trans_date) =${year};  "

Description

1, conditional statement, need to add double quotation marks: "", as follows:

--where "v1= ' ${today} '"

2. Note that the Sqoop statement needs to be written in a single line of execution.

3,-M 1 description is a map execution.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Sqoop exporting data from a relational library to hive