Sqoop import--hive-import--hive-overwrite--connect jdbc:oracle:thin:@192.168.92.136:1521:cyporcl--username ODS-- Password ' od154ds$! ('-M 1--hive-database ODS--table q_tra_disputestatus--fields-terminated-by ' \001 '--hive-drop-import-delims--null- String ' \\n '--null-non-string ' \\n '--map-column-java disputeresult=string
Disputeresult This field is Oracleclob type, there is a carriage return, loaded into hive resulting in increased records.
--map-column-java and--map-column-hive two parameters are shown, and you need to set up--hive-drop-import-delims to export the data smoothly.
the imported columns have line breaks to resolve
I thought everything was all right, never thought the imported column has the NCLOB type, which stores an article, bound to have a newline character. Sure enough, hive query data all garbled, query data found--hive-delims-replacement (replace/n,/r and other symbols with your specified characters) and--hive-drop-import-delims parameters. Can be added up to find that there is no egg use, no effect. Baidu Goole A long time, finally found a post, http://stackoverflow.com/questions/28076200/ Hive-drop-import-delims-not-removing-newline-while-using-hcatalog-in-sqoop
it means to use the--map-column-java parameter to display the specified column as a string type. My solution for--map-column-java my clob column =string sure enough, the problem is solved, the line break is all removed. However, I heard that changing the storage format may also avoid this problem, and I hope the great God with experience can answer it. summed up on so much, hoping to help everyone
Sqoop CLOB Import from Oracle to hive carriage return causes record increase