Objective:
use Sqoop to import data from Oracle into HBase and automatically generate composite row keys !
Environment:Hadoop2.2.0hbase0.96sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gzoracle11gjdk1.7ubuntu14 Server here about the environment Spit groove Sentence: The latest version of the Sqoop1.99.3 function is too weak, only support import data to HDFs, no other option, too dirt! (If you have a different opinion, please discuss the solution.)
command:
sqoop Import--connect jdbc:oracle:thin:@192.168.0.147:1521:orclgbk--username zhaobiao--p--table CMS_NEWS_0625- -hbase-create-table--hbase-table 147patents--column-family patentinfo--split-by create_time--hbase-row-key "Create _time,publish_time,title "
Note the points:
the table name of 1.Oracle must be capitalized (--table cms_news_0625) ;
2. The username must be in capital letters (--username Zhaobiao);
3. The field names in the combined row key parameter must be capitalized ( --hbase-row-key "Create_time,publish_time,operate_time,title");
4. There can be no null value for several fields as a composite row key, otherwise an error will be found, please confirm before executing the statement.
Exception ResolutionError encountered during: Error:java.io.IOException:Could not insert row with null value for Row-key column:operate_time &NB Sp at Org.apache.sqoop.hbase.ToStringPutTransformer.getPutCommand (tostringputtransformer.java:125) at org.apache.sqoop.hbase.HBasePutProcessor.accept (hbaseputprocessor.java:142) at Org.apache.sqoop.mapreduce.delegatingoutputformat$delegatingrecordwriter.write ( delegatingoutputformat.java:128) at org.apache.sqoop.mapreduce.delegatingoutputformat$ Delegatingrecordwriter.write (delegatingoutputformat.java:92) at Org.apache.hadoop.mapred.maptask$newdirectoutputcollector.write (maptask.java:634) at Org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write (taskinputoutputcontextimpl.java:89) at Org.apache.hadoop.mapreduce.lib.map.wrappedmapper$context.write (wrappedmapper.java:112) at Org.apache.sqoop.mapreduce.HBaseImportMapper.map (hbaseimportmapper.java:38) at Org.apache.sqoop.mapreduce.HBaseImportMapper.map (hbaseimportmapper.java:31) at Org.apache.hadoop.mapreduce.Mapper.run (mapper.java:145) at Org.apache.sqoop.mapreduce.AutoProgressMapper.run (autoprogressmapper.java:64) at Org.apache.hadoop.mapred.MapTask.runNewMapper (maptask.java:763) at Org.apache.hadoop.mapred.MapTask.run (maptask.java:339) at Org.apache.hadoop.mapred.yarnchild$2.run (yarnchild.java:162) at Java.security.AccessController.doPrivileged (Native Method) at Javax.security.auth.Subject.doAs (subject.java:415) at Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1491) at Org.apache.hadoop.mapred.YarnChild.main (yarnchild.java:157)
cause 1:--hbase-row-key "create_time,publish_time,operate_time,title" field name is lowercase, you need to change it to uppercase--hbase-row-key "Create_time, Publish_time,title "
Cause 2:
The field does exist with a null value in the original table.
