Zerocopyliteralbytestring Cannot access superclass

Problem description

When running a mapreduce job on HBase, the following exception is reported: Illegalaccesserror:class com.google.protobuf.HBaseZeroCopyByteString cannot access its Superclass com.google.protobuf.LiteralByteString

Use the HBase environment as follows: CDH5.0.1, hbase version: 0.96.1

Cause of the problem

This isssue occurs because of the optimization introduced in HBASE-9867 that inadvertently introduced a classloader Depende Ncy. This affects both jobs using the-libjars option and "Fat jar," the jobs which package their runtime dependencies in a nested Lib folder.

This problem occurred due to the optimization of HBASE-9867, inadvertently introduced a dependency class loader. It affects jobs that use the-libjars parameter and two modes using the Fat jar.

A special feature of the Fat Jar mode Hadoop is the ability to read jar files for all libraries contained in the/lib directory in the operating directory, placing the jar that runs the job dependency in the Lib directory in the Jar.

Solution Solutions

1. Permanent Solution

Cd/opt/cloudera/parcels/cdh-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop

Ln-s/opt/cloudera/parcels/cdh-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar Hbase-protocol-0.96.1.1-cdh5.0.0.jar

2: Temporary Solution

hadoop_classpath=/opt/cloudera/parcels/cdh-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/ hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf Hadoop jar Myjar.jar Myjobmainclass

Detailed error information

14/07/16 14:40:50 WARN Client. hconnectionmanager$hconnectionimplementation:encountered problems when prefetch Hbase:meta table:
Org.apache.hadoop.hbase.client.RetriesExhaustedException:Failed after attempts=10, exceptions:
Wed Jul 14:32:49 CST, [email protected],Java.lang.IllegalAccessError:class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass Com.google.protobuf.LiteralByteString
Wed Jul 14:32:52 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:32:57 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:33:07 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:33:28 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:34:08 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:35:49 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:37:29 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:39:10 CST, [email protected], java.lang.illegalaccesserror:com/google/protobuf/ Hbasezerocopybytestring
Wed Jul 14:40:50 CST, [email protected],java.lang.illegalaccesserror:com/google/protobuf/hbasezerocopybytestring


At Org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries (rpcretryingcaller.java:134)
At Org.apache.hadoop.hbase.client.HTable.getRowOrBefore (htable.java:705)
At Org.apache.hadoop.hbase.client.MetaScanner.metaScan (metascanner.java:144)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.prefetchregioncache ( HCONNECTIONMANAGER.JAVA:1102)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.locateregioninmeta ( hconnectionmanager.java:1162)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.locateregion ( hconnectionmanager.java:1054)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.locateregion ( hconnectionmanager.java:1011)
At Org.apache.hadoop.hbase.client.HTable.finishSetup (htable.java:326)
At Org.apache.hadoop.hbase.client.htable.<init> (htable.java:192)
At Org.apache.hadoop.hbase.client.htable.<init> (htable.java:150)
At org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf (tableoutputformat.java:206)
At org.apache.hadoop.util.ReflectionUtils.setConf (reflectionutils.java:73)
At Org.apache.hadoop.util.ReflectionUtils.newInstance (reflectionutils.java:133)
At Org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs (jobsubmitter.java:455)
At Org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal (jobsubmitter.java:343)
At Org.apache.hadoop.mapreduce.job$10.run (job.java:1295)
At Org.apache.hadoop.mapreduce.job$10.run (job.java:1292)
At java.security.AccessController.doPrivileged (Native Method)
At Javax.security.auth.Subject.doAs (subject.java:415)
At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1548)
At Org.apache.hadoop.mapreduce.Job.submit (job.java:1292)
At Org.apache.hadoop.mapreduce.Job.waitForCompletion (job.java:1313)
At MapReduce. Importfromfile.run (importfromfile.java:228)
At Org.apache.hadoop.util.ToolRunner.run (toolrunner.java:70)
At Org.apache.hadoop.util.ToolRunner.run (toolrunner.java:84)
At MapReduce. Importfromfile.main (importfromfile.java:185)
At Sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
At Sun.reflect.NativeMethodAccessorImpl.invoke (nativemethodaccessorimpl.java:57)
At Sun.reflect.DelegatingMethodAccessorImpl.invoke (delegatingmethodaccessorimpl.java:43)
At Java.lang.reflect.Method.invoke (method.java:606)
At Org.apache.hadoop.util.programdriver$programdescription.invoke (programdriver.java:72)
At Org.apache.hadoop.util.ProgramDriver.run (programdriver.java:144)
At Org.apache.hadoop.util.ProgramDriver.driver (programdriver.java:152)
At MapReduce. Driver.main (driver.java:29)
At Sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
At Sun.reflect.NativeMethodAccessorImpl.invoke (nativemethodaccessorimpl.java:57)
At Sun.reflect.DelegatingMethodAccessorImpl.invoke (delegatingmethodaccessorimpl.java:43)
At Java.lang.reflect.Method.invoke (method.java:606)
At Org.apache.hadoop.util.RunJar.main (runjar.java:212)
caused by:java.lang.illegalaccesserror:com/google/protobuf/hbasezerocopybytestring
At Org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier (requestconverter.java:897)
At Org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest (requestconverter.java:131)
At Org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore (protobufutil.java:1402)
At Org.apache.hadoop.hbase.client.htable$2.call (htable.java:701)
At Org.apache.hadoop.hbase.client.htable$2.call (htable.java:699)
At Org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries (rpcretryingcaller.java:120)
... More
Exception in thread "main" java.lang.illegalaccesserror:com/google/protobuf/hbasezerocopybytestring
At Org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier (requestconverter.java:897)
At Org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest (requestconverter.java:131)
At Org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore (protobufutil.java:1402)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.locateregioninmeta ( hconnectionmanager.java:1176)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.locateregion ( hconnectionmanager.java:1054)
At Org.apache.hadoop.hbase.client.hconnectionmanager$hconnectionimplementation.locateregion ( hconnectionmanager.java:1011)
At Org.apache.hadoop.hbase.client.HTable.finishSetup (htable.java:326)
At Org.apache.hadoop.hbase.client.htable.<init> (htable.java:192)
At Org.apache.hadoop.hbase.client.htable.<init> (htable.java:150)
At org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf (tableoutputformat.java:206)
At org.apache.hadoop.util.ReflectionUtils.setConf (reflectionutils.java:73)
At Org.apache.hadoop.util.ReflectionUtils.newInstance (reflectionutils.java:133)
At Org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs (jobsubmitter.java:455)
At Org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal (jobsubmitter.java:343)
At Org.apache.hadoop.mapreduce.job$10.run (job.java:1295)
At Org.apache.hadoop.mapreduce.job$10.run (job.java:1292)
At java.security.AccessController.doPrivileged (Native Method)
At Javax.security.auth.Subject.doAs (subject.java:415)
At Org.apache.hadoop.security.UserGroupInformation.doAs (usergroupinformation.java:1548)
At Org.apache.hadoop.mapreduce.Job.submit (job.java:1292)
At Org.apache.hadoop.mapreduce.Job.waitForCompletion (job.java:1313)
At MapReduce. Importfromfile.run (importfromfile.java:228)
At Org.apache.hadoop.util.ToolRunner.run (toolrunner.java:70)
At Org.apache.hadoop.util.ToolRunner.run (toolrunner.java:84)
At MapReduce. Importfromfile.main (importfromfile.java:185)
At Sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
At Sun.reflect.NativeMethodAccessorImpl.invoke (nativemethodaccessorimpl.java:57)
At Sun.reflect.DelegatingMethodAccessorImpl.invoke (delegatingmethodaccessorimpl.java:43)
At Java.lang.reflect.Method.invoke (method.java:606)
At Org.apache.hadoop.util.programdriver$programdescription.invoke (programdriver.java:72)
At Org.apache.hadoop.util.ProgramDriver.run (programdriver.java:144)
At Org.apache.hadoop.util.ProgramDriver.driver (programdriver.java:152)
At MapReduce. Driver.main (driver.java:29)
At Sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
At Sun.reflect.NativeMethodAccessorImpl.invoke (nativemethodaccessorimpl.java:57)
At Sun.reflect.DelegatingMethodAccessorImpl.invoke (delegatingmethodaccessorimpl.java:43)
At Java.lang.reflect.Method.invoke (method.java:606)
At Org.apache.hadoop.util.RunJar.main (runjar.java:212)

Reference

https://issues.apache.org/jira/browse/HBASE-11118

Http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.9.0/bk_releasenotes_hdp_2.0/content/ch_relnotes-hdpch_ Relnotes-hdp-2.0.9.0-knownissues-hbase.html

http://my.oschina.net/cloudcoder/blog/291486

Zerocopyliteralbytestring Cannot access superclass