This error occurs when Hadoop Mr or spark operates HBase
This is a bug for HBASE and you can see the problem on Jira: https://issues.apache.org/jira/browse/HBASE-10304
Error message:
the/ ,/ - +: -: -ERROR yarn. Applicationmaster: User class threw exception:org. Apache. Hadoop. HBase. Donotretryioexception: Java. Lang. Illegalaccesserror: Classcom. Google. Protobuf. HbasezerocopybytestringCA nnot Access its superclasscom. Google. Protobuf. Literalbytestring org.apache.hadoop.hbase.DoNotRetryIOException:Java. Lang. Illegalaccesserror: Classcom. Google. Protobuf. HbasezerocopybytestringCannot access its superclasscom. Google. Protobuf. Literalbytestringat org. Apache. Hadoop. HBase. Client. Rpcretryingcaller. Translateexception(Rpcretryingcaller. Java: About) at Org. Apache. Hadoop. HBase. Client. Rpcretryingcaller. Callwithretries(Rpcretryingcaller. Java:121) at Org. Apache. Hadoop. HBase. Client. Rpcretryingcaller. Callwithretries(Rpcretryingcaller. Java: -) at Org. Apache. Hadoop. HBase. Client. Clientscanner. Nextscanner(Clientscanner. Java: the) at Org. Apache. Hadoop. HBase. Client. Clientscanner. Initializescannerinconstruction(Clientscanner. Java:169) at Org. Apache. Hadoop. HBase. Client. Clientscanner.<init> (Clientscanner. Java:164) at Org. Apache. Hadoop. HBase. Client. Clientscanner.<init> (Clientscanner. Java:107) at Org. Apache. Hadoop. HBase. Client. Htable. Getscanner(htable. Java:736) at Org. Apache. Hadoop. HBase. Client. Metascanner. Metascan(Metascanner. Java:178) at Org. Apache. Hadoop. HBase. Client. Metascanner. Metascan(Metascanner. Java: the) at Org. Apache. Hadoop. HBase. Client. Hconnectionmanager$HConnectionImplementation. Istableavailable(Hconnectionmanager. Java:962) at Org. Apache. Hadoop. HBase. Client. Hbaseadmin. Istableavailable(Hbaseadmin. Java:1081) at Org. Apache. Hadoop. HBase. Client. Hbaseadmin. Istableavailable(Hbaseadmin. Java:1089) atcom. Umeng. DP. Yuliang. Play. Hbasetoes$. Main(hbasetoes. Scala: -) atcom. Umeng. DP. Yuliang. Play. Hbasetoes. Main(hbasetoes. Scala) at Sun. Reflect. Nativemethodaccessorimpl. Invoke0 (Native Method) at Sun. Reflect. Nativemethodaccessorimpl. Invoke(Nativemethodaccessorimpl. Java: $) at Sun. Reflect. Delegatingmethodaccessorimpl. Invoke(Delegatingmethodaccessorimpl. Java: +) at Java. Lang. Reflect. Method. Invoke(Method. Java:606) at Org. Apache. Spark. Deploy. Yarn. Applicationmaster$ $anon $. Run(Applicationmaster. Scala:483) caused By:java. Lang. Illegalaccesserror: Classcom. Google. Protobuf. HbasezerocopybytestringCannot access its superclasscom. Google. Protobuf. LiteralbytestringAt Java. Lang. ClassLoader. DefineClass1 (Native Method) at Java. Lang. ClassLoader. DefineClass(ClassLoader. Java: -) at Java. Security. Secureclassloader. DefineClass(Secureclassloader. Java:142) at Java. NET. URLClassLoader. DefineClass(URLClassLoader. Java:449) at Java. NET. URLClassLoader. Access$ (URLClassLoader. Java: in) at Java. NET. URLClassLoader$. Run(URLClassLoader. Java:361) at Java. NET. URLClassLoader$. Run(URLClassLoader. Java:355) at Java. Security. Accesscontroller. doprivileged(Native Method) at Java. NET. URLClassLoader. Findclass(URLClassLoader. Java:354) at Java. Lang. ClassLoader. LoadClass(ClassLoader. Java:425) at Java. Lang. ClassLoader. LoadClass(ClassLoader. Java:358) at Org. Apache. Hadoop. HBase. Protobuf. Requestconverter. Buildregionspecifier(Requestconverter. Java:930) at Org. Apache. Hadoop. HBase. Protobuf. Requestconverter. Buildscanrequest(Requestconverter. Java:434) at Org. Apache. Hadoop. HBase. Client. Scannercallable. Openscanner(scannercallable. Java:297) at Org. Apache. Hadoop. HBase. Client. Scannercallable. Call(scannercallable. Java:157) at Org. Apache. Hadoop. HBase. Client. Scannercallable. Call(scannercallable. Java: $) at Org. Apache. Hadoop. HBase. Client. Rpcretryingcaller. Callwithretries(Rpcretryingcaller. Java: the) ... -More
Hadoop Yarn Solutions:
-
Submit Job method
$ export Hadoop_classpath="/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar " $./hadoop-2.2 .0 /bin/hadoop--config/home/stack/conf_hadoop/jar. Hbase/hbase-assembly/target/hbase-0.99 .0 - Snapshot-job.jar org.apache .hadoop .hbase .mapreduce usertable
-
Add hadoop_classpath to Linux environment variable
Add the following to BASHRC or Bash_ Profile or profile, this is the
Export hadoop_classpath= $HADOOP _classpath:/home/cluster/apps/hbase/lib/in the Linux environment variable. Hbase-protocol-0.98.1-cdh5.1.0.jar
Spark Solutions:
Submit Job Method
-conf Add Spark.driver.extraClassPath & Spark.executor.extraClassPath
Spark-submit--class com.Umeng.DP.Yuliang.Play.hbasetoes--Master Yarn-Cluster--conf"Spark.Driver.Extraclasspath=/Home/Cluster/Apps/HBase/Lib/HBase-Protocol-0.98.1-Cdh5.1.0.Jar" --conf"Spark.Executor.Extraclasspath=/Home/Cluster/Apps/HBase/Lib/HBase-Protocol-0.98.1-Cdh5.1.0.Jar" --Jars/Home/Cluster/Apps/HBase/Lib/HBase-Protocol-0.98.1-Cdh5.1.0.Jar Scalamr-0.0.1-Jar- with-Dependencies.Jar
-
Add the following configuration to the $spark_home/conf/spark-defaults.conf file
Spark.driver .extraclasspath /Home /cluster/apps/hbase/lib/hbase-protocol-0.98 .1 -cdh5.1 .0 .jar spark.executor .extraclasspath /home/cluster/apps/hbase/lib/hbase-protocol- 0.98 .1 -cdh5.1 .0 .jar
Spark Maillist:
http://apache-spark-user-list.1001560.n3.nabble.com/ Java-lang-illegalaccesserror-class-com-google-protobuf-hbasezerocopybytestring-cannot-access-its-supg-tc24303.html
Respect the original, without permission not reproduced:
http://blog.csdn.net/stark_summer/article/details/47750017
Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.
Java.lang.IllegalAccessError:class com.google.protobuf.HBaseZeroCopyByteString