Oozie Frequently Asked Questions

Http://zhangrenhua.com Blog has moved

With the exception information, you can guess that the configuration was not read when the task was executed, so the default 0.0.0.0:8030 address was used. In order to verify whether this is the cause, we can modify the log level of log4j in the oozie/conf directory for debugging.
Then, by viewing and tracking the source code of Hadoop, the correctness of the conjecture is verified, because Oozie will read the configuration in Oozie/conf/hadop-conf/core-site.xml when the job is submitted, we simply add the configuration in this configuration file:
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>

When developing with the above versions of Hadoop and Oozie, when Hadoop is started, processes on the NN node have NAMENODE,RESOURCEMANAGER,DN nodes with Datanode,nodemanager. Hadoop starts up normally, and it's OK to execute your own M/R program on it, but when you use Oozie to submit m/r tasks, the following exception is reported:
caused by:com.google.protobuf.ServiceException: Java.net.ConnectException:Call from slave4/10.95.3.65 to 0.0.0.0:10020 failed on connection exception: Java.net.ConnectException: Deny connection; For more details see: http://wiki.apache.org/hadoop/connectionrefused
 at Org.apache.hadoop.ipc.protobufrpcengine$invoker.invoke (protobufrpcengine.java:212)
 at com.sun.proxy.$ Proxy28.registerapplicationmaster (Unknown Source)
 at Org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster ( amrmprotocolpbclientimpl.java:100)
  .... More

10020 This port is the jobhistory port of Hadoop, for Oozie when submitting the job why to go to connect jobhistory is not very understanding, in Map-site.xml also configured this option:
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>

Later, by debugging and tracking the Hadoop source code, it was found that Oozie actually connected to Jobhistory after the task was submitted.
After opening to debug mode, in Org.apache.hadoop.mapred.ClientCache, this class:
Protected Mrclientprotocol Instantiatehistoryproxy ()
Throws IOException {
Final String serviceaddr = Conf.get (jhadminconfig.mr_history_address);
if (Stringutils.isempty (serviceaddr)) {
return null;
}
Log.debug ("Connecting to Historyserver at:" + serviceaddr);
Final YARNRPC RPC = yarnrpc.create (conf);
Log.debug ("Connected to Historyserver at:" + serviceaddr);
Usergroupinformation CurrentUser = Usergroupinformation.getcurrentuser ();
Return Currentuser.doas (New privilegedaction<mrclientprotocol> () {
@Override
Public Mrclientprotocol Run () {
Return (Mrclientprotocol) rpc.getproxy (Hsclientprotocol.class,
NETUTILS.CREATESOCKETADDR (SERVICEADDR), conf);
}
});
}

With the code above you know that Historyserver is actually connected, so I pass the command
mr-jobhistory-daemon.sh start Historyserver This service to start, so on the NN will start the jobhistoryserver process, the process of the listening port is 10020, I thought this problem can be solved, Unexpectedly in the log log in Oozie still reported the above error, the only possible reason is that Oozie did not read to jobhistory configuration, so I put this configuration:
<property>
<name>mapreduce.jobhistory.address</name>
<value>master:10020</value>
</property>

Again in the Oozie/conf/hadoop-conf/core-site.xml file added again, the above error disappears, the job can also run up normally.

Kill error [Hadoop@hdp-s1 ~]$ oozie Job-oozie http://hdp-m2:11000/oozie-kill 0000000-140806112151829-oozie-hado-c
Error:E0509:E0509:User [?] not authorized for Coord job [0000000-140806112151829-OOZIE-HADO-C] set the following attribute in the Oozie-site.xml file to False