Oozie suspend resolution when submitting a job due to insufficient request resources

1, environmental conditions

Installed CDH5.4, use cm to manage individual components (Yarn, Spark, HDFS, Hive, Oozie ...). ）。

2, submit Oozie job under shell command

①sudo–u HDFs Hadoop fs–mkdir/user/cdhfive creates a dedicated user directory, which is cdhfive

②sudo–u HDFs hadoop–chown–r cdhfive/user/cdhfive Permissions Change

③hadoop fs–put examples/user/cdhfive/Upload the sample file to the Cdhfive user directory on HDFs

④oozie Job-oozie http://localhost:11000/oozie-config examples/apps/map-reduce/job.properties–run submit job to Oozie

⑤oozie job--oozie http://localhost:11000/oozie-info 0000001-151009154615306-oozie-oozi-w Query submitted jobs

3. Problems encountered

① Hadoopaccessorexception exception occurred during commit

There are a number of reasons for this: the Jobtracker port used in Oozie examples is the 8021 (job.properties file), which is the MR1.0 configuration port. If the cluster is configured to use yarn, you need to change the port to 8032

② on the Oozie Web interface, the query job status appears:

Action[[email protected]] Error starting action [Mr-node]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException:Invalid resource request, requested memory < 0, or requested memory > max configured, requestedmemory=1536, maxmemory=1225

At Org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils.validateResourceRequest ( schedulerutils.java:203)

At Org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.validateAndCreateResourceRequest ( rmappmanager.java:377)

At ORG.APACHE.HADOOP.YARN.SERVER.RESOURCEMANAGER.RMAPPMANAGER.CREATEANDPOPULATENEWRMAPP (rmappmanager.java:320)

At Org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication (rmappmanager.java:273)

At Org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication (clientrmservice.java:574)

At Org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication ( applicationclientprotocolpbserviceimpl.java:213)

The reason is that two configuration values in Yarn-site.xml are too small to satisfy the job's request criteria

put Yarn-site.xml two configurations in the add-on:

Container Memory YARN.NODEMANAGER.RESOURCE.MEMORY-MB

Maximum container memory YARN.SCHEDULER.MAXIMUM-ALLOCATION-MB

Oozie suspend resolution when submitting a job due to insufficient request resources