Analysis of configuration parameters of Hadoop yarn (3)-mapreduce related parameters

MapReduce related configuration parameters are divided into two parts, jobhistory server and application parameters, and Job history can run on a separate node. Application parameters can be stored as default parameters in Mapred-site.xml, or can be specified separately when the application is submitted, noting that if the user specifies a parameter, the default parameter is overwritten.

The following parameters are all set in Mapred-site.xml.

1.&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; MapReduce jobhistory Related configuration parameters

Configured in the mapred-site.xml of the Jobhistory node.

(1) mapreduce.jobhistory.address

Parameter explanation: MapReduce jobhistory server address.

Default value: 0.0.0.0:10020

(2) Mapreduce.jobhistory.webapp.address

Parameter explanation: MapReduce jobhistory Server Web UI address.

Default value: 0.0.0.0:19888

(3) Mapreduce.jobhistory.intermediate-done-dir

Parameter explanation: The log storage location generated by the MapReduce job.

Default value:/mr-history/tmp

(4) Mapreduce.jobhistory.done-dir

Parameter explanation: The location of the log stored by MR jobhistory Server Management.

Default value:/mr-history/done

2. MapReduce Job Configuration parameters

Can be configured in the client's mapred-site.xml as the default configuration parameter for the MapReduce job. You can also personalize these parameters when the job is submitted.

Note that MRv2 renames all configuration parameters in MRv1, but is compatible with the old parameters in MRv1, except that a warning log is printed to prompt the user parameter to expire. MapReduce the old and new parameter table can refer to Java class Org.apache.hadoop.mapreduce.util.ConfigUtil, for example as follows:

You may also like:

1. Hadoop Yarn Configuration Parameters Analysis (1)-RM and NM related parameters

2. Hadoop Yarn Configuration parameter Analysis (2)-Permissions and log aggregation related parameters