Analysis of configuration parameters of Hadoop yarn (3)-mapreduce related parameters

Source: Internet
Author: User
Keywords Profiling nbsp; default value

MapReduce related configuration parameters are divided into two parts, jobhistory server and application parameters, and Job history can run on a separate node. Application parameters can be stored as default parameters in Mapred-site.xml, or can be specified separately when the application is submitted, noting that if the user specifies a parameter, the default parameter is overwritten.

The following parameters are all set in Mapred-site.xml.

1.&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; MapReduce jobhistory Related configuration parameters

Configured in the mapred-site.xml of the Jobhistory node.

(1) mapreduce.jobhistory.address

Parameter explanation: MapReduce jobhistory server address.

Default value: 0.0.0.0:10020

(2) Mapreduce.jobhistory.webapp.address

Parameter explanation: MapReduce jobhistory Server Web UI address.

Default value: 0.0.0.0:19888

(3) Mapreduce.jobhistory.intermediate-done-dir

Parameter explanation: The log storage location generated by the MapReduce job.

Default value:/mr-history/tmp

(4) Mapreduce.jobhistory.done-dir

Parameter explanation: The location of the log stored by MR jobhistory Server Management.

Default value:/mr-history/done

2. MapReduce Job Configuration parameters

Can be configured in the client's mapred-site.xml as the default configuration parameter for the MapReduce job. You can also personalize these parameters when the job is submitted.

Note that MRv2 renames all configuration parameters in MRv1, but is compatible with the old parameters in MRv1, except that a warning log is printed to prompt the user parameter to expire. MapReduce the old and new parameter table can refer to Java class Org.apache.hadoop.mapreduce.util.ConfigUtil, for example as follows:

You may also like:

1. Hadoop Yarn Configuration Parameters Analysis (1)-RM and NM related parameters

2. Hadoop Yarn Configuration parameter Analysis (2)-Permissions and log aggregation related parameters

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.