Analysis of configuration parameters for Hadoop yarn (2)-Permissions and log aggregation related parameters

Note that before you configure these parameters, you should fully understand the implications of these parameters in order to prevent the pitfalls caused by the misuse of the cluster. In addition, these parameters need to be configured in Yarn-site.xml.

1.&http://www.aliyun.com/zixun/aggregation/37954.html ">nbsp; Permission-related configuration parameters

The permissions here consist of three parts: (1) How administrators and ordinary users differentiate (2) service-level permissions, such as which users can submit ResourceManager submissions to the cluster, (3) queue-level permissions, such as which users can submit jobs to queue a.

The administrator list is specified by the parameter yarn.admin.acl.

Service-level permissions are implemented by configuring Hadoop-policy.xml, similar to Hadoop 1.0

Queue-level permissions are configured internally by the corresponding resource scheduler, such as fair Scheduler or capacity Scheduler, and read the following sections.

2. Log aggregation related configuration parameters

Log aggregation is the central management function provided by yarn, which can upload the completed container/task log to HDFs, reduce nodemanager load and provide a centralized storage and analysis mechanism. By default, the container/task log exists on each NodeManager, and additional configuration is required if the Log aggregation feature is enabled.

(1) yarn.log-aggregation-enable

Parameter explanation: Whether the Log aggregation feature is enabled.

Default value: False

(2) Yarn.log-aggregation.retain-seconds

Parameter explanation: The maximum number of times a log is stored on a hdfs.

Default value: 1

(3) Yarn.log-aggregation.retain-check-interval-seconds

Parameter interpretation: How often to check the log, and will meet the deletion of the condition, if it is 0 or negative, then 1/10 of the previous value.

Default value: 1

(4) Yarn.nodemanager.remote-app-log-dir

Parameter explanation: When the application is run, the log is transferred to the HDFs directory (valid when the log aggregation function is enabled).

Default value:/tmp/logs

(5) Yarn.log-aggregation.retain-seconds

Parameter explanation: The remote log directory subdirectory name (valid when logging aggregation is enabled).

Default value: Log will be transferred to directory

${yarn.nodemanager.remote-app-log-dir}/${user}/${thisparam}