Cloudera learning3:hadoop Configuration and daemon logs

Services:haddoop components that can be deployed on cluster, such as Hdfs,yarn,hbase.

Roles: When the service is configured, it is created by Cloudera Manager. For example, Namenode is a role of the HDFs service.

Role group: The management of role can divide the same category of roles (such as datanode) into different role groups. Each role group can have its own series of configurations.

Role Instance: A single instance (which can be considered a process) that makes up the most basic service. An HDFS service is composed of namenode roles and datanode roles. The Datanode role also contains n Datanode instances.

Configuration Priority:role Instance > Role Group > Service

Cloudera Manager automatically creates a role groups based on the resource availability. role can move between role group.

Each service managed by Cloudera Manager has two profiles. One is the service default configuration file, CM is called the client configuration, stored in the/etc/hadoop/conf; one is the configuration file that the Cloudera agent needs to access when it runs, placed in/var/run/ The cloudera-scm-agent/process/directory.

The log for the Hadoop daemon mainly includes the log of the Namenode,datanodes,nodemanager,jobhistory server.

Application log includes the log of yarn containers, and the log of the service such as Spark,hive

. Out file: Contains stdout and stderr at the start of the process.

. log file: The service's log file.

Cloudera learning3:hadoop Configuration and daemon logs