Ambari management of large data cluster master node memory expansion operation Step Description

1. Turn off host add memory

Log on to the specified host system (as per table two), switch to the root user, enter the Poweroff command to shut down the system, and the IT owner to expand the memory after the system shuts down.

2. boot Check memory information

The IT owner checks the memory information of the expansion host to determine if the memory capacity is complete before entering the system.

3. log on to the system startup service

Log in to the system and switch to the root user to do the following.

Note: Start the service order for each host node, which is Ambari, Zookeeper, YARN, hdfs,hive (below).

3.1. Check the PID file

The PID directory is the/var/run/service name, checking for the existence of the PID file for the corresponding service, and deleting it if it exists (rm–r file name).

3.2. Start ambari-agent (all nodes)

Input command: Service--status-all Check whether the services are running, mainly check ambari-agent and Ambari-server

If Ambari-agent is not running, perform the following startup steps

Input command: Service ambari-agent start

3.2. Start Ambari-server (22 nodes)

Input command: Service ambari-server start (when service is stopped)

Check log: Tail-200/var/log/ambari-server/ambari-server.out

3.3. Start Zookeeper

Log on to the Ambari-server Administration page and start the Zookeeper service manually.

3.4. Start Yarn related services

Log in to the Ambari-server Yarn Management page and start the yarn service manually.

3.5. Start HDFS Related services

Log in to the Ambari Administration home page, and start the service sequentially, as listed in the previous table, the main operating services:

A, Zkfailovercontroller (zkf responsible for Namenode HA, priority start)

B, Journalnodes

C, Namenode

3.6. Start Hive Related services

Log in to the Ambari Administration home page, and start the service sequentially, as listed in the previous table, the main operating services:

A, MySQL (MySQL check if the database is available, because Metastore relies on MySQL service)

B, Hive Metastore

C, HiveServer2

D, Webhcat Server

3.7. Start other services

Log on to the Ambari Administration home page, and start with the main running services listed in the previous table.

4. Check the main services of each node

4.1, zookeeper inspection

Log on to the target host node and use zkcli.sh to switch to the Zookeeper command line to check if the root directory can be queried properly;

Check the zookeeper boot log to see if the service is working properly based on the log.

4.2, Namenode Inspection

Log in to the Ambari-server HDFs Administration page and check the Namenode ha status;

Check the master-slave namenode node log to see if the service is functioning properly based on the log output.

5. Trouble Shooting

The service described in step 3 above encounters an exception such as startup failure, first check the log output, and troubleshoot the issue according to the log description.

Exceptions that may occur

The Q:pid file was not deleted causing the service to start failing.

A: Remove the PID file restart service;

Q:namenode ha State exception.

A: Check whether the Zkfailovercontroller service starts normally;

Q:mysql Service failed to start.

A: Remove the Mysql.sock, mysqld.pid files, and restart the service.

Ambari management of large data cluster master node memory expansion operation Step Description