Install Hadoop Cluster Monitoring Tool Ambari

Apache Ambari is a Web-based open-source project that monitors, manages, and manages Hadoop lifecycles. It is also a project that selects management for the Hortonworks data platform. Ambari supports the following management services:

Apache HBase
Apache HCatalog
Apache Hadoop HDFS
Apache Hive
Apache Hadoop MapReduce
Apache Oozie
Apache Pig
Apache Sqoop
Apache Templeton
Apache Zookeeper

Ambari allows you to install hadoop clusters, manage hadoop cluster services, and monitor clusters easily. Currently, ambari supports rhel/CentOS 5 and rhel/centos6.

The installation procedure is

1. First, you must use the svn client to obtain the ambari source code.

$ Svn checkout http://svn.apache.org/repos/asf/incubator/ambari/trunk ambari

2. Package and compile the ambari source code

$ Yum install rpm-build
$ Cd hmc/package/rpm
$./Create_hmc_rpm.sh

$ Cd mon_dashboard/package/rpm
$./Create_dashboard_rpm.sh if the compilation is successful, the following rpm file will be generated

1) hmc/package/rpm/build/rpmbuild/RPMS/noarch/ambari-1.0.0-1.noarch.rpm

2) hmc/package/rpm/build/rpmbuild/RPMS/noarch/ambari-agent-1.0.0-1.noarch.rpm

3) mon_dashboard/package/rpm/build/rpmbuild/RPMS/noarch/hdp_mon_dashboard-0.0.1-2.noarch.rpm

3. Steps required before installing ambari

Divide the hadoop cluster to be installed according to namenode and datanode, and use namenode as the ambari master node and other nodes as the ambari slave.

Set ssh settings for non-password login from namenode nodes to other nodes.

Stop the firewall of each node from service iptables stop

4. start installing ambari

Because ambari depends on the puppet ruby php apache module, you must first install these components using the yum source.

Set the system yum source on each node

$ Sudo rpm-Uvh http://dl.Fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm
OR
$ Sudo rpm-Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

$ Sudo rpm-Uvh http://passenger.stealthymonkeys.com/rhel/5/passenger-release.noarch.rpm
$ Sudo rpm-Uvh http://public-repo-1.hortonworks.com/HDP-1.0.1.14/repos/centos5/hdp-release-1.0.1.14-1.el5.noarch.rpm (for rhel5/centos5) ORrpm-Uvh http://public-repo-1.hortonworks.com/HDP-1.0.1.14/repos/centos6/hdp-release-1.0.1.14-1.el6.noarch.rpm (for rhel6/centos6)

Install puppet php and other modules on the master node of ambari

$ Sudo yum install php-pecl-json # version 1.2.1 compatible with php-5.1 or php-5.2
$ Sudo yum install php-pdo check whether posix is installed
$ Php-m | grep posix if the posix module is not found, use

$ Sudo yum install php-process.

$ Sudo yum install pdsh
$ Sudo yum install httpd-devel
$ Sudo yum install mod_passenger
$ Sudo yum install puppet-2.7.9
$ Sudo yum install after the modules on ruby-devel are installed, install them on the ambari master node.

$ Sudo rpm-iv ambari-1.0.0-1.noarch.rpm
$ Sudo rpm-iv hdp_mon_dashboard-0.0.1-2.noarch.rpm is installed on other slave nodes

$ Sudo rpm-Uvh ambari-agent-1.0.0-1.noarch.rpm

Because hadoop needs jdk environment to run, so need to download to http://www.Oracle.com/technetwork/java/javase/downloads/jdk-6u26-download-400750.html

1) jdk-6u26-linux-x64.bin

2) jdk-6u26-linux-i586.bin

Install the package and put it in the/var/run/hmc/downloads/folder.
4. Start ambari server and start cluster Installation

Start the ambari server Service on the master node of ambari master.

Service ambari start and then open it in the browser

Http: // AMBARIMASTER/hmc/html/address

To install the cluster, the root user's SSH Private Key File in the ambari master node is required. The path is/root/. ssh/id_rsa.

Then, all the hostnames of the Server Load balancer nodes to be installed are separated into files by one row.

After selecting a file on the page, you can install it. It takes some time to install the service, because all hadoop is downloaded and installed through yum.

After the installation is successful, you can manage and monitor the cluster.

Reference http://incubator.apache.org/ambari/install.html

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)