Ganglia monitoring Hadoop cluster installation deployment

First, the installation environment

Ubuntu Server 12.04

Installation of Gmetad machine: 192.168.52.105

Installation of Gmond machine: 192.168.52.31,192.168.52.32,192.168.52.33,192.168.52.34,192.168.52.35,192.168.52.36,192.168.52.37,192.168.52.3 8,192.168.52.105

To browse the machine that monitors the Web page: 192.168.52.105

Second, introduce

The Ganglia monitoring kit consists of three main sections: Gmond,gmetad, and web interface, often called Ganglia-web.

Gmond is a daemon that runs on every node that needs to be monitored, collects monitoring statistics, and sends and accepts statistics on the same multicast or unicast channel.

Gme Tad is also a daemon that periodically checks gmonds, pulls data from there, and stores their metrics in the RRD storage engine. It can query multiple clusters and aggregate metrics. RRD is also used to build the Web front-end of the user interface.

Ganglia-web, as the name suggests, should be installed on a machine with Gmetad running to read RRD files.

In general a cluster of each machine need to install Gmond, as long as the responsibility to display the Web interface of the machine installed Gmetad can.

Third, installation

1.apt-get mode installation

First find a node as a web display node, use the following command one-click installation:

Apt-get-y Install Ganglia-monitor Ganglia-webfrontend

Installing the Ganglia-webfrontend is equivalent to installing Gmetad, Ganglia-web, and RRDtool at the same time.

The above installed version is 3.1.7, because the Ubuntu official source only supports to ganglia-webfrontend-3.1.7, from the use of ganglia Web Frontend version is also 3.1.7, the version is too low, the display of the interface is very unfriendly, so later article I will explain how to manually install ganglia Web frontend version 3.5.7 (currently the highest version has to 3.6). If for convenience, for the monitoring of the interface requirements are not high, but also the plan, you can use the above command one-click installation.

In addition to installing Ganglia-webfrontend hosts, other hosts only need to install Ganglia-monitor.

2. Link to Web server

When you install Ganglia-webfrontend, you have installed the Apache Web server,ganglia-webfrontend default installation location is/usr/share/ganglia-webfront, It is not possible to access the ganglia monitoring interface through the browser at this time, and you will need to execute the following command to link the ganglia file to the Apache default directory.

Ln-s/usr/share/ganglia-webfront/var/www/ganglia

Four, configuration

1. Unicast and multicast

Before you make a ganglia cluster configuration, you should first understand unicast and multicast.

Unicast: Can propagate across network segments, sending information only to specified machines. To configure as a unicast you should specify one (or more) accepted hosts.

Multicast: Sends broadcasts to all machines located on the same network segment in the network segment of the machine. If you are using multicast transmission, then you do not need to change anything because this is ganglia package installed by default. The only thing to do is to point the Gmetad to one or several hosts running GMO nd. It is not necessary to list each individual host because GMO ND is set to accept mode and contains a list of all hosts and statistics for the entire cluster.

For more detailed and related configuration of unicast and multicast, refer to the Ganglia Quick Start wizard (translated from the official wiki) or directly to the official wiki.

2. Cluster configuration

My ganglia monitoring cluster uses unicast mode. All configurations are in the/etc/ganglia directory, and/ETC/GANGLIA/GMOND.CONF is configured on each node.

The red font is the modified item.

Globals {

Daemonize = yes

Setuid = yes

user = root//Run ganglia users/*

Debug_level = 0

Max_udp_msg_len = 1472

Mute = no

deaf = No

Host_dmax =/*secs *

Cleanup_threshold =/*secs *

Gexec = No

Send_metadata_interval = 15/* time interval for sending data * *

}

Cluster {

Name = "HADOOP_HF"/* Cluster Name * *

Owner = "Root" * * User running Ganglia/

Latlong = "Unspecified"

url = "Unspecified"

}

Udp_send_channel {

# mcast_join = 192.168.52.105/* comment off the multicast *

Host = 192.168.52.105/* Sent to install Gmetad machine * *

Port = 8649

TTL = 1

}