Cloudera Manager and CDH 4 Ultimate Installation

System Environment

Operating system: CentOS 6.5

Cloudera Manager Version: 4.8.1

CDH Version: 4.5.0

Each machine preparation work:

Yum-y Groupinstall "Development tools"

yum-y install wget

Cloudera-manager Zip package Address: http://archive.cloudera.com/cm4/cm/4/cloudera-manager-el6-cm4.8.1_x86_64.tar.gz

CDH Download Address: http://archive.cloudera.com/cdh4/parcels/

IMPALA Download Address: http://archive.cloudera.com/impala/parcels/

Cloudera Search (SOLR) Download address: http://archive.cloudera.com/search/parcels/latest/

Official Installation Reference

http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/ Cloudera-manager-installation-guide/cmig_install_path_c.html

There are three ways to install this website:

　　

We chose the third method.

The reason is that the first method must require that all machines be connected, and that foreign sites are not very stable. Once failed, reloading is very painful.

The second method has not been studied carefully and it seems to be downloading many packages.

The third method has the least intrusive system, the biggest advantage can be fully offline installation, and re-install what is very convenient. The later cluster Unified package upgrade is also very good.

preparatory work

All of the following operations are done as root.

0 modifying host names

Modify the/etc/sysconfig/network file

Networking=yes

HOSTNAME=YC01

Effective after reboot: Service network restart

Modify/etc/hosts file, similar to

127.0.0.1		yc01 (cloudera-manager-server)

       192.168.1.191 yc01
192.168.1.192 yc02
192.168.1.193 yc03
192.168.1.194 yc04
192.168.1.19 5 YC05

Note that the hosts on the Cloudera-manager-server host put 127.0.0.1 hadoop001 in front of the/opt/cloudera/parcels because the agent downloads the parcels package from the server and extracts it to Under the/opt/cloudera/parcels directory

1 Install salt (can be ignored)

Slightly.

2 getting through SSH

All machines, use ssh-keygen-t RSA all the way to press ENTER on the line.

What had been done just now. The main is to generate SSH key and key storage path. Under the ~/.ssh.

Open ~/.ssh There are at least two files

Id_rsa, private key

Id_rsa.pub, Public key

On the main node:

Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

SCP files to other machines:

SCP ~/.ssh/authorized_keys root@yc02:~/.ssh/

Now log on to other machines without a password.

3 Installing Java

Because CDH4 support JAVA7, consider CDH5 only support JAVA7, decisive on. (later MySQL also used the latest 5.6.16, later found that the tragedy, do not know which reason, so the JDK has changed to the official recommendation version, or not, and the MySQL back to the 5.1.X version, the final can be.) Personal guess JDK can still use 7, MySQL can only use 5.5, and then go up on it. Reference http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/ cdh4-requirements-and-supported-versions/cdhrsv_topic_2.html)

Download and unzip the JDK from the Oracle website. Tar zxvf jdk-7u51-linux-x64.gz under/opt/java.

It seems to have seen before, CDH need a/usr/java/default of the soft chain, will be recognized. Just in case, build one, at least it's easier to manage:

Mkdir-p/usr/java; Cd/usr/java;
Ln-s/opt/java/jdk1.7.0_51 Latest; Ln-s Latest Default

Finally Java Add environment variable

Export Java_home=/usr/java/default
export path= $JAVA _home/bin: $PATH
export classpath=.: $JAVA _home/lib/ Dt.jar: $JAVA _home/lib/tools.jar

Make configuration effective

Source/etc/profile

4 Installing the MySQL database (The installation process has been skipped directly, directly yum-y install Mysql-server to build the required database)

Cloudera Manager's own database of needs, it now supports MySQL, PostgreSQL and Oracle, I have not used PostgreSQL, Oracle selectmen, decisive on MySQL.

Installing and configuring a MySQL Database

Official online said a lot, summed up three words:

1) Install a Mysql

Download Address http://dev.mysql.com/downloads/mysql/

Download the client and server

RPM-IVH mysql-server-5.6.16-1.el6.x86_64.rpm

RPM-IVH mysql-client-5.6.16-1.el6.x86_64.rpm

The installation server is a random code that will prompt for root generation.

　　　　　　　

SET PASSWORD = PASSWORD (' ycmedia_2014 ');

The default MySQL does not generate a/etc/my.cnf file, cp/usr/my.cnf/etc/my.cnf

2) Configure some parameters of MySQL itself, and CDH does not have any relationship. Emphasis is placed on the value of the max_connections. The recommendation is the number of databases *100+50. Large clusters (more than 50 units) are recommended to be divided into open.

Procedure: VI/ETC/MY.CNF, modify which parameters, refer to the above link. This step can be skipped directly for general purposes only.

3) Build database, authorize. What libraries are needed, free version: Amon, Hmon, Hive, Smon, SCM (Cloudera Manager itself, I named myself)

The names of these libraries are actually customizable. The other is the name of the official website example, that's it. Remain consistent. Only the last library was named after me.

Create DATABASE hive DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;

Create database Amon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;

Create database Smon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;

Create database Hmon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;

Authorization to the user, similar to the following:

Grant all on *. root@ "%" identified by "123456";

5 shutting down firewalls and SELinux

To turn off the firewall:

Service iptables Stop (temporary shutdown)

Chkconfig iptables off (effective after reboot)

To turn off SELinux:

Setenforce 0 (Temporary entry into force)

Modify selinux=disabled under/etc/selinux/config (permanent after reboot)

officially started

attached with Yum installation:

/etc/yum.repos.d/cloudera-manager.repo Configuring CDH4 sources

[Java] view plain copy [Cloudera-manager] name = Cloudera Manager, Version 4.8.3 baseurl =/http Archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.8.3/gpgkey = http://archive.cloudera.com/redhat/cdh/ Rpm-gpg-key-cloudera Gpgcheck = 1 cloudera-manager-server use: Yum install cloudera-manager-daemons cloudera-manager-s Erver

cloudera-manager-agent use: Yum install cloudera-manager-daemons cloudera-manager-agent

You can also download and install it.

1 Installing Cloudera Manager Server and Agents

Unpack the Tarball:

Tar xzf/opt/package/cloudera-manager*.tar.gz-c/opt/; mv/opt/cm-4.8.1/opt/cloudera-manager-4.8.1

Add CLOUDERA-SCM User

Useradd--system--home=/opt/cloudera-manager-4.8.1/run/cloudera-scm-server--no-create-home--shell=/bin/false-- Comment "Cloudera SCM User" CLOUDERA-SCM

For convenience, the following is called/opt/cloudera-manager-4.8.1 <tarball root>

Modify <tarball Root>/etc/cloudera-scm-agent/config.ini below the Server_port and server_host, put Server_port Change to Clouderamanager run hostname or IP, default server_port port is 7182

Cloudera Manager will save some stuff in the directory below.