Cloudera Manager and CDH 4 ultimate Installation

System Environment

Operating System: CentOS 6.5

Cloudera Manager version: 4.8.1

CDH version: 4.5.0

Preparations for each machine:

yum -y groupinstall "Development tools"yum -y installwget

Cloudera-manager package address: http://archive.cloudera.com/cm4/cm/4/cloudera-manager-el6-cm4.8.1_x86_64.tar.gz

CDH: http://archive.cloudera.com/cdh4/parcels/

IMPALA: http://archive.cloudera.com/impala/parcels/

Cloudera Search (SOLR): http://archive.cloudera.com/search/parcels/latest/

 

Official Installation Reference

Http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/Cloudera-Manager-Installation-Guide/cmig_install_path_C.html

The official website introduces three installation methods:

We chose the third method.

The reason is that the first method requires that all machines be connected to the Internet, and foreign websites are not stable. Once a failure occurs, reinstallation is very painful.

The second method has not been carefully studied. It seems that many packages need to be downloaded.

The third method is the least invasive to the system. The biggest advantage is that it can be fully installed offline, and it is very convenient to reinstall anything. The cluster uniform package upgrade is also very good in the future.

 

Preparations

All of the following operations are performed as root.

0. Modify the Host Name

Modify the/etc/sysconfig/network File

　　　　NETWORKING=yes　　　　HOSTNAME=yc01

 

Effective after restart: service network restart

 

Modify the/etc/hosts file, similar

        192.168.1.191 yc01　　　　　　192.168.1.192 yc02　　　　192.168.1.193 yc03　　　　192.168.1.194 yc04　　　　192.168.1.195 yc05

 

1 install salt (negligible)

.

2. ssh connection

Run ssh-keygen-t rsa on all servers and press Enter.

What have you done just now? It mainly generates ssh key and key storage path. In ~ /. Ssh.

Open ~ /. Ssh contains at least two files

Id_rsa, Private Key

Id_rsa.pub, Public Key

On the master node:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

Scp file to other machines:

scp ~/.ssh/authorized_keys root@yc02:~/.ssh/

Now you do not need to use a password when logging on to another machine.

3. install java

Because CDH4 supports Java 7, considering that CDH5 only supports Java 7, it is decisive. (Mysql later used the latest version 5.6.16, and later found the tragedy. I don't know why, so JDK was changed to the officially recommended version, I rolled back Mysql to version 5.1.X. I guess JDK can still use 7, Mysql can only use 5.5, and it won't work if it goes up. Reference http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Requirements-and-Supported-Versions/cdhrsv_topic_2.html)

Download and decompress JDK from the official Oracle website. Put tar zxvf jdk-7u51-linux-x64.gz under/opt/java.

Previously, it seems that CDH needs a soft chain of/usr/java/default to be recognized. Just in case, a new one is created, at least so it is easier to manage:

mkdir -p /usr/java ; cd /usr/java ;ln -s /opt/java/jdk1.7.0_51 latest ; ln -s latest default

Add environment variables in JAVA

export JAVA_HOME=/usr/java/defaultexport PATH=$JAVA_HOME/bin:$PATHexport CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

Make the configuration take effect

source /etc/profile

 

4. install the Mysql database. (the installation process is skipped and you can directly run yum-y install mysql-server to create the required database)

Cloudera Manager's own database needs. Currently, it supports Mysql, PostgreSQL, and Oracle. I have never used PostgreSQL. Oracle is just a mentor, and I decided to use Mysql.

Installing and inserting ing a MySQL Database

There are a lot of things on the official website, and three sentences are summarized:

1) install a Mysql

Http://dev.mysql.com/downloads/mysql/

Download the client and server

Rpm-ivh MySQL-server-5.6.16-1.el6.x86_64.rpm

Rpm-ivh MySQL-client-5.6.16-1.el6.x86_64.rpm

Installing the server prompts you to generate a random code for the root user.

 

Set password = PASSWORD ('ycmedia _ 000000 ');

By default, mysql does not generate the/etc/my. cnf file, cp/usr/my. cnf/etc/my. cnf

2) configuring MySql parameters does not matter to CDH. Only the value of max_connections is highlighted. The recommended number of databases is * 100 + 50. Large clusters (more than 50 servers) are recommended to be opened separately.

Process: vi/etc/my. cnf. For the parameters to modify, refer to the link above. Skip this step for general purposes only.

3) Create and authorize databases. What databases are needed? Free Version: amon, hmon, hive, smon, and scm (I name Cloudera Manager myself)

The names of these databases can be customized. The others are the names of the official website examples. Always consistent. Only the last database is named by myself.

Create database hive default charset utf8 COLLATE utf8_general_ci;

Create database amon default charset utf8 COLLATE utf8_general_ci;

Create database smon default charset utf8 COLLATE utf8_general_ci;

Create database hmon default charset utf8 COLLATE utf8_general_ci;

Authorize a user, for example:

Grant all on *. * to root @ "%" Identified by "123456 ";

 

5. Disable the firewall and SELinux.

Disable Firewall:

Service iptables stop (temporarily disabled)

Chkconfig iptables off (effective after restart)

Disable SELINUX:

Setenforce 0 (temporarily effective)

Modify selinux = disabled under/etc/SELINUX/config (it takes effect permanently after restart)

1 2 3 4 Next Page