System Environment
Operating system: CentOS 6.5
Cloudera Manager Version: 4.8.1
CDH Version: 4.5.0
Each machine preparation work:
Yum-y Groupinstall "Development tools"
yum-y install wget
Cloudera-manager Zip package Address: http://archive.cloudera.com/cm4/cm/4/cloudera-manager-el6-cm4.8.1_x86_64.tar.gz
CDH Download Address: http://archive.cloudera.com/cdh4/parcels/
IMPALA Download Address: http://archive.cloudera.com/impala/parcels/
Cloudera Search (SOLR) Download address: http://archive.cloudera.com/search/parcels/latest/
Official Installation Reference
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM4Ent/latest/ Cloudera-manager-installation-guide/cmig_install_path_c.html
There are three ways to install this website:
We chose the third method.
The reason is that the first method must require that all machines be connected, and that foreign sites are not very stable. Once failed, reloading is very painful.
The second method has not been studied carefully and it seems to be downloading many packages.
The third method has the least intrusive system, the biggest advantage can be fully offline installation, and re-install what is very convenient. The later cluster Unified package upgrade is also very good.
preparatory work
All of the following operations are done as root.
0 modifying host names
Modify the/etc/sysconfig/network file
Networking=yes
HOSTNAME=YC01
Effective after reboot: Service network restart
Modify/etc/hosts file, similar to
127.0.0.1 yc01 (cloudera-manager-server)
192.168.1.191 yc01
192.168.1.192 yc02
192.168.1.193 yc03
192.168.1.194 yc04
192.168.1.19 5 YC05
Note that the hosts on the Cloudera-manager-server host put 127.0.0.1 hadoop001 in front of the/opt/cloudera/parcels because the agent downloads the parcels package from the server and extracts it to Under the/opt/cloudera/parcels directory
1 Install salt (can be ignored)
Slightly.
2 getting through SSH
All machines, use ssh-keygen-t RSA all the way to press ENTER on the line.
What had been done just now. The main is to generate SSH key and key storage path. Under the ~/.ssh.
Open ~/.ssh There are at least two files
Id_rsa, private key
Id_rsa.pub, Public key
On the main node:
Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
SCP files to other machines:
SCP ~/.ssh/authorized_keys root@yc02:~/.ssh/
Now log on to other machines without a password.
3 Installing Java
Because CDH4 support JAVA7, consider CDH5 only support JAVA7, decisive on. (later MySQL also used the latest 5.6.16, later found that the tragedy, do not know which reason, so the JDK has changed to the official recommendation version, or not, and the MySQL back to the 5.1.X version, the final can be.) Personal guess JDK can still use 7, MySQL can only use 5.5, and then go up on it. Reference http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/ cdh4-requirements-and-supported-versions/cdhrsv_topic_2.html)
Download and unzip the JDK from the Oracle website. Tar zxvf jdk-7u51-linux-x64.gz under/opt/java.
It seems to have seen before, CDH need a/usr/java/default of the soft chain, will be recognized. Just in case, build one, at least it's easier to manage:
Mkdir-p/usr/java; Cd/usr/java;
Ln-s/opt/java/jdk1.7.0_51 Latest; Ln-s Latest Default
Finally Java Add environment variable
Export Java_home=/usr/java/default
export path= $JAVA _home/bin: $PATH
export classpath=.: $JAVA _home/lib/ Dt.jar: $JAVA _home/lib/tools.jar
Make configuration effective
Source/etc/profile
4 Installing the MySQL database (The installation process has been skipped directly, directly yum-y install Mysql-server to build the required database)
Cloudera Manager's own database of needs, it now supports MySQL, PostgreSQL and Oracle, I have not used PostgreSQL, Oracle selectmen, decisive on MySQL.
Installing and configuring a MySQL Database
Official online said a lot, summed up three words:
1) Install a Mysql
Download Address http://dev.mysql.com/downloads/mysql/
Download the client and server
RPM-IVH mysql-server-5.6.16-1.el6.x86_64.rpm
RPM-IVH mysql-client-5.6.16-1.el6.x86_64.rpm
The installation server is a random code that will prompt for root generation.
SET PASSWORD = PASSWORD (' ycmedia_2014 ');
The default MySQL does not generate a/etc/my.cnf file, cp/usr/my.cnf/etc/my.cnf
2) Configure some parameters of MySQL itself, and CDH does not have any relationship. Emphasis is placed on the value of the max_connections. The recommendation is the number of databases *100+50. Large clusters (more than 50 units) are recommended to be divided into open.
Procedure: VI/ETC/MY.CNF, modify which parameters, refer to the above link. This step can be skipped directly for general purposes only.
3) Build database, authorize. What libraries are needed, free version: Amon, Hmon, Hive, Smon, SCM (Cloudera Manager itself, I named myself)
The names of these libraries are actually customizable. The other is the name of the official website example, that's it. Remain consistent. Only the last library was named after me.
Create DATABASE hive DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;
Create database Amon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;
Create database Smon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;
Create database Hmon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci;
Authorization to the user, similar to the following:
Grant all on *. root@ "%" identified by "123456";
5 shutting down firewalls and SELinux
To turn off the firewall:
Service iptables Stop (temporary shutdown)
Chkconfig iptables off (effective after reboot)
To turn off SELinux:
Setenforce 0 (Temporary entry into force)
Modify selinux=disabled under/etc/selinux/config (permanent after reboot)
officially started
attached with Yum installation:
/etc/yum.repos.d/cloudera-manager.repo Configuring CDH4 sources
[Java] view plain copy [Cloudera-manager] name = Cloudera Manager, Version 4.8.3 baseurl =/http Archive.cloudera.com/cm4/redhat/6/x86_64/cm/4.8.3/gpgkey = http://archive.cloudera.com/redhat/cdh/ Rpm-gpg-key-cloudera Gpgcheck = 1 cloudera-manager-server use: Yum install cloudera-manager-daemons cloudera-manager-s Erver
cloudera-manager-agent use: Yum install cloudera-manager-daemons cloudera-manager-agent
You can also download and install it.
1 Installing Cloudera Manager Server and Agents
Unpack the Tarball:
Tar xzf/opt/package/cloudera-manager*.tar.gz-c/opt/; mv/opt/cm-4.8.1/opt/cloudera-manager-4.8.1
Add CLOUDERA-SCM User
Useradd--system--home=/opt/cloudera-manager-4.8.1/run/cloudera-scm-server--no-create-home--shell=/bin/false-- Comment "Cloudera SCM User" CLOUDERA-SCM
For convenience, the following is called/opt/cloudera-manager-4.8.1 <tarball root>
Modify <tarball Root>/etc/cloudera-scm-agent/config.ini below the Server_port and server_host, put Server_port Change to Clouderamanager run hostname or IP, default server_port port is 7182
Cloudera Manager will save some stuff in the directory below.