Centos6.5 installation configuration Cloudera Manager CDH5.6.0 Hadoop

Environmental planning
Operating system version: Centos-6.5-x86_64-minimal.iso
192.168.253.241 Master virtual Machine 4G memory
192.168.253.242 slave1 virtual Machine 2G memory
192.168.253.243 slave2 virtual Machine 2G memory

First, environment initialization
1, modify the hostname (each machine is executed)

192.168.253.241 master192.168.253.242 slave1192.168.253.243 Slave2

2, host name resolution (each machine is executed)

Echo ' 192.168.253.241 master192.168.253.242 slave1192.168.253.243 slave2 ' >>/etc/hosts

3. SSH Login without password
#主机: Master Execution command

[Email protected] ~]# ssh-keygen-t rsa[[email protected] ~]# ssh-copy-id-i ~/.ssh/id_rsa.pub [email protected][[email p Rotected] ~]# ssh-copy-id-i ~/.ssh/id_rsa.pub [email protected][[email protected] ~]# ssh-copy-id-i ~/.ssh/id_rsa.pub [E Mail protected]

4. Close the iptables rule (each machine is executed)

[[email protected] opt] #service iptables stop && chkconfig iptables off

5. Install JDK software and configure the JDK environment (each machine executes)

[Email protected] soft]# cd/root/soft/#JDK下载: http://www.oracle.com/technetwork/java/javase/downloads/ Java-archive-downloads-javase7-521261.html#jdk-7u80-oth-jpr/jdk-7u80-linux-x64.rpm[[email protected] soft]# RPM- IHV jdk-7u80-linux-x64.rpm [[email protected] soft]# vim/etc/profile #文件末尾增加下面内容export java_home=/usr/java/ Defaultexport classpath=.: $JAVA _home/jre/lib/rt.jar: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jarexport PATH=$ PATH: $JAVA _home/bin# Refresh environment variables [[email protected] soft]# source/etc/profile && java-version

6, modify the kernel parameters (each machine is executed)

echo 0 >/proc/sys/vm/swappiness

7, master node installation MySQL(master on the execution)

[[email protected] soft]# yum install -y http://www.percona.com/downloads/ Percona-release/percona-release-0.0-1.x86_64.rpm[[email protected] soft]# yum install  -y percona-server-shared-compat[[email protected] soft]# yum install -y  percona-server-server-55 percona-server-client-55[[email protected] soft]# service  mysql start[[email protected] soft]# mysql -u root -pmysql>  grant all on *.* to [email protected] ' localhost '  identified by  ' 123456 '; query ok, 0 rows affected  (0.00 sec) mysql> grant all on * . * to [email protected] ' 127.0.0.1 '  identified by  ' 123456 '; query ok, 0 rows affected  (0.00 sec) mysql> delete from user  where password= "; Query ok, 4 rows affected  (0.00 sec) mysql> flush privileges; query ok, 0 rows affected  (0.00 SEC)

Second, formally start the installation
1, Cloudera Manager default directory is under/opt, extracted into the/opt directory. (performed on Master)

[Email protected] soft]# cd/root/soft/[[email protected] soft]# wget http://archive-primary.cloudera.com/cm5/cm/5/ Cloudera-manager-el6-cm5.6.0_x86_64.tar.gz[[email protected] soft]# tar zxvf cloudera-manager-el6-cm5.6.0_x86_64. Tar.gz-c/opt/[[email protected] opt]# lltotal 8drwxr-xr-x 4 1106 592 4096 Feb 12:05 clouderadrwx------5 root root 4 096 Mar 1 14:09 cm-5.6.0

#安装mysql-connector-java

[Email protected] soft]# yum-y install mysql-connector-java[[email protected] soft]# rpm-e JAVA-1.5.0-GCJ--nodeps #卸载j Ava old version

2. Initialize the database

[Email protected] opt]#/opt/cm-5.6.0/share/cmf/schema/scm_prepare_database.sh MySQL cm-hlocalhost-uroot-p123456--    Scm-host localhost SCM SCM scm[[email protected] opt]# mysql-uroot-p123456-e "show databases;" #查看下cm数据库是否创建成功

#修改配置文件server_host为master节点的主机名

[Email protected] soft]# sed-i ' s/server_host=localhost/server_host=master/g '/opt/cm-5.6.0/etc/cloudera-scm-agent /config.ini[[email protected] soft]# scp-rp/opt/cm-5.6.0 slave1:/opt/[[email protected] soft]# scp-rp/opt/cm-5.6.0 SLA ve2:/opt/

#每台服务器执行创建系统用户 (Take master as an example)

[Email protected] soft]# useradd--system--home=/opt/cm-5.6.0/run/cloudera-scm-server--no-create-home--shell=/bin /false--comment "Cloudera SCM User" CLOUDERA-SCM

3, the configuration of the database

[Email protected] soft]# mysql-u root-p123456#hivecreate database hive DEFAULT CHARSET UTF8 COLLATE utf8_general_ci; #ac tive monitorcreate database Amon DEFAULT CHARSET UTF8 COLLATE utf8_general_ci; #授权master主机grant all on *. * to ' root ' @ ' Maste R ' identified by ' PASSWD ';

4. Prepare parcels installation package to/opt/cloudera/parcel-repo/

#CDH 5.6.0 Download: [[email protected] Parcel-repo] #wget http://archive-primary.cloudera.com/cdh5/parcels/5.6.0/ Cdh-5.6.0-1.cdh5.6.0.p0.45-el6.parcel[[email protected] Parcel-repo] #wget http://archive-primary.cloudera.com/ Cdh5/parcels/5.6.0/cdh-5.6.0-1.cdh5.6.0.p0.45-el6.parcel.sha1[[email protected] Parcel-repo] #wget/http Archive-primary.cloudera.com/cdh5/parcels/5.6.0/manifest.json[[email protected] Parcel-repo] #mv CDH-5.6.0-1.CDH5.6.0.P0.45-EL6.PARCEL.SHA1 Cdh-5.6.0-1.cdh5.6.0.p0.45-el6.parcel.sha

5. Start the service

#启动master节点的server和agent脚本

/opt/cm-5.6.0/etc/init.d/cloudera-scm-agent start/opt/cm-5.6.0/etc/init.d/cloudera-scm-server Start

#启动客户端节点的agent脚本

/opt/cm-5.6.0/etc/init.d/cloudera-scm-agent start

#server端口启动较慢, be sure to check ports 7180 and 7182 to be OK.

[[email protected] parcel-repo]# netstat-tunlp|grep javatcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 14644/java TCP 0 0 0.0.0.0:7182 0.0.0.0:* LIS TEN 14644/java

Three, CDH5 installation configuration

1. Access 192.168.253.241:7180 start Installation (login: admin password: admin)

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M01/7C/F5/wKiom1bdM13zZkTrAABHxmWnRFo886.png "title=" 1.png " alt= "Wkiom1bdm13zzktraabhxmwnrfo886.png"/>

2. Consent Agreement

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M01/7C/F3/wKioL1bdNA_TWo2oAACjfMWIO2o317.png "title=" 2.png " alt= "Wkiol1bdna_two2oaacjfmwio2o317.png"/>

3. Select a version to install and select the first free version here.

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/7C/F3/wKioL1bdNFLTDGczAACPF-_DQWk504.png "title=" 3.png " alt= "Wkiol1bdnfltdgczaacpf-_dqwk504.png"/>

4. Thank you for choosing Cloudera Manager and CDH.

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M01/7C/F5/wKiom1bdNADReqbwAACCxWkhO64314.png "title=" 4.png " alt= "Wkiom1bdnadreqbwaaccxwkho64314.png"/>

5. Specify a host for the CDH cluster installation

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/7C/F5/wKiom1bdNDjT4m0CAABTyG9eeRM063.png "title=" 5.png " alt= "Wkiom1bdndjt4m0caabtyg9eerm063.png"/>

6. Cluster installation

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M01/7C/F3/wKioL1bdNOfyaBHsAADLQJVhAAY656.png "title=" 6.png " alt= "Wkiol1bdnofyabhsaadlqjvhaay656.png"/>

7. Installing the selected parcel

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/7C/F5/wKiom1bdNKPRotJ-AAAy5Raazqk817.png "title=" 7.png " alt= "Wkiom1bdnkprotj-aaay5raazqk817.png"/>

8. Check Host correctness

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/7C/F5/wKiom1bdNM6A1SwqAABdxp9TCz4811.png "title=" 8.png " alt= "Wkiom1bdnm6a1swqaabdxp9tcz4811.png"/>

9. Select the CDH5 service you want to install on the cluster

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/7C/F3/wKioL1bdNX6y5aI5AABuFev7L5A985.png "title=" 9.png " alt= "Wkiol1bdnx6y5ai5aabufev7l5a985.png"/>

10. Custom Role Assignments
At the top of the form you can customize role assignments for the new cluster here, but if the assignment is incorrect (for example, too many roles assigned to a host) can affect service performance. Cloudera does not recommend changing allocations unless you have special needs, such as pre-selecting specific hosts for a particular role.

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M01/7C/F5/wKiom1bdNUnjbJrKAABhBf_vPLM431.png "title=" 10.png "alt=" Wkiom1bdnunjbjrkaabhbf_vplm431.png "/>

11. Database Settings

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M01/7C/F5/wKiom1bdNXODEgixAACE8qDnkWE513.png "title=" 11.png "alt=" Wkiom1bdnxodegixaace8qdnkwe513.png "/>

12. Audit changes

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M01/7C/F3/wKioL1bdNiDTO-gqAACU6ubo188320.png "title=" 12.png "alt=" Wkiol1bdnidto-gqaacu6ubo188320.png "/>

13. First Run

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/7C/F3/wKioL1bdNlTypYvsAABjT7YJIK0236.png "title=" 13.png "alt=" Wkiol1bdnltypyvsaabjt7yjik0236.png "/>

14. Completion of the landing screen

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/7C/F3/wKioL1bdNsGAcAD8AABrv8BH9Lw368.png "title=" 14.png "alt=" Wkiol1bdnsgacad8aabrv8bh9lw368.png "/>

Note: Previously deployed Hadoop cluster is source code compiled, installation configuration is not very convenient, after understanding Cloudera manager CDH, feel cm management is very convenient. But in the installation process, also encountered a lot of problems, but also asked a lot of friends, eventually finally took care of. In order to allow beginners to learn hadoop faster, so I wrote a detailed installation of the configuration documentation for everyone to learn and progress with each other.

This article is from the "Chengdu @ Ah-like" blog, please be sure to keep this source http://azhuang.blog.51cto.com/9176790/1748432

Centos6.5 installation configuration Cloudera Manager CDH5.6.0 Hadoop