How to install Cloudera Manager in CentOS

How to install Cloudera Manager in CentOS
There are a lot of articles about CDH, although this article does not describe all the problems. However, the overall process is clearly described.

Question guidance:
1. What preparations are required for installation?
2. How to install CM?
3. How to install CDH through CM?
4. What are the installation processes of CM and CDH?



I believe that everyone will have a general understanding of Cloudera Manager and CDH installation through this article.

Contents 1 preparation ................................... ........................................ ........................................ ........................ 3
1.1 description ...................................... ........................................ ........................................ ......................... 3
1.2 introduction to the cluster network environment ................................... ........................................ ........................................ ....... 3
2. Build an environment ..................................... ........................................ ........................................ ...................... 5
2.1 set IP ..................................... ........................................ ........................................ ...................... 5
2.2 set hosts ..................................... ........................................ ........................................ ................. 5
2.3 close the firewall ..................................... ........................................ ........................................ ............... 6
2.4 disable SElinux ..................................... ........................................ ........................................ ............... 8
2.5 set the machine name .................................... ........................................ ........................................ ................ 8
3 ClouderaManager install ..................................... ........................................ ...................................... 11
3.1 download the Cloudera Manager installation file .................................. ........................................ ......................... 11
3.2 download the rpm package required by Cloudera Manager ................................ ........................................ .................... 11
3.3 install the rpm file .................................... ........................................ ........................................ ........... 11
3.4 install CM ..................................... ........................................ ........................................ .................. 11
3.5 log on to CM ..................................... ........................................ ........................................ .................. 12
4 ClouderaManager configuration and local ground source configuration ................................. ........................................ ..................... 13
4.1 configure config. INIFILE ....................................... ........................................ ......................................... 13
4.2 configure the local source .................................... ........................................ ........................................ .............. 14
5. Install Hadoop and its components ................................... ........................................ ........................................... 17
5.1 select Cloudera .................................... ........................................ ........................................ .... 17
5.2 select the managed host .................................. ........................................ ........................................ ...... 17
5.3 select the installation method (Parcel is recommended for Cloudera ).............................. ........................................ ............. 18
5.4 allocate Parcel to the host .................................. ........................................ ........................................... 18
5.5 Select Services ..................................... ........................................ ........................................ ................. 19
5.6 test database connection .................................... ........................................ ........................................ ....... 20
5.7 wait for all services to start ................................... ........................................ ........................................ ..... 20

6. add nodes to the cluster ................................... ........................................ ........................................ ............ 22
6.1 add a host ..................................... ........................................ ........................................ ................. 22
6.2 enter the IP address or host name ................................... ........................................ ........................................ ........ 23
6.3 provide SSH Login creden ................................... ........................................ ........................................ ..... 24
6.4 install ...................................... ........................................ ........................................ ....................... 24




Use Cloudera Manager to install the Hadoop running environment hardware and software environment l host operating system: Windows 64 bit, dual-core 4-thread, clock speed 2.2 GB, 8 GB memory
L virtual software: VMware®Workstation 9.0.0 build-812388
L virtual machine operating system: CentOs 64bit, single core, 2 GB memory
LJDK: 1.7.0 _ 45 64 bit
LHadoop: 2.2.0



The cluster network environment cluster contains three nodes: One namenode and one datanode. the nodes are connected to each other through the LAN and can be pinged to each other. The IP addresses and host names of nodes are distributed as follows:
L192.168.188.141hadoo1 (namenode ))
L192.168.188.139hadoo2 (datanode01)
L192.168.188.140hadoo3 (datanode03)
All nodes are CentOS and the firewall iptables is disabled.

Environment Construction
This cluster is divided into three nodes. First install the node hadoop1 (192.168.188.141) and complete the process from 2.1 ~ After step 2.6, copy the VM in step 2.7 and name it hadoop2 (192.168.188.139) and hadoop3 (192.168.188.140 ).


Set IP1. click System --> Preferences --> Network Connections, as shown in:




2. Delete the original connection information, create a new "" connection, set the connection as a manual method, and set the following information:
IP Address: 10.88.147.221 subnet mask: 255.255.0 Gateway: 10.88.147.1 DNS: 10)

3. Check the configured IP address in the command line. If the IP address is incorrect, Run "2.5" to restart the machine and confirm again:





Set Host file 1. Edit the/etc/hosts file and set the ing between the IP address and the machine name. The settings are as follows:
L 10.88.147.141 hadoop1.localdomain
L 10.88.147.139 hadoop2.localdomain
L 10.88.147.140 hadoop2.localdomain






2. Run the following command to restart the network settings:
Sudo/etc/init. d/network restart

3. Verify that the settings are successful.



Disable Firewall
Disable firewall and SElinux during Hadoop installation; otherwise, exceptions may occur.

1. view the Firewall status in service iptables status. The following shows that iptables is enabled.




2. Run the following command to disable iptables. After the command is executed, restart the machine to take effect.
Chkconfig iptables off

Disable SElinux1. run the getenforce command to check whether SElinux1.





2. Modify the/etc/selinux/config file
Change SELINUX = enforcing to SELINUX = disabled. Execute this command and restart the machine to take effect.




Set the machine name to log on as a root user and use vi/etc/sysconfig/network to open the configuration file. Modify the host name as hadoop1.



Copy the VM to shut down the machine and copy the current node, respectively named hadoop2 and hadoop3


SSH password-less authentication configuration (this step can be omitted)
3. Use ssh-keygen-t rsa in the three nodes to generate the private key and public key;





4. Name the Public Key authorized_keys_hadoop1, authorized_keys_hadoop2, and authorized_keys_hadoop3 in the three nodes respectively;





5. Use the scp authorized_keys_hadoop2 jan @ hadoop1:/home/jan/. ssh command to send the two slave node public keys to the/home/jan/. ssh folder of the hadoop1 node;







6. Save the public key information of the three nodes to the authorized_key file.

Use the cat authorized_keys_hadoop1> authorized_keys command





7. Distribute the file to the other two slave nodes.
Use scp authorized_keys jan @ hadoop2:/home/jan/. ssh to distribute the password file







8. Test whether ssh password-free logon takes effect





Install Cloudera Manager
Download the Cloudera Manager Installation File
Cloudera Manager can be obtained from the following URL: http://archive.cloudera.com/cm5/installer/select the latest version of Cloudera Manager 5.0.1http: // container
Download the rpm package required by Cloudera Manager
URL: http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/5.0.1/RPMS/x86_64/
Install the rpm file
L put the downloaded rpm package into the folder rpm (the folder name is random)
L cd./rpm (enter the rpm directory)
L yum localinstall -- nogpgcheck *. rpm (install the rpm package)

3.4 install rpm file l go to the cloudera-manager-installer.bin download directory
L chmod u + x cloudera-manager-installer.bin (binfile permission for download)
L run./cloudera-manager-installer.bin (about 10 minutes)

3.5 log on to the CM browser and enter localhost: 7180 First Login Username: admin Password: admin



Cloudera Manager file configuration and local source configuration
Configure the config. ini file
Modify host name
Changed to: localhost, changed to: localhost. localdomain






You can download the. Parcel file and the manifest. json file from the following URL:
Bytes
Note: The Parcel file corresponds to different system versions.
CentOS 6. X for CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcelCentOS 5. X for CDH-5.0.1-1.cdh5.0.1.p0.47-el5.parcel
L copy the downloaded. parcel file and manifest. json file to the/opt/cloudera/parcel-repo/folder.
L new file named: CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel.sha
L open the manifest. json file to view the number corresponding to the CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel and copy it to the CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel.sha File

Here is the number of the CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel in manifest. json, here is: ec68971d2969a5a31e720a2a79ce7a7c1d38e397
{"ParcelName": "CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel", "components": [{"pkg_version": "0.7.0 + cdh5.0.1 + 0", "pkg_release": "1. cdh5.0.1.000030 "," name ":" bigtop-tomcat "," version ":" 6.0.37-cdh5.0.1 "},{" pkg_version ":" 0.9.0 + cdh5.0.1 + 23 ", "pkg_release": "1. cdh5.0.1.000025 "," name ":" crunch "," version ":" 0.9.0-cdh5.0.1 "},


.........

{"Pkg_version": "0.9.0 + cdh5.0.1 + 8", "pkg_release": "1. cdh5.0.1.000025 "," name ":" whirr "," version ":" 0.9.0-cdh5.0.1 "},{" pkg_version ":" 3.4.5 + cdh5.0.1 + 30 "," pkg_release ": "1. cdh5.0.1.000031 "," name ":" zookeeper "," version ":" 3.4.5-cdh5.0.1 "}]," replaces ":" IMPALA, SOLR, SPARK "," hash ": "ec68971d2969a5a31e720a2a79ce7a7c1d38e397 "}
L now the local source has been configured. In short, the parcel-repo file contains the following three files:
CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel
CDH-5.0.1-1.cdh5.0.1.p0.47-el6.parcel.sha
Manifest. json

L restart the system


Select Cloudera for installing Hadoop and its components


Select managed hosts




Select the installation method (Parcel is recommended for Cloudera)




Allocate Parcel to host









Add Service



Test Database Connection



Wait for all services to start and wait for all services to start. This may take a long time.









Add hosts to nodes in the Cluster



Enter the IP address or host name





Provide SSH Login creden



Install