Install Cloudera Manager5.2.0 and CDH5 2.0 Offline

The first installation encountered various problems, especially for those not familiar with linux systems! I would like to record that we hope our friends can avoid detours.

 

 

1. Add a route to the machine (you can ignore it based on your machine)  Route add-net 172.17.2.0 netmask 255.255.255.0 gw 192.168.0.1 (windows)  Route/p add 172.17.2.0 mask 255.255.255.0 192.168.0.1 (linux)   2. Cluster host configuration2.1 configure static IP2.2 and disable virtual machine firewall
When the firewall is enabled, run the following two commands.
Permanently disable the firewall: chkconfig iptables off
Close Command: service iptables stop
Run the two commands at the same time. After running, check the firewall shutdown status.
Service iptables status
2.3 disable selinux
1. Command: sudo vi/etc/sysconfig/selinux
2. The information after "SELINUX =" is "disabled" 3. Restart the machine command reboot2.4 and change the host name
Vi/etc/sysconfig/network
2.5. Account creation and permission allocation
# Useradd ai
# Passwd ai
Assign sudo permissions
Install yum-y install sudo directly without the sudo command
1. Add user cube
Adduser cube
2. Add a password to the user
Passwd cube
Enter the password twice.
3. Check whether sudo is installed
Rpm-qa | grep sudo
4. Modify the sudoers file
Export do-f/etc/sudoers
Add after root ALL = (ALL) ALL
Cube ALL = (ALL) NOPASSWD: ALL
# You do not need to enter a password when switching.

2.6 modify the hosts file
Modify the/etc/hosts file
As follows: 172.17.2.146 hd01.domain.com unzip hd02.domain.com unzip hd03.domain.com unzip hd04.domain.com unzip hd05.domain.com hd05 This step requires you to set 2.7 on each machine and configure ssh password-less login
1. Check the sshd configuration file (root) on the local machine)
$ Vi/etc/ssh/sshd_config

Find the following content and remove the annotator "#"
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile. ssh/authorized_keys
2. If the configuration file is modified, restart the sshd service (root); otherwise, ignore this step.
$ Sudo/sbin/service sshd restart
3. Log on to the system through ssh and run the test command.
$ Ssh localhost
$ Ssh-p 10001 cb2
Press enter and you will be prompted to enter the password because the certificate has not yet been generated.
4. Create a public/private key for the certificate
$ Ssh-keygen-t dsa-p'-f ~ /. Ssh/id_dsa
$ Cat ~ /. Ssh/id_dsa.pub> ~ /. Ssh/authorized_keys
5. Log on to the remote server through authentication
Note: The OpenSSH service of the remote server must be started.
$ Cat ~ /. Ssh/id_dsa.pub | ssh remote user name @ remote server ip 'cat-> ~ /. Ssh/authorized_keys'
Example: $ cat ~ /. Ssh/id_dsa.pub | ssh [email protected] 'cat-> ~ /. Ssh/authorized_keys'
(All machines must have no password for intercommunication. Otherwise, problems may occur in the future)
For example, the ssh port has been changed from 22 to 10001.
Cat ~ /. Ssh/id_dsa.pub | ssh-P 10001 [email protected] 'cat-> ~ /. Ssh/authorized_keys'
Permissions are important. The directory permission of the. ssh client is generated by ssh-keygen. The default. ssh directory is 700, and id_rsa is 600,
If you have the write permission, ssh considers the file unreliable and may be rewritten. The. ssh Directory of the machine to be logged on is 700. At least the write permission is not allowed. Authorized_keys permission 644

2.8 modify MaxStartup on the machine used as the cluster Master.
A. Execute the command: cd/etc/ssh
B. Execute the command: sudo vi sshd_config
C. In the last 12 rows, set MaxStartups to 100 and remove #.
D. Execute the command: sudo service sshd restart
2.9 modify the limit on the number of opened files on all hosts.
A. Execute the command: cd/etc/security
B. Run the following command: sudo vi limits. conf.
C. Add a row: *-nofile 65536 3. Create a yum local source(Note: If some machines use YUM for installation, the system will prompt that the link cannot be resolved. ping www.baidu.com to check whether the DNS resolution is successful. If not, run echo "nameserver 8.8.8.8">/etc/resolv. conf: try again. If not, check whether the machine can connect to the Internet) 3.1 install web service # yum install httpd 3.2 on the source server, start httpd # service httpd start3.3, and install the source creation tool (this step is not required) yum install createrepo3.4 run the following statement on each host: sudo service ntpd stopsudo ntpdate pool. ntp. orgsudo service ntpd startsudo chkconfig ntpd on 4. Install postgresqlView the default packages of IPVs and execute rpm-qa | grep postgres to uninstall the packages. Run rpm-e postXXXXX

1. Install the postgresql9.0 yum repository

Rpm-I http://yum.postgresql.org/9.2/redhat/rhel-6-x86_64/pgdg-redhat92-9.2-7.noarch.rpm

2. Install the new Postgresql version.

Yum install postgresql92-server postgresql92-contrib

3. Initialize the database

/Etc/init. d/postgresql-9.2 initdb

4. Start the database

/Etc/init. d/postgresql-9.2 start

Note: After postgresql is started, it can be controlled by service postgresql-9.2 start/restart/stop.

After hitting the service, press p does not prompt postgresql-9.2, but you can use the hand to lose.

5. Add postgresql to the auto-start list

Cd/etc/init. d

Chkconfig -- add postgresql-9.2

6. Check the auto-start list.

Chkconfig -- listchkconfig postgresql-9.2 on set startup

Here we can see that postgresql is already in it.

7. By default, the PostgreSQL database creates an IPVs database user as the database administrator. The default password is blank. We need to change it to the specified password, which is set to 'ipvs'

 

Enter the following command in the console:

# Su-postgres

$ Psql

# Alter user login s with password 'users'; # select * from pg_shadow;

# Create database david;

# \ C david

David = # create table test (id integer, name text );

David = # insert into test values (1, 'David'); installation successful. # Env view all environment variable content set environment variable export PATH = (copy the original PATH) +:/var/lib/pgsql:/etc/init. d 5. Build cm local source1. download the directory connection. Select the appropriate version based on your needs. Note: CM and CDH versions must be consistent http://archive-primary.cloudera.com/cm5/repo-as-tarball http://archive-primary.cloudera.com/cm5/installer http://archive-primary.cloudera.com/cdh5/repo-as-tarball I'm using a 5.2.0 version of http://archive-primary.cloudera.com/cm5/repo-as-tarball/5.2.0/cm5.2.0-centos6.tar.gz http://archive-primary.cloudera.com/cm5/installer/5.2.0/cloudera-manager-installer.bin tar-xvf cm5.2.0-centos6.tar.gz tar-xvf cdh5.2.0-centos6.tar.gz after decompression is complete visit http on the browser: // ip/cm/5.2.0 check whether access is allowed. If yes, the local cm source is set up. 6. Install CM  6.1. Install/etc/yum on cm5. repos. d/to create a new cloudera-manager.repo for local installation, file content: [cloudera-manager] name = Cloudera Manager, Version 5.2.0baseurl = http: // ip/cm/5.2.0/(IP as the Master host IP) gpgcheck = 0 start cloudera-manager-installerd.bincd/var/www/htmlsudo. /cloudera-manager-installerd.bin next, one way to next, if the source is no problem, you can install the successful possible problems: installation to 40% may not go through, view the installation log, possible causes: cause 1: postgresql is not installed; cause 2: postgresql version is not supported (I started to install a version 9.3, If the cm5 installation is successful, you will be prompted to access the host installed on the cm server through the 7180 mark. Http: // ip: 7180 when prompted, enter the account (admin) and secret (admin) to go to the installation wizard and continue to the next step. To add a host to be installed, you can search by machine name and IP address and set the default ssh port based on your machine Port: 22. Next, select the custom repository, enter the address of the local source. Http: // ip/cdh/5.2.0/Similarly, the cloudera Manager agent also selects a custom repository. Enter the local address http: // ip/cm/5.2.0/to continue the next step, it will be in the installation phase. Possible problems in the installation phase: Problem 1: will stop obtaining the security lock solution: Check whether the host configured in vi/etc/hosts is correct and whether ssh can log on to each other without a password 2: cdh installation failed. Unable to receive detection signals from the Agent. Solution: Basically, it is also the host reason. Also, check whether the firewall on all machines is disabled. If not, follow the above command to close the firewall. If no problem occurs, the system will prompt that the installation is complete and the next step is continued.

Next, check the server. The following problems may occur:

Cloudera recommends setting/proc/sys/vm/swappiness to 0. The current value is 60. Run the sysctl command to change the setting and edit/etc/sysctl. conf to save the setting after restart. You can continue the installation, but may encounter problems. Cloudera Manager reports that your host is not performing well due to switching. The following hosts are affected:

PassEcho 0>/proc/sys/vm/swappiness.

In the next step, select the installed service (based on your own needs) service configuration. Generally, you can keep the default configuration. (Cloudera Manager will automatically configure the service based on the machine configuration. If you need special adjustments, you can set it by yourself): Next is the database settings. After the check is passed, you can proceed to the next step: in this step, the database will select Postgresql by default to record the account and password, and prompt that the installation is successful. The subsequent items will be set according to your own needs. Hope everyone can install them smoothly.

Install Cloudera Manager5.2.0 and CDH5 2.0 Offline