Hadoop offline installation of cdh5.1 Chapter 2: cloudera manager and Agent installation

Create a cloudera-SCM user (with a branch, This is a Linux Command ,)

[[email protected] cloudera-manager-5.1.0]$ sudo useradd --system --home=/opt/cloudera-manager-5.1.0/run/cloudera-scm-server --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm

Change server_port and server_host in/opt/cloudera-manager-5.1.0/etc/cloudera-SCM-agent/config. ini and change server_port to the host name or IP address of clouderamanager. The default server_port port is 7182.

[[email protected] cloudera-scm-agent]$ sudo vim config.ini[General]# Hostname of the CM server.server_host=localhost# Port that the CM server is listening on.server_port=7182

To create a database for cloudera manager, you need to download the JDBC driver of MySQL. This time, you need to download the latest version from the Sohu image.

Http://mirrors.sohu.com/mysql/Connector-J/mysql-connector-java-5.1.32.tar.gz

Decompress the package and find the mysql-connector-java-5.1.32-bin.jar in the/opt/cloudera-manager-5.1.0/share/CMF/lib/directory.

Reference link:

Http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installation-Guide/cm5ig_install_path_C.html

How to configure http://www.cloudera.com/content/cloudera-content/cloudera-docs/CM5/latest/Cloudera-Manager-Installation-Guide/cm5ig_installing_configuring_dbs.html#cmig_topic_5_2_unique_1 Database

[[Email protected] mysql-connector-java-5.1.32] $ sudo CP mysql-connector-java-5.1.32-bin.jar

 /opt/cloudera-manager-5.1.0/share/cmf/lib/

Generate a database: (there is a small pitfall in the password. If you have ever encountered a problem, trust me)

[[email protected] schema]$ pwd/opt/cloudera-manager-5.1.0/share/cmf/schema[[email protected] schema]$ sudo ./scm_prepare_database.sh mysql -h localhost -u root -phadoop --scm-host localhost scm scm scm       Verifying that we can write to /opt/cloudera-manager-5.1.0/etc/cloudera-scm-serverCreating SCM configuration file in /opt/cloudera-manager-5.1.0/etc/cloudera-scm-serverExecuting:  /usr/java/jdk1.7.0_67/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cloudera-manager-5.1.0/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager-5.1.0/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.[                          main] DbCommandExecutor              INFO  Successfully connected to database.All done, your SCM database is configured correctly!

Start SCM Server

[[email protected] init.d]$ sudo ./cloudera-scm-server startStarting cloudera-scm-server:                              [  OK  ][[email protected] init.d]$ pwd/opt/cloudera-manager-5.1.0/etc/init.d

Set to boot

[[email protected] init.d]$ sudo cp cloudera-scm-server /etc/init.d/cloudera-scm-server[[email protected] init.d]$ sudo chkconfig cloudera-scm-server on

Copy the client script to the client

First, SCP compiled the directory to all machines, and then enable the agents end on each machine:

[[email protected] opt]$ sudo scp -r /opt/cloudera-manager-5.1.0/  [email protected]:/opt/[[email protected] opt]$ sudo scp -r /opt/cloudera-manager-5.1.0/  [email protected]:/opt/[[email protected] opt]$ sudo scp -r /opt/cloudera-manager-5.1.0/  [email protected]:/opt/Starting cloudera-scm-agent:                               [FAILED]

Troubleshooting... view the log and find a few libpython items.

/opt/cloudera-manager-5.1.0/lib64/cmf/agent/build/env/bin/python: error while loading shared libraries: libpython2.4.so.1.0: cannot open shared object file: No such file or directory

Flip the wall, omnipotent Google, found that someone is also complaining, because rhel6 default point is python2.6, rhel5 is python2.4 condition, https://groups.google.com/a/cloudera.org/forum! MSG/SCM-users/nz8o8ern1ra/jk0puoahqukj

Hi Simon,On Wed, Dec 21, 2011 at 6:44 AM, Simon <[email protected]> wrote:    Hello,    during the installation on all hosts process I get a few installation    errors:    1. the master (runs the manager) installs all hadoop packages, but    when it comes to Installing Clouder Manager agent package... it says    that it requires libpython2.4.so.1.0()(64bit), of course python2.4 is    not installed but 2.6 is, is there any workaround?    This also occurs if I try to install the agent manually via yumWhat OS are you running?  Is this an Amazon AMI?Usually, when folks report this error, they‘re using RHEL/CentOS 6 (which has python 2.6) but their yum repos are pointed to our RHEL/CentOS 5 repos (5 has python 2.4).

Suddenly, I suddenly realized that my problem was also a version issue. After turning to the previous steps, I downloaded the EL5 version cdh5.1.

Download the offline cloudera Manager installation package, place it in the/OPT directory, decompress it, and rename it

12	[[email protected] opt]$ sudo tar -xzvf cloudera-manager-el5-cm5.1.0_x86_64.tar.gz[[email protected] opt]$ sudo mv cm-5.1.0 cloudera-manager-5.1.0

It is easy to find the cause. Delete

[[email protected] opt]$ sudo rm -rf /opt/cloudera[[email protected] opt]$ sudo rm -rf /opt/cloudera-manager-5.1.0/[[email protected] opt]$ sudo rm -rf /opt/cloudera-manager-el5-cm5.1.0_x86_64.tar.gz[[email protected] opt]$ ls

Again, this time be careful to see el6650) This. width = 650; "src =" http://img.baidu.com/hi/jx2/j_0063.gif "alt =" j_0063.gif "/>, down to master. hadoop/opt/

wget http://archive-primary.cloudera.com/cm5/cm/5/cloudera-manager-el6-cm5.1.1_x86_64.tar.gz

[[email protected] opt]$ sudo tar -xzvf cloudera-manager-el6-cm5.1.0_x86_64.tar.gz

Check after installation

[[email protected] opt]$ lscloudera  cloudera-manager-el6-cm5.1.1_x86_64.tar.gz  cm-5.1.1

Delete the cloudera-SCM created between us

# Confirm the following content in/etc/passwd and disappear after deletion cloudera-SCM: X: 497: 497: cloudera SCM User:/opt/cloudera-manager-5.1.0/run/cloudera-SCM-Server: /bin/false [[email protected] Opt] $ sudo userdel cloudera-SCM # recreate cloudera-SCM user sudo useradd -- System -- home =/opt/cm-5.1.1/run/cloudera- SCM-server -- no-create-home -- shell =/bin/false -- Comment "cloudera SCM user" cloudera-SCM

Modify the configuration and copy the decompressed mysql-connector.

[[email protected] cm-5.1.1]$ sudo vim etc/cloudera-scm-agent/config.ini [General]# Hostname of the CM server.server_host=master.hadoop# Port that the CM server is listening on.server_port=7182[[email protected] cm-5.1.1]$sudo cp /home/hadoop/mysql-connector-java-5.1.32/mysql-connector-java-5.1.32-bin.jar /opt/cm-5.1.1/share/cmf/lib/mysql-connector-java-5.1.32-bin.jar

The previous SCM step, because we have created a database for the first time and do not need to delete the database.

[[email protected] cm-5.1.1]$ sudo ./etc/init.d/cloudera-scm-server startStarting cloudera-scm-server:                              [  OK  ]

Delete the previous boot project and add the new one (no comparison, it is estimated that el6 and EL5 are different)

[[email protected] cm-5.1.1]$ sudo rm /etc/init.d/cloudera-scm-server

[[email protected] init.d]$ sudo cp cloudera-scm-server /etc/init.d/cloudera-scm-server[[email protected] init.d]$ sudo chkconfig cloudera-scm-server on

[[email protected] /]$ sudo /opt/cm-5.1.1/etc/init.d/cloudera-scm-agent startStarting cloudera-scm-agent:                               [  OK  ][[email protected] /]$ sudo chkconfig /etc/init.d/cloudera-scm-agent onerror reading information on service /etc/init.d/cloudera-scm-agent: No such file or directory[[email protected] /]$ sudo chkconfig cloudera-scm-agent on

Open all slave services one after another

This article is from the "the old artisan" blog and will not be reproduced!

Hadoop offline installation of cdh5.1 Chapter 2: cloudera manager and Agent installation