ClouderaManager installation and deployment Manual

1. Prepare Ubuntu offline source and configure/etc/apt/source. list
Prepare Cloudera Hadoop offline source and configure/etc/apt/source. list. d/cloudera. list.
 Synchronize/etc/hosts of the Cluster
 Ssh password-free logon. The machine on which cloudera-scm-server is located can log on to other cloudera-scm-agent machines without a password. The root user password on all machines is the same.
2. System Environment OS: ubuntu12.04
CDH: cdh3u6
Cloudera manager: 4.8.2
3. Database Configuration 3.1 install and configure MySQLCloudera Manager supports three database types: MySQL, PostgreSQL, and Oracle. We chose to use MySQL.
Apt-get install mysql-server
Service mysql start
Create the mysql_cloudera_manager.cnf file under/etc/mysql/conf. d and add the following content:
[Mysqld]
Transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# To do so, uncomment this line:
# Symbolic-links = 0


Key_buffer = 16 M
Key_buffer_size = 32 M
Max_allowed_packet = 16 M
Thread_stack = 256 K
Thread_cache_size = 64
Query_cache_limit = 8 M
Query_cache_size = 64 M
Query_cache_type = 1
# Important: see accessing the Databases and Setting max_connections
Max_connections = 550


# Log-bin shoshould be on a disk with enough free space
Log-bin =/x/home/mysql/logs/binary/mysql_binary_log


# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
Binlog_format = mixed


Read_buffer_size = 2 M
Read_rnd_buffer_size = 16 M
Sort_buffer_size = 8 M
Join_buffer_size = 8 M


# InnoDB settings
Innodb_file_per_table = 1
Innodb_flush_log_at_trx_commit = 2
Innodb_log_buffer_size = 64 M
Innodb_buffer_pool_size = 4G
Innodb_thread_concurrency = 8
Innodb_flush_method = O_DIRECT
Innodb_log_file_size = 512 M


[Mysqld_safe]
Log-error =/var/log/mysqld. log
Pid-file =/var/run/mysqld. pid
3.2 configure innodbmv/var/lib/mysql/ib_logfile *~
Restart mysql
Service mysql restart
3.3 install mysql-connectorapt-get install libmysql-java
3.4 List of databases to be created when creating a database:
Amon smon ran hmon hive
Mysql-u root-p iflytek
Create database amon default character set utf8;
Grant all on amon. * TO 'amon' @ '%' identified by 'amon _ password ';
Create database smon default character set utf8;
Grant all on smon. * TO 'smon' @ '%' identified by 'smon _ password ';
Create database rman default character set utf8;
Grant all on rman. * TO 'rman '@' % 'identified by 'rman _ password ';
Create database hmon default character set utf8;
Grant all on hmon. * TO 'hmon '@' % 'identified by 'hmon _ password ';
Create database hive default character set utf8;
Grant all on hive. * TO 'hive' @ '%' identified by 'hive _ password ';
Back up database
Mysqldump-piflytek amon>/tmp/amon-backup. SQL
And other cluster Databases
3.5 install Cloudera Manager to add the Cloudera Manager Source
Create the/etc/apt/source. list. d/cloudera-manager.list file and add
Deb [arch = amd64] http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib
Deb-src http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh precise-cdh4 contrib
Deb http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib
Deb-src http://archive.cloudera.com/cm4/ubuntu/precise/amd64/cm precise-cm4 contrib
Save and download the key
Curl-s http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh/archive.key | sudo apt-key add-
If the preceding command fails to be executed, download the key file and import it yourself. Apt-key add [file path]
Run
Apt-get install cloudera-manager-daemons cloudera-manager-server
Run
Apt-get installation cloudera-manager-agent cloudera-manager-daemons
3.6 prepare the database for cloudera manager for execution/usr/share/cmf/schema/scm_prepare_database.sh database-type [options] database-name username password
For example
./Scm_prepare_database.sh mysql amon root xxx
./Scm_prepare_database.sh mysql smon root xxx
./Scm_prepare_database.sh mysql hmon root xxx
./Scm_prepare_database.sh mysql rman root xxx
./Scm_prepare_database.sh mysql hive root xxx
3.7 install HDFS and specify a machine for namendoe
Apt-get install namenode
Run the following command on the specified DataNode machine:
Apt-get install datanode
3.8 start the cloudera manager Server and agent to create some log storage directories on all machines:
Mkdir-p/var/log/cloudera-scm-headlamp
Mkdir-p/var/log/cloudera-scm-firehose
Mkdir-p/var/log/cloudera-scm-alertpublisher
Mkdir-p/var/log/cloudera-scm-eventserver
Mkdir-p/var/lib/cloudera-scm-headlamp
Mkdir-p/var/lib/cloudera-scm-firehose
Mkdir-p/var/lib/cloudera-scm-alertpublisher
Mkdir-p/var/lib/cloudera-scm-eventserver
Modify owner after creation
Chown-R cloudera-scm: cloudera-scm/var/lib/cloudera-scm -*
Chown-R cloudera-scm: cloudera-scm/var/log/cloudera-scm -*
If you want to use the tar package for installation, chown needs to go to the cloudera-scm user group. Chown is not required for apt-get installation. Root is used by default.
Modify JDK configurations of all agents
Add/etc/default/cloudera-scm-agent
Export CMF_AGENT_JAVA_HOME =/usr/local/jdk6
Run the following command on the Server:
Service cloudera-scm-server start
Log on to the agent machine and run the following commands:
Service cloudera-scm-agent start
3.9 log on to the web terminal and deploy http: // ServerIP: 7180 with the scm-server machine.
Username: admin
Password: admin
3.10 set the Database Host Name: localhost, User name: root, the five above the database name, password: xxxx, that is, the MySQL database password.
4. web Client usage