UBUNTU14 use HDP to install Hadoop

Source: Internet
Author: User
Tags chmod flush parent directory postgresql file permissions mysql database root directory hortonworks
1. Prepare
1) Install Java
Copy jdk-7u80-linux-x64.tar.gz to this machine, unpack/test
Vi/etc/profile
Export java_home=/test/jdk1.7.0_80
Export classpath=.: $JAVA _home/lib: $JAVA _home/jre/lib: $CLASSPATH
Export path= $JAVA _home/bin: $JAVA _home/jre/bin: $PATH
Source/etc/profile
2) Change the host name
Vi/etc/hostname
Vi/etc/hosts
127.0.1.1 Hdp01.test changed to 173.168.0.121 hdp01.test
173.168.0.121 Hdp01.test
173.168.0.122 Hdp02.test
3 sudo apt-get install NTP
Service NTP start
4) Installation Postgres,mysql
First step: Install PostgreSQL, phppgadmin and Apache2
sudo apt-get-y install PostgreSQL postgresql-contrib phppgadmin


Step Two: Configure the PostgreSQL user
Su-postgres
Psql
\password
\q


3rd Step: Configure PostgreSQL
Vim/etc/postgresql/9.1/main/postgresql.conf



Installing MySQL on Ubuntu is simple and requires a few commands to complete.
sudo apt-get install Mysql-server
sudo apt-get install mysql-client Libmysql-java
sudo apt-get install Libmysqlclient-dev
sudo apt-get install Libmysql-java


2. Switch to root, set SSH
1) Install SSH
Dpkg--list | grep ssh
sudo apt-get install Openssh-server
2) Generate RSA key
Ssh-keygen-t Rsa-p '
Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
3) change root, ssh, and file permissions
chmod o+w ~/
chmod 700-r ~/.ssh
chmod ~/.ssh/authorized_keys
chmod 755/root
4) Change Sshd_config
Vi/etc/ssh/sshd_config
Change the Permitrootlogin Prohibit-password in the/etc/ssh/sshd_config file to Permitrootlogin Yes (or remove the Permitrootlogin Prohibit-password can also be, for security can be backed up, comment out can be
Sudo/etc/init.d/ssh restart
5)
VI ~/.ssh/config
Host 173.168.0.121
User [Your_user_name]
6 on the Slave machine:
Under the root directory
mkdir. SSH
Copy id_rsa.pub to. SSH on Master
Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod o+w ~/
chmod 700-r ~/.ssh
chmod ~/.ssh/authorized_keys
chmod 755/root


3. Download packages from the following three locations, which can be downloaded directly on Windows with HTTP
Nohup wget-c http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.2.1.0/ ambari-2.2.1.0-ubuntu14.tar.gz > 1.log 2>&1 &
Nohup wget-c http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/2.4.0.0/ hdp-2.4.0.0-ubuntu14-deb.tar.gz > 1.log 2>&1 &
Nohup wget-c http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.20/repos/ubuntu14/ hdp-utils-1.1.0.20-ubuntu14.tar.gz > 2.log 2>&1 &
1 copy these three files to/var/www/html/hadoop
Cd/var/www/html
mkdir Hadoop
2 run HTTP service, unzip three files to/var/www/html/hadoop


4. Other machines in the net, add official source
sudo wget-nv Http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.2.1.0/ambari.list-O/etc/apt/ Sources.list.d/ambari.list
sudo apt-key adv--recv-keys--keyserver keyserver.ubuntu.com B9733A7A07513CAD
5. Modify official Source configuration file (Dist Superior directory)
sudo vi/etc/apt/sources.list.d/ambari.list


#VERSION_NUMBER =2.2.1.0-161
#deb http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/2.2.1.0 Ambari Main


Deb Http://173.168.0.121/hadoop/AMBARI-2.2.1.0/ubuntu14/2.2.1.0-161/Ambari Main


6. Update the source and install the program
sudo apt-get update
Master:sudo Apt-get Install Ambari-server
Slave:sudo Apt-get Install Ambari-agent


7. Configure the Ambari service
1 sudo ambari-server setup
Enter the value of the Java_home, and the other default
2) ambari-server Setup--jdbc-db=mysql--jdbc-driver=/usr/share/java/mysql.jar


8. Start Service
Master:sudo ambari-server Start
Slave:sudo ambari-agent Start
sudo vi/etc/ambari-agent/conf/ambari-agent.ini
Hostname=hdp01.test


9. Configure the Hive,ambari database
1)
CREATE USER ' hive ' @ ' localhost ' identified by ' hive ';
GRANT all privileges in *.* to ' hive ' @ ' localhost ';
CREATE USER ' hive ' @ '% ' identified by ' hive ';
GRANT all privileges in *.* to ' hive ' @ '% ';
CREATE USER ' hive ' @ ' hdp01.test ' identified by ' hive ';
GRANT all privileges in *.* to ' hive ' @ ' hdp01.test ';
FLUSH privileges;
Create DATABASE Hive;


CREATE USER ' ambari ' @ '% ' identified by ' bigdata ';
GRANT all privileges in *.* to ' ambari ' @ '% ';
CREATE USER ' ambari ' @ ' localhost ' identified by ' bigdata ';
GRANT all privileges in *.* to ' ambari ' @ ' localhost ';
CREATE DATABASE Ambari;
Use Ambari;
Source/var/lib/ambari-server/resources/ambari-ddl-mysql-create.sql;


CREATE USER ' oozie ' @ ' localhost ' identified by ' Oozie ';
GRANT all privileges in *.* to ' oozie ' @ ' localhost ';
CREATE USER ' oozie ' @ '% ' identified by ' Oozie ';
GRANT all privileges in *.* to ' oozie ' @ '% ';
CREATE USER ' Oozie ' @ ' hdp01.test ' identified by ' Oozie ';
GRANT all privileges in *.* to ' Oozie ' @ ' hdp01.test ';
FLUSH privileges;
Create DATABASE Oozie;


2)
sudo vi/etc/mysql/my.cnf
Add ' # ' to comment out "bind-address = 127.0.0.1"
sudo service MySQL restart


10. Accessing the Web interface
http://ip:8080
Use the default Admin/admin login.
1) Click on the top left corner icon or choose Launch Install Wizard
2 The second step select Stack, where the source is selected, fill in the path (Dist parent directory)
http://173.168.0.121/hadoop/HDP/ubuntu14/2.x/updates/2.4.0.0/
http://173.168.0.121/hadoop/HDP-UTILS-1.1.0.20/repos/ubuntu14/


11. In step 3rd


1) Only fill hdp01.test
sudo cat Id_rsa
Copy private key To box, click Next, see Sucess, click Back
2 The Hdp02.test also fill in
Choose to install ambari-agent manually
When you see all two sucess, click Next


12. Prohibition of HDP (transparent hugepages)
Vi/etc/default/grub, add options in Grub_cmdline_linux transparent_hugepage=never
Grub-mkconfig-o/boot/grub/grub.cfg

Reboot


Hive, Oozie Select the existing MySQL database, set the database password





Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.