UBUNTU14 installing Hadoop with HDP

Source: Internet
Author: User
Tags chmod flush parent directory postgresql ssh file permissions mysql database hortonworks
1. Preparation
1) Installing Java
Copy jdk-7u80-linux-x64.tar.gz to this machine, unzip/test
Export java_home=/test/jdk1.7.0_80
Export classpath=.: $JAVA _home/lib: $JAVA _home/jre/lib: $CLASSPATH
Export path= $JAVA _home/bin: $JAVA _home/jre/bin: $PATH
2) Change host name
Vi/etc/hosts Hdp01.test changed to hdp01.test Hdp01.test Hdp02.test
3) sudo apt-get install NTP
Service NTP start
4) Install Postgres,mysql
First step: Installing PostgreSQL, Phppgadmin and Apache2
sudo apt-get-y install PostgreSQL postgresql-contrib phppgadmin

Step Two: Configure PostgreSQL users

3rd Step: Configure PostgreSQL

Installing MySQL on Ubuntu is very simple and requires just a few commands to complete.
sudo apt-get install Mysql-server
sudo apt-get install mysql-client Libmysql-java
sudo apt-get install Libmysqlclient-dev
sudo apt-get install Libmysql-java

2. Switch to root, set up SSH
1) Install SSH
Dpkg--list | grep ssh
sudo apt-get install Openssh-server
2) Generate RSA key
Ssh-keygen-t Rsa-p '
Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
3) change root, ssh, and file permissions
chmod o+w ~/
chmod 700-r ~/.ssh
chmod ~/.ssh/authorized_keys
chmod 755/root
4) Change Sshd_config
Change Permitrootlogin Prohibit-password in the/etc/ssh/sshd_config file to permitrootlogin Yes (or delete permitrootlogin Prohibit-password can also, for security can be backed up, commented out can be)
Sudo/etc/init.d/ssh restart
VI ~/.ssh/config
User [Your_user_name]
6) on the Slave machine:
Under the root directory
mkdir. SSH
Copy the id_rsa.pub on master to. SSH
Cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod o+w ~/
chmod 700-r ~/.ssh
chmod ~/.ssh/authorized_keys
chmod 755/root

3. Download the package from the following three locations, which can be downloaded directly on Windows using HTTP
Nohup wget-c http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/ ambari- > 1.log 2>&1 &
Nohup wget-c http://public-repo-1.hortonworks.com/HDP/ubuntu14/2.x/updates/ hdp- > 1.log 2>&1 &
Nohup wget-c http://public-repo-1.hortonworks.com/HDP-UTILS- hdp-utils- > 2.log 2>&1 &
1) Copy these three files to/var/www/html/hadoop
mkdir Hadoop
2) Run HTTP service, extract three files to/var/www/html/hadoop

4. Other machines in the network, add the official source
sudo wget-nv Http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/ Sources.list.d/ambari.list
sudo apt-key adv--recv-keys--keyserver keyserver.ubuntu.com B9733A7A07513CAD
5. Modify the official source configuration file (dist parent directory)
sudo vi/etc/apt/sources.list.d/ambari.list

#deb http://public-repo-1.hortonworks.com/ambari/ubuntu14/2.x/updates/ Ambari Main

Deb Http:// Main

6. Update the source and install the program
sudo apt-get update
Master:sudo Apt-get Install Ambari-server
Slave:sudo Apt-get Install Ambari-agent

7. Configuring the Ambari Service
1) sudo ambari-server setup
Enter a value for Java_home, other default
2) ambari-server Setup--jdbc-db=mysql--jdbc-driver=/usr/share/java/mysql.jar

8. Start the service
Master:sudo ambari-server Start
Slave:sudo ambari-agent Start
sudo vi/etc/ambari-agent/conf/ambari-agent.ini

9. Configuring the Hive,ambari Database
CREATE USER ' hive ' @ ' localhost ' identified by ' hive ';
GRANT all privileges on * * to ' hive ' @ ' localhost ';
CREATE USER ' hive ' @ '% ' identified by ' hive ';
GRANT all privileges on * * to ' hive ' @ '% ';
CREATE USER ' hive ' @ ' hdp01.test ' identified by ' hive ';
GRANT all privileges on * * to ' hive ' @ ' hdp01.test ';
FLUSH privileges;
Create DATABASE Hive;

CREATE USER ' ambari ' @ '% ' identified by ' bigdata ';
GRANT all privileges on * * to ' ambari ' @ '% ';
CREATE USER ' ambari ' @ ' localhost ' identified by ' bigdata ';
GRANT all privileges on *. * to ' ambari ' @ ' localhost ';
Use Ambari;

CREATE USER ' oozie ' @ ' localhost ' identified by ' Oozie ';
GRANT all privileges on *. * to ' oozie ' @ ' localhost ';
CREATE USER ' oozie ' @ '% ' identified by ' Oozie ';
GRANT all privileges on * * to ' oozie ' @ '% ';
CREATE USER ' Oozie ' @ ' hdp01.test ' identified by ' Oozie ';
GRANT all privileges on * * to ' oozie ' @ ' hdp01.test ';
FLUSH privileges;
Create DATABASE Oozie;

sudo vi/etc/mysql/my.cnf
Add ' # ' comment out of "bind-address ="
sudo service MySQL restart

10. Accessing the Web interface
Use the default Admin/admin login.
1) Click the top left icon or select Launch Install Wizard
2) Step two select Stack, fill in the path (Dist parent directory) where you choose the source

11. In the 3rd step

1) Fill Hdp01.test only
sudo cat Id_rsa
Copy the private key into the box, click Next, after you see sucess, click Back
2) Fill in the Hdp02.test, too.
Select Manual Installation Ambari-agent
After you see all two sucess, click Next

12. Prohibition of HDP (Transparent hugepages)
Vi/etc/default/grub, add options in Grub_cmdline_linux transparent_hugepage=never


Hive, Oozie Select an existing MySQL database, set the database password

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.