Installation Guide for Ambari and HDP

Installation Guide for Ambari and HDP

The Big Data Platform involves many software products. If you just download the software package from Hadoop and manually configure the file, it is not that intuitive and easy.

Ambari provides an option to install and manage hadoop clusters in a graphical manner. Ambari won't introduce it anymore. The Ambari software is intuitive, but the installation experience is poor. It is better to install and control it on your own. After an error or failure occurs, "retry" seems to be the only method.

Now we have started to install it. During the installation process, I will introduce the pitfalls I encountered and how to crawl them out again.

1. log on to the official page first.

Https://docs.hortonworks.com/index.html

2. Select the desired version.

3. Click installation.

4. Select Install Ambari

5. Go to the document page.

Official installation steps. Note: select the installation document of the version you want to install.

1 https://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/download_the_ambari_repo_lnx6.html

6. According to the above steps, you can enter the configuration yum source. Because the source is abroad and the speed is too slow, I will first download the software package to the local device.

Wget-nv http://public-repo-1.hortonworks.com/ambari/CentOS6/2.x/updates/2.4.2.0/ambari.repo-O/etc/yum. repos. d/ambari. repo
Yum repolist
Yum install-downloadonly ambari-server -- downloaddir =/tmp/
Yum install -- downloadonly HDP-2.5.0.0 -- downloaddir =/tmp/
Yum install -- downloadonly HDP-2.5. * -- downloaddir =/tmp/
[Root @ db_monitor] # ll
-Rw-r -- 1 root 5996696005 Aug 27 2016 HDP-2.5.0.0-centos6-tars-tarball.tar.gz
-Rw-r -- 1 root 833424206 Oct 26 HDP-UTILS-1.1.0.21-centos6.tar.gz

7. After downloading to a local machine, create an HTTP service on a server for subsequent installation.

Yum install httpd-y
# Service httpd restart
# Chkconfig httpd on

Add the downloaded HDP package to the/var/www/html/directory.

You can access it through the web page.

The preparation is complete and installation is started.

8. Configure ssh password-free login for the node. You can only access other nodes on the server where ambari is installed.

$ Ssh-keygen-t rsa-f ~ /. Ssh/id_rsa

Press enter.

Copy the key to all nodes (only ambari does not need a password to access other servers, but a password is required for reverse access)

Ssh-copy-id server2
Ssh-copy-id server3
Ssh-copy-id server4

9. Configure ambaris and disable selinux first (you can disable selinux by modifying the configuration file when I use ansible)

Ansible all-m selinux-a 'state = disabled'

10. Disable Firewall

Ansible all-m service-a 'name = iptables state = stopped enabled = off'

11. Install JDK and ambari automatically, but the installation speed is slow. We recommend that you install it on your own. All nodes must be installed.

Download JDK (intranet address)

Wget http://10.21.xx.xx/deploy/jdk1.7.0_79.tar.gz

Install JDK on all nodes

# Tar-xzvf jdk1.7.0_79.tar.gz-C/usr/local

Add the following information to. bash_profile

Export JAVA_HOME =/usr/local/jdk1.8.0 _ 144
Export JRE_HOME =/usr/local/jdk1.8.0 _ 144/jre
Export CLASSPATH =.: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jar: $ JRE_HOME/lib: $ CLASSPATH
Export PATH = $ JAVA_HOME/bin: $ PATH
Source/etc/profile

12. Now the installation of ambaris has started.

(Note: If you choose to install jdk on your own and later use kerberos authentication, you still need JCE. Otherwise, KDC cannot be connected normally. This was discovered only when I encountered problems later. In addition, you can only download JCE 1.8 from the Oracle official website. 1.7 and below, you need a metalink account to download. Therefore, if the network speed is acceptable, we recommend that you select 1 or 2 to install JDK)

[Root @ ip-10-21-23-29 ambari] # ambari-server setup
Using python/usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'Disabled'
Customize user account for ambari-server daemon [y/n] (n )?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7 <important
[3] Custom JDK
========================================================== ==============================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for login ing Kerberos security. If you plan to use Kerberos, please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME:/usr/local/jdk1.7.0 _ 79
Validating JDK on Ambari Server... done.
Completing setup...
Processing database...
Enter advanced database configuration [y/n] (n )? ----- If you do not want to install the default postgres database, select Y
Processing database...
Default properties detected. Using built-in database.
Processing ing ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database: [OK]
About to start PostgreSQL
Refreshing local database...
Connecting to local database... done.
Processing PostgreSQL...
Restarting PostgreSQL
Extracting system views...
Ambari-admin-2.4.1.0.22.jar
.......
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

14. Start ambari

[Root @ ip-10-21-23-29 ambari] # ambari-server start
Using python/usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at/var/lib/ambari-server/resources...
Ambari database consistency check started...
No errors were found.
Ambari database consistency check finished
Server PID at:/var/run/ambarriers-server/ambari-server.pid
Server out at:/data1/log/ambari-server/ambari-server.out
Server log at:/data1/log/ambari-server/ambari-server.log
Waiting for server start ....................
Ambari Server 'start' completed successfully.

15. But now, the web page cannot be opened. Install the httpd service.

[Root @ ip-10-21-23-29 ambari] # service httpd status
Httpd: unrecognized service

Httpd needs to be installed

[Root @ ip-10-21-23-29 ambari] # yum install httpd
[Root @ ip-10-21-23-29 ambari] # service httpd status
Httpd is stopped
[Root @ ip-10-21-23-29 ambari] # service httpd start
Starting httpd: [OK]

16. At the beginning of page installation, many people on the Internet have written about how to install it.

Select the prepared HTTP connection. The path is as follows:

HDP Base URL	Http: // <web. server>/hdp/HDP/<OS>/2.x/ updates/<latest. version>
HDP-UTILS Base URL	Http: // <web. server>/hdp/HDP-UTILS-<version>/repos/<OS>

17. Fill in the host name. Make sure to fill in the complete host name, including the suffix. IP addresses cannot be used instead. Check whether/etc/hosts is properly parsed.

18. Install the ambari agent. If the agent fails, select the failed entry. You can "retry"

19. Even if it succeeds, we need to solve the problem if there is a warning below. Skipping is definitely not a wise choice. I encountered some problems.

Run the following command to disable THP, And Then Rerun checks to ensure that all of them are resolved.

Echo never>/sys/kernel/mm/RedHat_transparent_hugepage/enabled
Echo never>/sys/kernel/mm/redhat_transparent_hugepage/defrag
Echo "transparent_hugepage = never">/etc/grub. conf

20. Start installation after the problem is solved

21. Maybe you're thinking, it's done so quickly? You are wrong. I was tortured for a week before seeing the following interface. Besides, do you think my interface is too simple? Yes, I only deployed zookeeper and HDFS. Because the error is too many times. I can only try to install it from the simplest one. Reduce the probability of a problem.

22. After the operation is successful, you can add the service to gradually increase the resources you need.

23. service addition Process

24. Check it out. This is the interface for successful installation. It's not easy to see it.

I would like to complain about the inconvenience caused by Ambari.

If the installation fails, you can only retry the installation, or even reinstall the installation. The previously installed services will affect your next installation.

I was unable to bear all kinds of failures and planned to re-install the software by another name. However, the software that failed the last installation was still there and could not be re-installed. You can only Uninstall a previously installed service.

The problem is that Ambari cannot be uninstalled! Cannot uninstall! Cannot uninstall! You cannot skip this step if you restart the service or host. Still persistently stop at the wrong page.

Finally, after referring to others' blogs, find the command deletion method.

1. query Resources

Curl-u admin: admin-H "X-Requested-By: ambari"-x get http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/services/

2. delete a resource

Curl-u admin: admin-H "X-Requested-By: ambari"-x delete http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/services/KAFKA

3. If the deletion fails, stop before deleting

# Curl-u admin: admin-H "X-Requested-By: ambari"-x delete http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/services/APPCONFIGURATION
{
"Status": 500,
"Message": "org. apache. ambari. server. controller. spi. systemException: An internal system exception occurred: Cannot remove beta_eu/APPCONFIGURATION. one or more host components are in a non-removable state."

4. Stop the service

[Root @ ip-10-21-23-29 ~] # Curl-u admin: admin-H "X-Requested-By: ambari"-x put-d' {"RequestInfo": {"context": "Stop Service "}, "Body": {"ServiceInfo": {"state": "INSTALLED" }}'10.21.23.29: 8080/api/v1/clusters/beta_eu/services/APPCONFIGURATION
{"Href": "http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/requests/24 ",
"Requests ":{
"Id": 24,
"Status": "Accepted "}

5. delete a service

[Root @ ip-10-21-23-29 ~] # Curl-u admin: admin-H "X-Requested-By: ambari"-x delete http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/services/APPCONFIGURATION

6. Check again. No service is available.

[Root @ ip-10-21-23-29 ~] # Curl-u admin: admin-H "X-Requested-By: ambari"-x get http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/services/
Curl: (6) Couldn't resolve host 'ambari "'
{
"Href": "http: // 10.21.23.29: 8080/api/v1/clusters/beta_eu/services /",
"Items": []

This article permanently updates link: https://www.bkjia.com/Linux/2018-03/151489.htm