background
from the business development requirements, the big data platform needs to use spark as machine learning, data mining, real-time computing and so on, so decided to use the Cloudera Manager5.2.0 version number and CDH5. Once built Cloudera Manager4.8.2 and CDH4, when building the Cloudera Manager5.2.0 version number, found that the corresponding service Host monitor and service monitor can not configure the external table, Just opened was thought to be configuration error, later found that should be the new version number of the Cloudera storage change mode. Checked a lot of documents, and sure enough, the new version number Service Host Monitor and Servicmonitor e does not need to configure the database, using the built-in storage by default. and cannot be changed .
Overview
Cloudera Manager uses databases to store information on the Cloudera manager configuration, as well as information such As the health of the system or task progress. For quick, simple installations, Cloudera Manager can install and configure a embedded PostgreSQL database as part of the Cloudera Manager installation process. In addition, some CDH services use databases and is automatically configured to use a default database. If you plan to use the embedded and default databases provided during the Cloudera Manager installation, see installation Path a-automated installation by Cloudera Manager.
Although the embedded database is useful for getting started quickly, you can also use your ownPostgreSQL, MySQL, or Oracle databaseFor the Cloudera Manager Server and services, the use databases.
Required Database the Cloudera Manager Server , Activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navig Ator Audit Server , andCloudera Navigator Metadata ServerAll require databases. The type of data contained in the databases and their estimated sizes is as follows:
- Cloudera Manager-contains All the information on services you had configured and their role assignments, all Configu Ration history, commands, users, and running processes. This relatively small database (<100 MB) was the most important-to-back.
- Activity Monitor-contains information about past activities. In large clusters, this database can grow large. Configuring an Activity Monitor database was only necessary if a MapReduce service is deployed.
- Reports manager-tracks Disk utilization and processing activities over time. Medium-sized.
- Hive Metastore-contains Hive Metadata. relatively small.
- Sentry server-contains Authorization metadata. relatively small.
- Cloudera Navigator Audit server-contains auditing information. In large clusters, this database can grow large.
- Cloudera Navigator Metadata server-contains Authorization, policies, and audit report Metadata. relatively small.
The Cloudera Manager Service Host monitor and Service monitor roles has aninternal Datastore. Note This is explained here that the Host monitor and Service Monitor in the CM5 version number, cannot configure the external table, only can use the built-in table.
differs from CM4 version number)
Cloudera Manager provides three different installation methods, method A is self-initiated installation. Methods B and C are manually installed using RPM or tar:
- Path A automatically installs A embedded PostgreSQL database to meet the requirements of the services. This path reduces the number of installation, tasks to complete, and choices to make. In Path A can optionally choose to create external databases foractivity Monitor, Reports Manager, Hive Metastore, Sen Try Server, Cloudera Navigator Audit server, and Cloudera Navigator Metadata server.
- Path B and path C require you-to-create databases for the Cloudera manager Server, Activity Monitor, Reports Manager, Hive Metastore, Sentry server, Cloudera Navigator Audit server, and Cloudera Navigator Metadata server.
Using an external database requires a lot of other input and related work, but Cloudera provides a lot of other compatibility and extensibility that gives you the flexibility to choose your database and configuration.
Of course, you can install a number of different databases in a single set of systems. But this can bring a lot of uncertainties. So Cloudera recommends that you always use the same database.
In very diverse cases, you need to install the corresponding service with database on the same machine, which reduces network IO. Improve overall efficiency. Of course, you can also install the service and database separately on different machines. In large deployments or database administrators need this configuration, such as this scenario, Oracle DBA needs to manage the database independently.
Build the database configuration for the Examiner network. There are specific configuration steps: Build Cloudera Manager Server database for activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navigator Audit server, and Cloudera Navigator Metadata Server build an external database for hue. Oozie Building an external database
In the next article, I will specifically describe the storage mechanism of database in Cloudera Manager. How to configure, tune the top.
original article. Welcome reprint, Reprint please indicate the source
CDH using Cheats (a): Cloudera Manager and managed service database
