background
from the business development requirements, the big data platform needs to use spark as machine learning, data mining, real-time computing and so on, so decided to use Cloudera Manager5.2.0 version and CDH5. Previously built Cloudera Manager4.8.2 and CDH4, when building the Cloudera Manager5.2.0 version, found that the corresponding service Host monitor and service monitor can not configure the external table, Just opened was thought to be a configuration error, later found that should be the new version of the Cloudera storage change mode. Looked up a lot of documents, and sure enough, that the new version Service Host Monitor and Servicmonitor e does not require a configuration database, which uses the built-in storage mode by default and cannot be modified .
Overview
Cloudera Manager uses databases to store information on the Cloudera manager configuration, as well as information such As the health of the system or task progress. For quick, simple installations, Cloudera Manager can install and configure a embedded PostgreSQL database as part of the Cloudera Manager installation process. In addition, some CDH services use databases and is automatically configured to use a default database. If you plan to use the embedded and default databases provided during the Cloudera Manager installation, see installation Path a-automated installation by Cloudera Manager.
Although the embedded database is useful for getting started quickly, you can also use your ownPostgreSQL, MySQL, or Oracle databaseFor the Cloudera Manager Server and services, the use databases.
Required Database the Cloudera Manager Server , Activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navig Ator Audit Server , andCloudera Navigator Metadata ServerAll require databases. The type of data contained in the databases and their estimated sizes is as follows:
- Cloudera Manager-contains All the information on services you had configured and their role assignments, all Configu Ration history, commands, users, and running processes. This relatively small database (<100 MB) was the most important-to-back.
- Activity Monitor-contains information about past activities. In large clusters, this database can grow large. Configuring an Activity Monitor database was only necessary if a MapReduce service is deployed.
- Reports manager-tracks Disk utilization and processing activities over time. Medium-sized.
- Hive Metastore-contains Hive Metadata. relatively small.
- Sentry server-contains Authorization metadata. relatively small.
- Cloudera Navigator Audit server-contains auditing information. In large clusters, this database can grow large.
- Cloudera Navigator Metadata server-contains Authorization, policies, and audit report Metadata. relatively small.
The Cloudera Manager Service Host monitor and Service monitor roles has aninternal Datastore. (Note that this is explained here that the Host monitor and Service Monitor in the CM5 version, cannot configure external tables, only use built-in tables.) differs from CM4 version)
Cloudera Manager offers three different installation methods, method A is an automated installation, and methods B and C are manually installed using RPM or tar:
- Path A automatically installs A embedded PostgreSQL database to meet the requirements of the services. This path reduces the number of installation, tasks to complete, and choices to make. In Path A can optionally choose to create external databases foractivity Monitor, Reports Manager, Hive Metastore, Sen Try Server, Cloudera Navigator Audit server, and Cloudera Navigator Metadata server.
- Path B and path C require you-to-create databases for the Cloudera manager Server, Activity Monitor, Reports Manager, Hive Metastore, Sentry server, Cloudera Navigator Audit server, and Cloudera Navigator Metadata server.
The use of external databases requires more input and related work, but Cloudera provides more compatibility and extensibility, allowing you to flexibly select databases and configurations. Of course you can install a number of different databases in a set of systems, but this can lead to a lot of uncertainties, so Cloudera recommends that you always use the same database.
In many cases, you need to install the corresponding service with database on the same machine, reducing network IO and improving overall efficiency. Of course, you can also install service and database separately on different machines, in large deployments or database administrators need such a configuration, such as this scenario, Oracle DBA needs to manage the database independently.
Configure the Database configuration reference website with detailed configuration steps: Build Cloudera Manager Server database for activity Monitor, Reports Manager, Hive Metastore, Sentry Server, Cloudera Navigator Audit Server, and Cloudera Navigator Metadata Server build external database for Hue,oozie
In the next article, I'll describe in detail the storage mechanism of database in Cloudera Manager, how to configure it, and how to tune it.
original articles, welcome reprint, reproduced please indicate the source
CDH using Cheats (a): Cloudera Manager and Managed service database
