Zookeeper application-implementation of centralized Configuration Management System

1. Scenario Description

Refer to Region.

Term: Configuration Management)

A configured service program is deployed on several machines. If the configuration changes, the next operation is to stop the program on all machines, modify the configuration file corresponding to the program on each machine.

There are two problems in this scenario: first, when there are more machines, it is unreasonable to modify the configuration one by one. Second, the configuration is generally read at the startup of the program, so after the configuration is modified, the program should be restarted.

In some cases, configuration management is called "data publishing and subscription". As its name implies, it publishes data to ZK nodes for subscribers to dynamically obtain data and implement centralized management and dynamic updating of configuration information. For example, global configuration information and address list are very suitable for use.

Actual application environment:

1. The index information and the status of machine nodes in the cluster are stored on some designated nodes of ZK for subscription by each client. 2. system logs (processed) are stored. These logs are usually cleared 2-3 days later.

3. some configuration information used in the application is managed in a centralized manner. When the application is started, it takes the initiative to obtain the configuration information once and register a watcher on the node. The configuration will be updated each time and the application will be notified in real time, obtain the latest configuration information.

4. some global variables need to be used in business logic. For example, some message queues in message middleware usually have an offset, which is stored on zk, in this way, each sender in the cluster can know the current sending progress.

5. Some information in the system needs to be dynamically obtained and manually modified. Previously, interfaces are usually exposed, such as JMX interfaces. After ZK is available, you only need to store the information on the zk node.

2. Summary of requirements

A) Solve the Problem of unified configuration of service programs;

B) solve the hot loading problem of service configuration changes; (hot loading: The program changes the configuration and can respond quickly without restarting)

3. Technical Principles

Use the zookeeper feature (Omitted) to save the configuration information in a directory node of zookeeper, and then monitor the configuration status of all application machines to be modified. Once the configuration information changes, each application machine will receive a notification from zookeeper, and then obtain new configuration information from zookeeper and apply it to the system.

A) The znode path can be globally unique as a configuration item;

B) The znode content is used as the configuration item value and always exists in the memory for easy reading;

C) The znode permission serves as the configuration isolation mechanism between projects, which enables security management of project configurations;

The features of zookeeper enable the storage, reading, and monitoring of configurations, what we need to do is to develop a unified control terminal and provide an interface for the program to read configurations.

A) Centralized web control terminal: including adding, deleting, modifying, querying, and deploying configuration information;

B) classes and interfaces for Java to read configuration items;

C) C ++ reads the configured classes and interfaces;

4 Architecture

5. Configure the Storage Structure

Layer 3: string of the XML file.

5 Advantages

A) Unified configuration management to facilitate O & M;

B) Hot loading of service programs;

C) Configure distributed storage, which is stable and reliable;

6. Implementation

Function class confmng provides functions for loading xml configuration files to the zk system, reading functions for configuring map, and providing an abstract function that needs to be implemented by the user.

// Parse XML to save keyValue to map
Abstract Public Map <?, ?> Parserxml ();

 

The user program is used as follows:

Inherits the confmng class and implements parserxml (); Interface

Read configuration content.

 

Principle: The centralized configuration management class is designed to open the method for reading configuration XML to users, so as to achieve general configuration. It may be seamlessly integrated with the original program.