Zookeeper scenario practices: (2) centralized Configuration Management

1. Basic Introduction

In a distributed environment, multiple peer-to-peer programs may read the same configuration file. The program can be deployed on multiple machines, you need to deploy a configuration file for the machine where the program is deployed. Once you want to modify the configuration, it will be very troublesome. You need to modify multiple configuration files, which may cause inconsistency.
The idea of centralized configuration management is to centrally publish configuration data to zookeeper nodes for subscribers to dynamically obtain data. Centralized configuration management and dynamic updates are implemented. It can be simply understood that the configuration data is separated from the program.

2. Scenario Analysis (1). Centralized Configuration Management

Generally, most projects have agreed configuration file formats, such as INI and XML. Generally, there is a corresponding resolution Library Class. The basic working mode of this DNS Library Class is:

1. Read files (open)
2. Parsing a file (PARSE)
3. Provide external parameters (get)

If we put all the file content on a node of zookeeper. the resolution class downloads all the configuration data to the local device. After the resolution is complete, you can use a small change to complete the centralized configuration management requirements.

1. Read data from the corresponding path on zookeeper (read)
2. Parsing a file (PARSE)
3. Provide external parameters (get)

(2). Dynamic Update

Dynamic Update allows you to obtain the updated configuration in real time without restarting the program. In a single-host environment, such configuration data is usually stored in the database. You only need to update the database to modify the configuration.
To use zookeeper, You need to register a watcher for the node to monitor whether the configuration data has changed. If so, a new parsing class is called to re-parse the configuration data.
I personally think this feature can be implemented using zookeeper, but not all configurations require this feature. This feature is suitable for configuration-sensitive scenarios where configuration needs to be updated in real time.

3. hands-on practice

Here, I only implement the centralized configuration management function, and do not implement dynamic updates. If necessary, you can try to implement it by yourself.
Since I used to parse the Library Class of an INI file, I will directly change it here. According to the scenario analysis, you only need to modify the OPEN function.

View the original Open Function

/* Change the file name to the address and path */IntINIFILE: open (ConstString & filename) {release (); fname _ = filename; inisection * Section = NULL;/* The Data Reading method needs to be modified */File * fp = fopen (filename. c_str (), "R ");If(FP = NULL ){Return-1;} string line; string comment;// Add a default segmentSection =NewInisection (); sections _ [""] = Section;/* You need to modify the method for obtaining rows */    While(Getline (line, FP)> 0 ){...// Ignore the parsing of a single row} Fclose (FP );Return0 ;}

We have three main changes: input parameters, fopen and Getline. The following are the modified open functions.

/* Modify the input parameter. Host is the IP address and port address of zookeeper, and filepath is the path of configuration data */IntINIFILE: open2 (ConstString & host,ConstString & filepath) {release (); fname _ = filepath; inisection * Section = NULL;CharFP [2048] = {0 };/* Read from zookeeper */Zkopen (host, filepath, FP,Sizeof(FP ));If(FP [0] = 0 ){Return-1;} string line; string comment;// Add a default segmentSection =NewInisection (); sections _ [""] = Section;Char* P = FP;/* Adjust the input parameter of Getline */    While(Getline2 (line, p)> 0 ){...// Ignore the parsing of a single row}Return0 ;}

Zkopen reads data from the zookeeper node and saves it to FP. The Code is as follows:

String zkopen (ConstString & host,ConstString & filepath,Char* FP,IntLen ){IntTimeout = 30000;CharPath_buffer [512];IntBufferlen =Sizeof(Path_buffer );CharConf_data [2048];IntConf_len =Sizeof(Conf_data); zoo_set_debug_level (zoo_log_level_warn );// Set the log level to avoid other informationZhandle_t * zkhandle = zookeeper_init (host. c_str (), null, timeout, 0 ,(Char*) "Monitor test", 0 );If(Zkhandle = NULL) {fprintf (stderr, "error when connecting to zookeeper servers... \ n"); exit (exit_failure );}IntRet = zoo_get (zkhandle, filepath. c_str (), 0, conf_data, & conf_len, null );If(Ret! = Zok) {fprintf (stderr, "failed to get the data of PATH % s! \ N ", filepath. c_str (); conf_data [0] = 0;} zookeeper_close (zkhandle); strncpy (FP, conf_data, Len );ReturnConf_data ;}

Next we will compare the call changes.
Original call method:

  /** Read Test **/INIFILE ini; ini. Open (g_filepath );// Obtain the value of a specified item in a specified segment    IntRet = 0; string db_name = ini. getstringvalue ("common", "DB", RET); string db_passwd = ini. getstringvalue ("common", "passwd", RET );

Current call method:

  /** Read Test **/INIFILE ini; ini. open2 (g_host, g_filepath );/* Only changes here */    // Obtain the value of a specified item in a specified segment    IntRet = 0; string db_name = ini. getstringvalue ("common", "DB", RET); string db_passwd = ini. getstringvalue ("common", "passwd", RET );

It can be seen from the above that the configuration is still very easy, and the changes to the program are very small.
For the code, see https://github.com/Winnerhust/ZooKeeper-Exam/tree/master/Config.

5. Tips

Note that there are usually many line breaks in the configuration file, while the zookeeper client command line does not support character escape. For example, you want to save the content of the configuration file test. ini to zookeeper. The content of the file is as follows.
[Common]
DB = MySQL
Passwd = root

You may enter:
[zk: 172.17.0.36:2181(CONNECTED) 39] create /Conf/test.ini [COMMON]\nDB=mysql\nPASSWD=root\n
The result is different from what we want:
[zk: 172.17.0.36:2181(CONNECTED) 43] get /Conf/test.ini3[COMMON]\nDB=mysql\nPASSWD=root\n

Zookeeper does not escape strings. Therefore, you cannot use the zookeeper client to directly upload the configuration file. Therefore, I added an upload configuration function in the code. You only need to set the previous Parameter-rYou can. For example, upload the content of the test. ini file to zookeeper:
cat test.ini | testcase -r -p/Conf/test.ini -s172.17.0.36:2181

Zookeeper scenario practices: (2) centralized Configuration Management