Introduction to Zookeeper and the use of C APIs

First, the installation and configuration of zookeeper: 1. Zookeeper installation

Download: https://zookeeper.apache.org/recommended download Zookeeper 3.4.6 stable version of the 3.5 version of the cluster, the service startup failure will occur.

Installation: Unzip

Install the C language API:

Enter the./ZOOKEEPER/SRC/C Directory

./configure

Make

Make install

2. Cluster configuration

Zookeeper configuration file in the Conf directory, this directory has zoo_sample.cfg and log4j.properties, you need to rename Zoo_sample.cfg zoo.cfg, because Zookeeper will find this file at startup As the default configuration file. The meanings of each configuration item in this configuration file are described in detail below.

ticktime=2000

Datadir=/home/zoo/data

clientport=2181

• Ticktime: This time is the time interval between the Zookeeper server or between the client and the server to maintain the heartbeat, that is, each ticktime time sends a heartbeat.
• DataDir: As the name implies is Zookeeper to save the data directory, by default, Zookeeper will write the data log file is also stored in this directory.
• ClientPort: This port is the port that the client connects to the Zookeeper server, Zookeeper listens to the port and accepts the client's access request.

When these configuration items are configured, you can start Zookeeper now, after starting to check whether Zookeeper is already in service, you can check with the Netstat–ano command to see if there is a ClientPort port number you configured in the Listening service.

Cluster mode

Zookeeper can not only provide services, but also support multi-machine cluster to provide services. In fact, Zookeeper also supports another way of pseudo-clustering, that is, you can run multiple Zookeeper instances on a single physical machine, and the following describes the installation and configuration of cluster mode.

The installation and configuration of the Zookeeper cluster mode is also not very complex, all you have to do is add a few configuration items. The cluster mode adds the following configuration items in addition to the above three configuration items:

Initlimit=5

synclimit=2

server.1=192.168.211.1:2888:3888

server.2=192.168.211.2:2888:3888

• Initlimit: This configuration item is used to configure the Zookeeper accept client (the client here is not the client that connects the Zookeeper server, but the Leader that is connected to Follower in the Zookeeper server cluster) Server) The maximum number of heartbeat intervals that can be tolerated when a connection is initialized. The client connection failed when the Zookeeper server has not received the return information of the client after 10 heartbeats (that is, ticktime) length. The total length of time is 5*2000=10 seconds.
• Synclimit: This configuration item identifies the length of time that a message, request and response is sent between Leader and Follower, the maximum number of ticktime, and the total length of time is 2*2000=4 seconds
• Server.myid=ip:port1:port2: Where myID is a number, indicating this is the first server; IP is the IP address of this server; Port1 represents the port where this server exchanges information with Leader servers in the cluster Port2 indicates that in the event that the Leader server in the cluster is hung up, a port is needed to re-elect a new Leader, which is the port that the server communicates with each other when the election is performed. If it is a pseudo-cluster configuration, because the IP is the same, so different Zookeeper instance communication port number can not be the same, so they have to assign a different port number.

In addition to modifying the Zoo.cfg configuration file, in the cluster mode to configure a file myID, the file in the DataDir directory, the file contains a data is a value, Zookeeper startup will read this file, get the data inside and zoo.cfg The configuration information is compared to determine the server.

Second, the use of zookeeper: 1. Server-side start-up:

Enter the/bin directory and use the./zkserver.sh start to start the zookeeper service. Use the./zkserver.sh Stop service:/zkserver.sh status to view the service status (leader or follower).

2. Use of CLIENT commands:

Enter the/bin directory and use the./zkcli.sh–server host:port login service, such as

./zkcli.sh-server 192.168.1.91:2181, type any character that appears following the help command.

[Zk:localhost:2181 (CONNECTED) 1] Help
Zookeeper-server host:port cmd args
Connect Host:port
GetPath [Watch]
Lspath [Watch]
SetPath data [Version]
Delquota[-n|-b] Path
Quit
Printwatches On|off
Create [-S] [-e] path data ACL
Statpath [Watch]
Close
Ls2path [Watch]
History
Listquota Path
SetACL path ACL
Getacl Path
Syncpath
Redocmdno
Addauth Scheme auth
Delete path [version]
Setquota-n|-b Val Path

1. Create [-S] [-e]path data ACL
   Where "-S" means creating a sequential AutoNumber node, "-e" means creating a temporary node. Default is Persistent node

Example: Create a persistence node and a temporary node

[Zk:localhost:2181 (CONNECTED) 7] create/test null

Created/test

[Zk:localhost:2181 (CONNECTED) 8] create-e/test0 NULL

Created/test0

When the session exits, the temporary node is automatically deleted, and the temporary node has no child nodes.

The settings and use of ACLs are described separately in the next section.

2.get Path [watch] and set path data [version]

Get is the data that gets znode and related properties, and set is the data that modifies this znode.

3.ls Path [Watch]

View Znode's child nodes

4.stat Path [Watch]

To view the properties of Znode

5.delete path [Version]

Delete Znode, if there are child nodes, delete their child nodes first

6.addauth Scheme auth

Authentication authorization, if a node requires authentication to be able to view, you need this command, see the following section.

3. Use of ACLs

In traditional file systems, ACLs are divided into two dimensions, one is a group, one is permission, and the subdirectory/file inherits the ACL of the parent directory by default. In zookeeper, node's ACL is not inherited and is independently controlled.

Multi-cluster Common zookeeper also involves a permission isolation problem. The zookeeper itself provides an ACL mechanism, expressed as scheme:id:permissions, the first field indicates which mechanism to use, the second ID represents the user, and permissions represents the relevant permissions, such as read-only, read-write, manage, and so on.

scheme: scheme corresponds to which scheme to use for permission management, zookeeper implements a pluggable ACL scheme, which can extend the mechanism of ACL by extending scheme. The zookeeper-3.4.4 default supports the following scheme types:

? World: It has only one ID, called anyone, World:anyone on behalf of any person, zookeeper in the rights of all the nodes are belong to World:anyone

? Auth: It does not require an ID, as long as the user with authentication has permission (Zookeeper support via Kerberos (http://www.cnblogs.com/jankie/archive/2011/08/ 22/2149285.html) to perform authencation, and also support Username/password form of authentication)

? Digest: It corresponds to the ID of username:base64 (SHA1 (password)), it needs to be Username:password form first authentication

? IP: It corresponds to the ID of the client's IP address, set the time can be set up an IP segment, such as IP:192.168.1.0/16, representing the first 16 bits of the IP segment

? Super: In this scheme case, the corresponding ID has super privilege and can do anything (Cdrwa)

In addition, the zookeeper-3.4.4 code also provides support for SASL, but the default is not open, need to be configured to enable, how to configure in the following section.
* SASL: SASL's corresponding ID, is a id,zookeeper-3.4.4 SASL through SASL Authentication user is implemented by Kerberos, This means that the user only has to pass the Kerberos authentication to access the node that it has permission to. (About SASL Reference: http://www.wxdl.cn/cloud/zookeeper-sasl.html)

ID: ID is closely related to scheme and is described in the process of introducing scheme above.

permission: Zookeeper currently supports some of the following permissions:

Permissions	Describe	Note
CREATE	have permission to create child nodes
READ	have permission to read node data and child node lists
WRITE	have permission to modify node data	No permissions to create and delete child nodes
DELETE	have permission to delete child nodes
ADMIN	have permission to set node permissions

Client Example:

1. Create [-S] [-e] path data ACL
   Create/acl Test WORLD:ANYONE:RWD

Created/test

1. Create-s/test/test NULL DIGEST:TEST:V28Q/NYNI4JI3RK54H0R8O5KMUG=:RWCDA

created/test/test0000000000

1. Getacl/acl viewing ACL information for a path
2. Setacl/test Digest:test:v28q/nyni4ji3rk54h0r8o5kmug=:r
3. Setacl/test Auth:username:password:crdwa
4. Addauth/<node-name> Digest <username>:<password>

The principle of ACL:

Zookeeper's rights management is done through server and Client coordination:

(1) Server End

A zookeeper node stores two parts of the content: data and state , and the state contains ACL information. Creating a Znode generates an ACL list in which each ACL includes:

① Permissions perms

② Authentication Mode scheme

③ Specific content Expression:ids

For example, when Scheme= "Digest", Ids is the user name password, which is "ROOT:J0STY9BCUKUBTK1Y8PKBL7QOXSW". The ZooKeeper provides several authentication modes as follows:

① Digest: Client driven by username and password verification, e.g. User:pwd

② Host: Client driven by hostname verification, such as localhost

③ IP: Client driven by IP address verification, e.g. 172.2.0.0/24

④ World: Fixed user for anyone, open permissions for all client side

(2) client

The Client sets author information for the current session by calling the Addauthinfo () (Java, C-zoo_add_auth) function. The Server receives an operation request from the client ( in addition to exists, Getacl ), which requires ACL validation : The author carried on the request The plaintext information is encrypted and compared to the ACL information of the target node, and if the match has the appropriate permissions, the request is rejected by the server.

Three, zookeeper principle mechanism 1. File system

Zookeeper maintains a data structure similar to the file system:

Each subdirectory entry, such as Nameservice, is called Znode, and as with the file system, we are free to add, remove Znode, add and remove sub-znode under a znode, except that Znode can store data.

There are four types of Znode:

1. persistent-Persistent Directory node

The node still exists after the client disconnects from the zookeeper

2. persistent_sequential-persistent sequential numbered directory node

After the client disconnects from the zookeeper, the node still exists, but the node name is zookeeper sequentially numbered

3. ephemeral-Temp directory Node

After the client disconnects from zookeeper, the node is deleted

4. ephemeral_sequential-temporary sequential numbering directory node

After the client disconnects from the zookeeper, the node is deleted, but the node name is zookeeper sequentially numbered

2. Notification mechanism

The client registers to listen to the directory nodes it cares about, and zookeeper notifies the client when the directory node changes (data changes, deletions, and the subdirectory nodes are deleted).

The Zookeeper watch function is triggered once, that is, a watch event will be sent to the client when the data changes. For example, if the client performs an action GetData ("/znode1″, true"), and then/znode1 changes or deletes, the client will get a/znode1 watch event. If the/znode1 is changed again, the client is no longer sent a watch event if the client does not have a new watch set.

This means that an event is sent to the client, but it may not have been delivered to the watch client until the successful return value of the operation reached the client that initiated the change. Watch is sent asynchronously. But zookeeper guarantees a sequence: a client must not see a change in the value of watch before it receives the watch event. Network latency and other factors can cause different clients to see Watch and update return values at different times. But the point is that everything that each client sees is sequential.

Iv. use Scenario 1. To configure synchronization:

1. Build a zookeeper cluster and create a permanent node on the cluster server such as: Authorization
2. You need to update the cached client machine, connect to the cluster, and set watch on the authorization node through the Zookeeper watch mechanism.
3. When the database authentication information changes, the Authorization node data is updated to trigger the client's watch function to update the cache operation in the watch function.

Precautions:

1. Watch is triggered at once, so after the watch function is triggered, a re-registration is required to enable permanent monitoring.
2. Due to network reasons, or the update operation is too frequent, when the client processes the update operation, there is no new watch gap registration, the database authentication information changes, there will be client cache and database data inconsistency error, so after the watch function ends, By retrieving the information about the authorization (using MZXID in the program) and comparing it with the previous one, it is necessary to update the cache operation again if it is inconsistent.
3. Zookeeper session failure problem http://blog.csdn.net/kobejayandy/article/details/26289273 when the client receives Sessionexpired status information, Since the previously set watch will be invalidated, the connection before closing is made, the connection is restarted, and the watch function is set.

2. Cluster Management

Cluster management can be done using zookeeper, mainly for two points

1. Is there a machine to join or exit

Solve this problem, you can contract a parent directory, Gropmember, and then under the parent directory, each machine creates a temporary node, and listens to the parent Directory child node change message, once the machine exits the cluster, the corresponding child node will be deleted, the other machine will be notified. Also, when a new machine is added, other machines will be notified.

1. Election of Master

Master's election is roughly the same as the above principle, all machines create temporary sub-nodes and sequentially numbered, selecting the machine that corresponds to the smallest child node as master.

Other than that:

Zookeeper

Zoo_get (zhandle_t *zh, const char *path, int watch, Char *buffer,
int* Buffer_len, struct Stat *stat);

1. When passing parameters, the value of the Buffer_len is the size of buffer buffers, and when Zoo_get returns successfully, Buffer_len will be to the length of the corresponding node node data

2. How do I determine the size of the buffer? You can set an approximate value, and then when Zoo_get returns, get the node data length from the stat struct, compare it to the returned Buffer_len, if it doesn't match, the buffer is small, then you can get the correct length and get it again.

Stat structure:

struct Stat {
int64_t Czxid;
int64_t Mzxid;
int64_t CTime;
int64_t Mtime;
int32_t version;
int32_t cversion;
int32_t aversion;
int64_t Ephemeralowner;
int32_t datalength;
int32_t Numchildren;
int64_t Pzxid;
};

Introduction to Zookeeper and the use of C APIs