Zookeeper permission management and Quota management

Zookeeper permission management and Quota management

The ACL and Quota mechanisms of Zookeeper have less information on the Internet. Here is a summary for your reference.

1 Zookeeper ACL

The permission management of ZooKeeper, that is, the ACL control function is completed through coordination between the Server and Client:

Server:

A ZooKeeper node stores two parts: Data and status. The status contains ACL information. Creating a znode generates an ACL list. Each ACL in the list includes:

• Scheme)
• Content (Id) (when scheme = "digest", Id is the user name and password, for example, "root: J0sTy9BCUKubtK1y8pkbL7qoxSw = ")
• Permission (perms)

1.1 scheme

ZooKeeper provides the following authentication modes (scheme ):

• Digest: the Client is verified by the user name and password, for example, user: password. The digest password is generated in the base64 form of the Sha1 digest.
• Auth: No id is used to represent any confirmed user.
• Ip: the Client is verified by an ip address, for example, 172.2.0.0/24.
• World: The fixed user is anyone, which is open to all clients.
• Super: In this scheme scenario, the corresponding id has super permissions and can do anything (cdrwa)

Note that the exists and getAcl operations are not subject to ACL permission control. Therefore, any client can query the node status and node ACL.

The node permission (perms) mainly includes the following types:

• Create allows the Create operation on the subnode
• Read allows GetChildren and GetData operations on the current node.
• Write allows the SetData operation on this node
• Delete: allows the Delete operation on the subnode.
• Admin allows setAcl operations on this node

The Znode ACL permission is represented by an int-type number perms. The five binary bits of perms represent setacl, delete, create, write, and read, respectively. For example, 0x1f = adcwr, 0x1 = ---- r, 0x15 = a-c-r.

1.1.1 The fixed id of world scheme is anyone, which indicates that the permission is open to all clients:

[Zk: localhost: 2181 (CONNECTED) 13] create/123 "123"

Created/1, 123

[Zk: localhost: 2181 (CONNECTED) 14] getAcl/123

'World, 'Anyone

: Cdrwa

1.1.2 ip scheme: Set accessible ip addresses (such as 127.0.0.1) or ip address segments (such as 192.168.1.0/16)

10.194.157.58 create/test on this machine and set ip Access Permissions

[Zk: 10.194.157.58: 2181 (CONNECTED) 0] create/test "123"

Created/test

[Zk: 10.194.157.58: 2181 (CONNECTED) 1] setAcl/test ip: 10.194.157.58: crwda

CZxid = 0x740021e467

Ctime = Wed Dec 02 18:09:09 CST 2015

MZxid = 0x740021e467

Mtime = Wed Dec 02 18:09:09 CST 2015

PZxid = 0x740021e467

Cversion = 0

DataVersion = 0

AclVersion = 1

EphemeralOwner = 0x0

DataLength = 5

NumChildren = 0

[Zk: 10.194.157.58: 2181 (CONNECTED) 2] ls/test

[]

We can see that the local machine is accessible.

 

Log on to 10.205.148.152.

[Zk: 10.194.157.58: 2181 (CONNECTED) 1] ls/test

Authentication is not valid:/test

As you can see, the connected ip address is not authorized and an access error is prompted.

1.1.3 The id of digest scheme is represented as username: BASE64 (SHA1 (password ))

[Root @ rocket zookeeper-server1] # cd/usr/local/zookeeper-server1/

[Root @ rocket zookeeper-server1] # pwd

/Usr/local/zookeeper-server1

# Generating ciphertext

[Root @ rocket: zookeeper-server1] # java-cp. /zookeeper-3.4.6.jar :. /lib/log4j-1.2.16.jar :. /lib/slf4j-log4j12-1.6.1.jar :. /lib/slf4j-api-1.6.1.jar org. apache. zookeeper. server. auth. digestAuthenticationProvider test: test

Test: test-> test: V28q/NynI4JI3Rk54h0r8O5kMug =

Create an acl

After passing the authentication, you can access the data:

[Zk: localhost: 2181 (CONNECTED) 0]

[Zk: localhost: 2181 (CONNECTED) 0] ls/test_acl

Authentication is not valid:/test_acl

[Zk: localhost: 2181 (CONNECTED) 1] getAcl/test_acl

'Digest, 'test: V28q/NynI4JI3Rk54h0r8O5kMug =

: Cdrwa

[Zk: localhost: 2181 (CONNECTED) 2] addauth digest test: test

[Zk: localhost: 2181 (CONNECTED) 3] ls/test_acl

[]

[Zk: localhost: 2181 (CONNECTED) 4] get/test_acl

"Test"

CZxid = 0x33

Ctime = Wed Dec 02 00:10:47 PST 2015

MZxid = 0x33

Message time = Wed Dec 02 00:10:47 PST 2015

PZxid = 0x33

Cversion = 0

DataVersion = 0

AclVersion = 1

EphemeralOwner = 0x0

DataLength = 6

NumChildren = 0

1.2 SuperDigest super Administrator

What should I do if I have set the znode permission but forgot the password? Fortunately, Zookeeper provides a super administrator mechanism.

To verify the acl of a znode operation, the Client performs the following operations:

A) traverse all the ACLs of znode:

I. For each ACL, the operation type matches the permission (perms) first.

Ii. The session auth information is matched with the ACL username and password only when the matching permission is successful.

B) if both matches are successful, the operation is allowed; otherwise, the returned permission is not enough error (rc =-102)

NOTE: If any ACL in the znode ACL List does not have the setAcl permission, the superDigest permission cannot be modified. If the znode does not have the delete permission, all its subnodes will not be deleted. The only way is to manually delete the snapshot and log methods, roll the ZK back to a previous state, and then restart, of course, this will affect the normal application of other nodes outside the znode.

Procedures for setting superDigest

Modify zkServer. sh and add super permission settings.

-Dzookeeper. DigestAuthenticationProvider. superDigest = super: gG7s8t3oDEtIqF6DM9LlI/R + 9Ss =

Restart Zookeeper

#./ZkServer. sh restart

At this time

 

Instead of using test: test for authentication, super: super is used for authentication:

[Zk: localhost: 2181 (CONNECTED) 0] ls/test_acl

Authentication is not valid:/test_acl

[Zk: localhost: 2181 (CONNECTED) 1] addauth digest super: super

[Zk: localhost: 2181 (CONNECTED) 2] ls/test_acl

[]

[Zk: localhost: 2181 (CONNECTED) 3] get/test_acl

"Test"

CZxid = 0x33

Ctime = Wed Dec 02 00:10:47 PST 2015

MZxid = 0x33

Message time = Wed Dec 02 00:10:47 PST 2015

PZxid = 0x33

Cversion = 0

DataVersion = 0

AclVersion = 1

EphemeralOwner = 0x0

DataLength = 6

NumChildren = 0

1.3 ACL mechanism Defects

However, ACL, after all, is only access control, rather than comprehensive permission management. Using this method to isolate multiple clusters, there are many limitations:

The ACL does not have a recursive mechanism. After any znode is created, you must set the ACL separately and cannot inherit the ACL settings of the parent node.

In addition to scheme such as ip, the use of digest and auth is not transparent to users, which also brings a lot of cost to the use, many open-source frameworks that depend on zookeeper do not support ACL, such as hbase and storm.

2 Zookeeper quota

The ZooKeeper quota mechanism supports the number of nodes (znode) and the size of space (bytes ).

 

[Zk: localhost: 2181 (CONNECTED) 2] create/test_quota "12345"

Created/test_quota

[Zk: localhost: 2181 (CONNECTED) 3] listquota/test_quota

Absolute path is/zookeeper/quota/test_quota/zookeeper_limits

Quota for/test_quota does not exist.

# Quota is not set yet.

[Zk: localhost: 2181 (CONNECTED) 4] setquota-n 5/test_quota

Comment: the parts are option-n val 5 path/test_quota

#-N indicates znode count limit. Here, znode count under the path/test_quota is limited to 5 (including/test_quota)

#-B Indicates setting the znode Data byte size limit. This is not demonstrated here. If you are interested, go on to the experiment.

[Zk: localhost: 2181 (CONNECTED) 5] listquota/test_quota

Absolute path is/zookeeper/quota/test_quota/zookeeper_limits

Output quota for/test_quota count = 5, bytes =-1 # limit znode count to 5

Output stat for/test_quota count = 1, bytes = 7 # currently znode count is 1

[Zk: localhost: 2181 (CONNECTED) 3] create/test_quota/0 "0"

Created/test_quota/0

[Zk: localhost: 2181 (CONNECTED) 6] create/test_quota/1 "1"

Created/test_quota/1

[Zk: localhost: 2181 (CONNECTED) 7] create/test_quota/2 "2"

Created/test_quota/2

[Zk: localhost: 2181 (CONNECTED) 8] create/test_quota/3 "3"

Created/test_quota/3

[Zk: localhost: 2181 (CONNECTED) 9] create/test_quota/4 "4"

Created/test_quota/4

# Multiple znodes have been created.

Check the log of zookeeper and find the log with Quota exceeded. Here we want to explain that the Quota mechanism of zookeeper is moderate. Even if it exceeds the limit, we only need to report it in the log, it does not limit the behavior of the Client. The Client can continue to operate on znode.

In the actual project, the Client can view the data in the/zookeeper/quota directory to determine whether the data exceeds the quota limit, so as to generate some alarms.

[Zk: localhost: 2181 (CONNECTED) 4] get/zookeeper/quota/test_quota/zookeeper_limits

Count = 5, bytes =-1

[Zk: localhost: 2181 (CONNECTED) 5] get/zookeeper/quota/test_quota/zookeeper_stats

Count = 7, bytes = 25

-------------------------------------- Split line --------------------------------------

Ubuntu 14.04 installs distributed storage Sheepdog + ZooKeeper

CentOS 6 installs sheepdog VM distributed storage

ZooKeeper cluster configuration

Use ZooKeeper to implement distributed shared locks

Distributed service framework ZooKeeper-manage data in a distributed environment

Build a ZooKeeper Cluster Environment

Test Environment configuration of ZooKeeper server cluster

ZooKeeper cluster Installation

Zookeeper3.4.6 Installation

-------------------------------------- Split line --------------------------------------

This article permanently updates the link address: