Communication model of Zookeeper series

The topic of this article is to explain the zookeeper communication model, and this section will illustrate the zookeeper communication model through a schematic diagram.

communication architecture for zookeeper

In the zookeeper entire system, there are 3 roles in the service, client, Follower, leader. Where the client is responsible for initiating the application request, follower accepts the client initiated request, participates in the transaction confirmation process, leader chooses after the leader crash. and leader mainly undertake the coordination of affairs, of course, leader can also take the function of receiving customer requests, in order to facilitate the description, the following description is the communication between the client and follower, if the zookeeper configuration support leader receive client requests, Client-to-leader communication is exactly the same as client-to-follower communication patterns. The role between follower and leader may be converted at some point. A follower may be follower elected by the cluster (Quorum) after the leader crash is dropped. Once a leader is crash, joining the cluster again (Quorum) will exist as a follower role. In a cluster (Quorum), there are only 1 leader and multiple follower at any time except in the process of electing leader without follower and leader distinctions. The communication architecture between the Client, follower, and leader is as follows:

Between client and follower

In order for the client to have high throughput, NIO communication is used between client and follower. When the client needs to deal with the zookeeper service, it first reads the configuration file to determine all the server lists within the cluster and selects a follower as a communication target according to a certain load balance algorithm. In this way, there is a communication channel between the client and the follower that is formed by the NIO pattern. This channel is maintained until the client closes the session or because the client or follower either side interrupts the communication connection for some reason. Under normal circumstances, the client and follower have heartbeat detection when no request is initiated.

Between follower and leader

The communication between follower and leader is mainly due to the follower receiving the image (create, delete, SetData, SetACL, CreateSession, CloseSession, Sync) Such commands that require leader to reconcile the final result will result in communication between follower and leader. Because the relationship between leader and follower is a one-to-many relationship, it is very suitable for client/server mode, so they use the C/S mode, a socket server is created by leader, and the coordination request of each follower is monitored.

Cluster in the process of selecting leader

Since there is no leader in the process of selecting leader, any member of the cluster needs to communicate with all other members, and when the membership of the cluster becomes large, this traffic is large. The process of selecting leader occurs when the zookeeper system has just started or leader loses contact, the selection leader process will not be able to process the user's request, in order to improve the usability of the system, we must minimize the process time. Which way do you choose to make them available quickly to get the selection results? Zookeeper adopts the strategy mode in this process, and can dynamically insert the algorithm of selecting leader. The system provides 3 selection algorithms by default, Authfastleaderelection,fastleaderelection,leaderelection. where authfastleaderelection and leaderelection use UDP mode for communication, while Fastleaderelection still uses TCP/IP mode. In the new version of Zookeeper, a learner role has been added to reduce the number of participants who choose leader. Make the selection process faster. Generally zookeeper leader are very fast in their selection process, usually <200ms.

Zookeeper's communication process

To learn more about Zookeeper's communication processes, we first need to understand which client interfaces are provided by zookeeper, and we group by interfaces that have the same communication flow:

Zookeeper System administration commands

Zookeeper's system management interface refers to some of the commands used to view the zookeeper running state, all of which have a 4-letter command format. Mainly include:

1. ruok: Send this command to test whether zookeeper is functioning properly.
2. stat: Gets the status of the server side of the connection server and the status information for all customer service ports that are connected to the server.
3. reqs:  gets the request that the current client has committed but has not yet returned.
4. STMK: Turns on or off Zookeeper's trace level.
5. GTMK: Gets whether the current zookeeper trace level is turned on.
6. envi:  gets zookeeper Java-related environment variables.
7. srst: Resetting the server-side statistics status

When the user sends these commands to the server, there is no business processing logic because these requests are only relevant to the connected server, which is very simple. The zookeeper is processed with the fastest efficiency for these commands. These commands are sent to the server side using only a 4-byte int type to represent different commands, not string handling. When these commands are received on the server side, the results are returned immediately.

Session creation

Any client's business request is based on the existence of the session. The session is to maintain a communication channel between the client and the follower and maintain all the states between them from the start of creation. When starting a zookeeper client, first find out the follower according to certain algorithm, then establish NIO connection with follower. When the connection is established, send the Create session command to the server side to create a session for the connection to maintain the state of the connection. When the server receives the Create session command, first from the local session list to see if there is already the same SessionID, then close the original session to recreate the new session. The process of creating the session will need to be sent to leader, and then leader notify the other follower, most follower will log this action to the local log and then notify leader, leader send a commit command to all follower, The follower of the connection client returns the session response that was created successfully. The process of coordination between leader and follower will be explained in detail later. When the client successfully creates the session, the other business commands are processed properly.

Zookeeper Query command

The Zookeeper Query command is primarily used to query server-side data and does not change server-side data. All query commands can be immediately returned from the client-connected server without leader coordination, so the data obtained from the query command is likely to be stale data. However, due to any changes in the data, leader will publish the results of the changes to all follower, so generally speaking, follower data can be updated in a timely manner. These query commands include the following commands:

1. exists: Determines if node for the specified path exists and returns true if it exists, otherwise false.
2. GetData: from the specified Path Get the node of Data
3. Getacl: gets the specified Path of the ACL .
4. GetChildren: gets the specified Path of the node all the children of the knot.

All query commands can specify watcher to track data changes for the specified path. Once the specified data has changed (create,delete,modified,children_changed), the server will send a command to register the watcher. Watcher will be explained in detail in the Watcher of zookeeper.

Zookeeper modifying commands

zookeeper Modify command is primarily used to modify node data or structure, or permission information. Any modification commands need to be submitted to leader for coordination and will not be returned until the coordination is complete. The modification commands mainly include the following:

1. 1.    createsession : Request server Create a session
2. 2.    Create : Create a node
3. 4.    setData : Modify data for a node
4. 5.    setacl : Modify a node ACL
5. 6.    closesession : request server Close session

We know according to the previous communication diagram that any modification command requires leader coordination. In the leader coordination process, it takes 3 times between leader and follower to request a response. The transaction log records are also involved in this process, and worse still is the take snapshot operation. Therefore, this process can be time-consuming. But zookeeper communication is the biggest feature of the asynchronous, if the request is continuous, zookeeper processing is centralized processing logic, and then batch send, batch size is also controlled. If the request is small, send it immediately. In this way, when the load is very large can also guarantee a large throughput, time-sensitive also to a certain extent to ensure.

Zookeeper server-side business processing-processor chain

<!--endfragment-->

Zookeeper handles business requests through chained processor, each processor responsible for handling specific functions. The different Zookeeper roles of the server processor chain are not the same, the following respectively describes standalone Zookeeper server, leader and follower different processor chains.

The processor in zookeeper

2. Ackrequestprocessor: When leader sends proposal from follower, follower will send an ACK response, and leader will call this processor after receiving an ACK response for processing. It is primarily responsible for checking that the request has reached the confirmation of most follower, and if the condition is met, submit commitprocessor for commit processing
4. Commitprocessor: Processing requests from the commited queue that have been reconciled and commit by the leader or removed from the request queue for requests that do not require leader coordination to proceed with the next step.
6. Finalrequestprocessor: The processing of any request requires this processor, which is the last processor of the request processing, and is primarily responsible for wrapping different types of response packages based on different requests. Of course, after the coordinated request between follower and leader, there will be no need to send a response because there is no client connection (the code is reflected in the if (REQUEST.CNXN = = null) {return;} ）。
8. The first of the Followerrequestprocessor:follower processor chain is primarily responsible for coordinating the request for modification and the synchronization request to leader.
10. Preprequestprocessor: As the first processor on leader and standalone server, the main effect is to generate changelog for all modified commands.
12. The Proposalrequestprocessor:leader is used to package the request for proposal to follower request confirmation.
14. The sendackrequestprocessor:follower is used to send an ACK response to the leader.
16. Syncrequestprocessor: Responsible for writing the commit transaction to the transaction log and take snapshot.
18. Tobeappliedrequestprocessor: is responsible for transferring the Tobeapplied queue request to the next requests for processing.

Standalone Zookeeper processor Chain

Leader Processor Chain

Follower Processor Chain

Transfer from http://zoutm.iteye.com/blog/708447

Communication model of Zookeeper series (RPM)