HBase Master Parsing

The actual implementation of the master server for HBase is hmaster. It provides the client-facing hmasterinterface and the implementation of the Hmasterregioninterface for region server.

Here is an introduction to its components:

Infoserver
The component that presents the information. This is a Web server that responds to your HTTP request to access http://MasterHost:60010, which is essentially a jetty Web server.

Zookeeperwatcher
The observer of the zookeeper. All components that need to know and handle znode state changes need to register zookeeperlistener on it. It also provides the ability to manipulate nodes on zookeeper, and to manipulate them by acquiring their internal recoverablezookeeper objects.

Activemastermanager
The management object for active master. The main thing it does is to monitor the change of master Znode on zookeeper. Once it discovers that the current active master unavailable (that is, Master Znode is deleted), it competes to make the current master the new active master.

When Master starts, it calls its Blockuntilbecomingactivemaster method, attempting to become the active master of the cluster. If it fails, master's boot will be blocked on this method until the cluster stops or the current active master unavailable, then it will be awakened to compete for new.

Regionservertracker
Track the status of online region server through zookeeper and do the appropriate processing. If an RS znode is removed, it terminates the RS through Servermanager and moves out of the online servers list.

In particular, when it is ready to expire this RS, will first determine whether the RS is online, if not the line does not do exprire, do not move out of the list, just a warn log. I guess the purpose of this is: since RS is not on the line, I can not give it a close order, maybe after a while this RS back to normal and online, and re-establish znode in the zookeeper. But I think there is a problem, in case this RS will never recover, the online servers list will not be left a offline RS.

Drainingservertracker
Tracking draining region server status, draining RS is no longer needed to obtain more region Rs.

Rpcserver
The component that provides the RPC service. The specific service is supported by Rpcengine, and the engine implementation is writablerpcengine. After the 0.96 release HBase introduced the implementation of the Protocol BUFFER,RPC engine was also replaced.

Masterfilesystem
The operation of master on the underlying file system is abstracted. Include split log, check file system status, and more.

Servermanager
Manage Region server. Maintain a list of online and offline servers while handling RS startup and shutdown. The RS is reported to it after startup, and it detects the clock skew between master and RS and whether the same Hostport RS is already present. It also checks to see if the RS is in the dead list, preventing this RS from recovering quickly but already in the expiration process.

The load information for RS will also be given to it, and the closed and open region is also done through it.

AssignmentManager
The main job is region assign. While maintaining the region state, such as it knows a split completed event through zookeeper, it updates the status of the associated region. This is a heavyweight component, Master maintains the division of Region, and movement is related to it.

Catalogtracker
-root-and. META. Tracking device. The specific work is done by Rootregiontracker and Metanodetracker. The former tracks the state of the Znode "Root-region-server", which is traced by the latter. META. The corresponding Znode state (specifically this znode is "unassign/hregioninfo.first_meta_regioninfo.getencodedname ()") The location information for the catalog table can be obtained from this component.

Clusterstatustracker
Track the status of "shutdown" this znode, which can be used to control the cluster down or up. This znode the start-up time of the cluster. Its Setclusterdown method is done by removing the "shutdown" znode.

Memoryboundedlogmessagebuffer
Fatal error information from region server is stored and automatically cleaned up after buffer size is exceeded.

Executorservice
Event executor, various events are submitted to the inside of a different queue waiting to be executed. The default resources for different events are also different, for example, the Master_open_region event is handled by default with 5 threads.

This is not java.util.concurrent.ExecutorService but org.apache.hadoop.hbase.executor.ExecutorService.

LoadBalancer
Balances the load on Region server on region. It does not work when and only if there is no region in transition. Balancecluster is called periodically to generate Regionplan.

Its default implementation is defaultloadbalancer,0.96 after the default is Stochasticloadbalancer (HBASE-5959). The latter idea is to make several decisions randomly, comparing the cost function calculation results with the current optimal results, preserving the smaller ones. Finally, if the current optimal cost is less than the original cost, then the corresponding Regionplan is generated.

Balancerchore
Regular execution of Master.balance ().

Catalogjanitor
Regular cleanup. Meta. The parent region information that was left during the split. If the reference of the parent region is no longer being hold by its daughter, then the parent region is cleared.

An additional condition is that if the parent of parent region is not clean, then the parent should not be clean.

Logcleaner
Periodically clean the log under the. oldlogs directory.

It will load all baselogcleanerdelegate from the Hbase.master.logcleaner.plugins configuration. A log can be deleted, and only all delegate agree.

Hfilecleaner
Periodically clean up the. Archive directory under hfile.

It will load all basehfilecleanerdelegate from the Hbase.master.hfilecleaner.plugins configuration.
Only all delegate agree to be deleted.

Mastercoprocessorhost
Provides the execution environment and framework of the master-side coprocessor.

These coprocessor are packed in masterenvironment. When the action occurs, it iterates through all the masterenvironment in the Mastercoprocessorhost, gets the inside Masterobserver, and invokes the relevant method.

Snapshotmanager
The component that manages the table snapshot.

Provides methods for monitoring the execution status of a snapshot, snapshot recovery, snapshot deletion, and so on.

Healthcheckchore
This is not a necessary component, only you have configured Hbase.node.health.script.location to start it. It periodically executes the script at the location setting to detect health, and stops master if the number of failures within the failed window reaches the threshold.

And then we look at what hmaster the whole boot process:

1. first set various parameters in Hmaster's constructor and start Rpcserver, link zookeeper (that is, create Zookeeperwatcher), If the hbase.node.health.script.location configuration exists, a healthcheckchore is constructed.

2. then the Hmaster Run method is executed, and the method does the following:

Start Infoserver, call Becomeactivemaster to compete to become cluster's only active master.
Becomeactivemaster First initializes the Blockuntilbecomingactivemaster method that Activemastermanager and Clusterstatustracker last called Activemastermanager 。

If the competition succeeds and continues to go down, call finishinitalization, otherwise it is stuck in this step.

3. The Finishinitalization method is the most complicated process to start the whole thing.

First initialize the Masterfilesystem, and then write the cluster ID to zookeeper "Hbaseid" Znode
The Servermanager and Executorservice are then initialized.

The initializezkbasedsystemtrackers is then called to initialize various zookeeper-based components. This first initializes the Catalogtracker, then the LoadBalancer. Because the initialization of AssignmentManager needs loadbalancer, although loadbalancer and zookeeper have nothing to do with it. The Assignmentmanager,regionservertracker,drainingservertracker,clusterstatustracker,snapshotmanager is then initialized separately.

After the Initializezkbasedsystemtrackers method finishes, the Mastercoprocessorhost is initialized shortly thereafter.

Then call Startservicethreads to start the various services. First, this method starts the Master_open_region,master_close_region,master_server_operations,master_meta_server_operations,master. _table_operations the processing thread pool for these 5 events. Then started the Logcleaner,hfilecleaner, healthcheckchore these cycles of service. Finally, the openserver of Rpcserver is called, allowing the response request.

When Startservicethreads is finished, Servermanager's waitforregionservers is called to wait for all region server reports until some conditions occur before the wait is terminated. For example, the RS number has been reported to reach Hbase.master.wait.on.regionservers.maxtostart.

After the wait is over, we get the online region servers via Regionservertracker, which is registered on zookeeper. Traverse them and look for those that have already been registered on the zookeeper, but have no report yet. We also record these in Servermanager, except that their hserverload information is temporarily empty.

Next is the time-out monitoring to start AssignmentManager. It has a timeoutmonitor inside it, which actually starts it.

Get last failed RS, then do split log. Depending on the configuration, it is possible to do a distributed split log or the master own split log. Split log is used for playback later to recover data that was not written to the file system before it failed.

It then detects if the root region server is failed, if it is split log, then assign root. The same test was met region server, followed by assign Meta.

After the two directory tables are assign successful, call AssignmentManager's joincluster. This method scans the meta table and rebuild all existing region.

Finally initialize and start Balancerchore and Catalogjanitor.

The start-up process hides some details in order to see the main line.

The entire Hmaster architecture and startup process is almost like this, more master work details can read the relevant components of the source code.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

HBase Master Parsing