HMaster startup code analysis for source code analysis of HBase1.0.0 (2), hbase1.0.0hmaster

HMaster startup code analysis for source code analysis of HBase1.0.0 (2), hbase1.0.0hmaster

In the previous blog, my code analysis is the startup function of the startMaster core. This article mainly analyzes the corresponding components involved in the specific HMaster construction process and the startup of services, this article mainly analyzes the process. The specific starting process of each part will be analyzed in detail later, including the creation of RPC services, initialization of the zookeeper cluster management class, startup of various working threads, etc.

At the beginning of the introduction, it is necessary to understand the HMaster's inheritance system, such:

Next, analyze the startup process:
1. startup code

LogProcessInfo (getConf (); // save the system running configuration parameters and JVM status to the log CoordinatedStateManager csm = CoordinatedStateManagerFactory. getCoordinatedStateManager (conf); HMaster master = HMaster. constructMaster (masterClass, conf, csm );

2. the structure of the csm object, conf in this function. the function of getClass is meaningless. The code here indicates that we need to obtain a collaborative state management class, and the three parameters are represented as follows: 1. name of the custom class (used to obtain the corresponding class file), 2. ZKCoordinatedStateManager, 3. used to verify whether the obtained class is inherited from CoordinatedManager. the default class is to use zookeeper to implement CoordinatedManager to manage HBase clusters.
Then, the reflection mechanism is used to form a class instance.

public static CoordinatedStateManager getCoordinatedStateManager(Configuration conf) {  Class<? extends CoordinatedStateManager> coordinatedStateMgrKlass =    conf.getClass(HConstants.HBASE_COORDINATED_STATE_MANAGER_CLASS,      ZkCoordinatedStateManager.class, CoordinatedStateManager.class);  return ReflectionUtils.newInstance(coordinatedStateMgrKlass, conf);}

3. The next step is the construction process of the specific HMaster object,

/** * Utility for constructing an instance of the passed HMaster class. * @param masterClass * @param conf * @return HMaster instance. */public static HMaster constructMaster(Class<? extends HMaster> masterClass,    final Configuration conf, final CoordinatedStateManager cp)  {  try {    Constructor<? extends HMaster> c =      masterClass.getConstructor(Configuration.class, CoordinatedStateManager.class);    return c.newInstance(conf, cp);  }

Here we use the constructor with Configuration and CoordinatedStateManager as parameters to construct the constructor. But why does reflection need to be used here ?? Is it better to increase the scalability of the program by passing in type information, but if the scalability is increased, do you still need to modify the source code of the call?

Next let's take a look at this constructor:
First, call the constructor of the parent class: (photo here for related class digoal)
Super (conf, csm );
Assign values to various parameter variables. Here are several key steps.
(1) Create an RPC service

rpcServices = createRpcServices();

(2) connect to the Zookeeper Cluster

// Open connection to zookeeper and set primary watcherzooKeeper = new ZooKeeperWatcher(conf, getProcessName() + ":" +        rpcServices.isa.getPort(), this, canCreateBaseZNode());

(3) create a file system operation instance

this.fs = new HFileSystem(this.conf, useHBaseChecksum);

(4) initialize the CoordinatedStateManager object

this.csm = (BaseCoordinatedStateManager) csm;this.csm.initialize(this);this.csm.start();

(5) create various cluster tracking and management objects

masterAddressTracker = new MasterAddressTracker(getZooKeeper(), this);masterAddressTracker.start();clusterStatusTracker = new ClusterStatusTracker(zooKeeper, this);clusterStatusTracker.start();this.configurationManager = new ConfigurationManager();

Finally, start the rpc service and ui:
RpcServices. start ();
PutUpWebUI ()
So far, the instantiation process of the parent class HRegion ends, and the instantiation of the Extended Part of the transfer to HMaster
The instantiation part of the HMaster is complicated, and several key steps are analyzed here.

(1) activate a working HMaster

ActiveMasterManager = new ActiveMasterManager (zooKeeper, this. serverName, this); // register the ActiveMasterManager (ZooKeeperWatcher watcher, ServerName sn, Server master) {super (watcher); watcher. registerListener (this); this. sn = sn; this. master = master ;}

(2) Start the Jetty Service

int infoPort = putUpJettyServer();2015-03-23 13:40:49,143 INFO [main] http.HttpRequestLog: Http request log for http.requests.master is not defined2015-03-23 13:40:49,366 INFO [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter)2015-03-23 13:40:49,423 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master2015-03-23 13:40:49,425 INFO [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static2015-03-23 13:40:49,812 INFO [main] http.HttpServer: Jetty bound to port 16030

(3) Activate Master and startActiveMasterManager (infoPort );
There are several sub-steps. In startActiveMasterManager, first add a backupZNode to Zookeeper, and then explicitly Delete the node after it becomes activeMaster.
Master. ActiveMasterManager: Deleting ZNode for/hbase/backup-masters/xiaoyi-PC, 52777,1427089138770 from backup master directory
After activation, call finishActiveMasterInitialization (status) to complete the Startup Process of the corresponding worker thread of the Master.

(4) create a cluster Link

setupClusterConnection();

(5) initialize the trackers of the Zookeeper Cluster

initializeZKBasedSystemTrackers();

(6) Start various Working Service threads

startServiceThreads();// Start the executor service poolsthis.service.startExecutorService(ExecutorType.MASTER_OPEN_REGION,        conf.getInt("hbase.master.executor.openregion.threads", 5));this.service.startExecutorService(ExecutorType.MASTER_CLOSE_REGION,        conf.getInt("hbase.master.executor.closeregion.threads", 5));this.service.startExecutorService(ExecutorType.MASTER_SERVER_OPERATIONS,        conf.getInt("hbase.master.executor.serverops.threads", 5));this.service.startExecutorService(ExecutorType.MASTER_META_SERVER_OPERATIONS,        conf.getInt("hbase.master.executor.serverops.threads", 5));this.service.startExecutorService(ExecutorType.M_LOG_REPLAY_OPS,        conf.getInt("hbase.master.executor.logreplayops.threads", 10));

Wait for the RegionServer instance to join its Management Cluster

this.serverManager.waitForRegionServers(status);activeMasterManager] master.ServerManager: Waiting for region servers count to settle;

After all the configured regionservers are added to the cluster, the cluster Initialization is complete,
The next heavyweight component is LoadBalancer, which is mainly responsible for regions allocation between HRegions.
Check whether Hbase meta has been allocated,
Finally, several background threads are started for corresponding monitoring and processing. Now the initialization of the HMaster is complete.

// Start balancer and meta catalog janitor after meta and regions have// been assigned.status.setStatus("Starting balancer and catalog janitor");this.clusterStatusChore = new ClusterStatusChore(this, balancer);Threads.setDaemonThreadRunning(clusterStatusChore.getThread());this.balancerChore = new BalancerChore(this);Threads.setDaemonThreadRunning(balancerChore.getThread());this.catalogJanitorChore = new CatalogJanitor(this, this);Threads.setDaemonThreadRunning(catalogJanitorChore.getThread());