HBase Startup script/shell parsing

The commonly used HBase startup scripts are:
1. $HBASE _home/bin/start-hbase.sh
Start the entire cluster


2. $HBASE _home/bin/stop-hbase.sh
Stop the entire cluster


3. $HBASE _home/bin/hbase-daemons.sh
Start or stop, all regionserver or zookeeper or backup-master


4. $HBASE _home/bin/hbase-daemon.sh
Start or stop, single master or Regionserver or zookeeper


Starting with start-hbase.sh, you can look at some of the invocation relationships between scripts
The start-hbase.sh process is as follows:
1. Run hbase-config.sh (explanation behind the action)
2. Parsing parameters (version 0.96 and later only can take the unique parameter AutoRestart, the function is to restart)
3. Call hbase-daemon.sh to start master; Call hbase-daemons.sh to start Regionserver zookeeper Master-backup

The role of hbase-config.sh:
Load related configuration, such as Hbase_home directory, conf directory, Regionserver machine list, java_home directory, etc., it will call $hbase_home/conf/hbase-env.sh

The role of hbase-env.sh:
The main is to configure the JVM and its GC parameters, you can also configure the log directory and parameters, configure whether HBase manages ZK, configure the process ID directory, etc.

The role of hbase-daemons.sh:
Depending on the process you want to start,
If it is zookeeper, call zookeepers.sh
If it is regionserver, call regionservers.sh
If it is master-backup, call master-backup.sh

The role of zookeepers.sh:
If Hbase_manages_zk "=" true "in hbase-env.sh, then parse the XML configuration file by Zkservertool this class, Get a list of ZK nodes (that is, the configuration values for Hbase.zookeeper.quorum) and then send remote commands to these nodes via SSH:
CD ${hbase_home};
$bin/hbase-daemon.sh--config ${hbase_conf_dir} start/stop Zookeeper


The role of regionservers.sh:
Similar to zookeepers.sh, get the list of Regionserver machines through the ${hbase_conf_dir}/regionservers configuration file, then SSH to these machines to send remote commands:
CD ${hbase_home};
$bin/hbase-daemon.sh--config ${hbase_conf_dir} start/stop regionserver


The role of master-backup.sh:
By ${hbase_conf_dir}/backup-masters This configuration file, get the Backup-masters machine list (default configuration, this profile does not exist, so does not start backup-master), Then SSH sends the remote command to these machines:
CD ${hbase_home};
$bin/hbase-daemon.sh--config ${hbase_conf_dir} start/stop Master--backup


The role of hbase-daemon.sh:
Either zookeepers.sh or regionservers.sh or master-backup.sh, the local hbase-daemon.sh is eventually called, and the process is as follows:
1. Run hbase-config.sh, load various configurations (Java environment, log configuration, process ID directory, etc.)
2. If it is the start command.
Scroll out output file, scroll GC log file, log file output start time +ulimit-a information, such as
"Mon-Nov-10:31:42 CST starting master on Dwxx.yy.taobao"
".. Open files (-N) 65536: "
3. Call $hbase_home/bin/hbase start Master/regionserver/zookeeper
4. Execute wait, wait for the open process in 3 to end
5. Execute Cleanznode, delete regionserver on ZK registered node, the purpose is: in the case of accidental exit of the regionserver process, you can avoid 3 minutes of ZK heartbeat timeout wait, directly from master for downtime recovery
6. If it is a stop command.
Checks whether a process exists based on the process ID, invokes the KILL command, and waits until the process does not exist
7. If it is a restart command.
After calling stop, call Start again ...


$HBASE the role of _home/bin/hbase:
The implementation of the final launch is executed by this script
1. You can view its usage by typing $hbase_home/bin/hbase
DBA TOOLS
Shell Run the HBase shell
Hbck run the HBase ' fsck ' tool
Hlog Write-ahead-log Analyzer
hfile Store File Analyzer
Zkcli Run the ZooKeeper shell
PROCESS MANAGEMENT
Master Run an HBase hmaster node
Regionserver Run an HBase hregionserver node
Zookeeper run a zookeeper server
Rest run an HBase REST server
Thrift Run the HBase thrift server
Thrift2 Run the HBase Thrift2 server
Avro run an HBase Avro server


Package MANAGEMENT
Classpath Dump HBase classpath
Version Print the version
Or
CLASSNAME run the class named CLASSNAME


2.bin/hbase Shell, this is a common shell tool, operation of the common DDL and DML will go through this, its implementation (call to HBase) is written in Ruby


3.bin/hbase HBCK, operations and maintenance tools, check the data consistency status of the cluster, its execution is directly called
The main function in Org.apache.hadoop.hbase.util.HBaseFsck


4.bin/hbase Hlog, log Analysis tool, whose execution is directly called
The main function in Org.apache.hadoop.hbase.regionserver.wal.HLogPrettyPrinter


5.bin/hbase hfile, hfile analysis tool, whose execution is directly called
The main function in Org.apache.hadoop.hbase.io.hfile.HFile




6.bin/hbase zkcli, view/Manage ZK shell tools, very useful, often used, such as you can through (Get/hbase-tianwu-94/master) it learned that the current active master, you can pass (GET/ Hbase-tianwu-94/root-region-server), you can also pass the current root region server in the test (delete/hbase-tianwu-94/rs/ Dwxx.yy.taobao), Analog regionserver with ZK disconnect,,,
Its execution is called the main function of the Org.apache.zookeeper.ZooKeeperMain




7. Regression to just hbase-daemon.sh the call to this script is:
$HBASE _home/bin/hbase Start Master/regionserver/zookeeper
Its execution is directly called
Org.apache.hadoop.hbase.master.HMaster
Org.apache.hadoop.hbase.regionserver.HRegionServer
Org.apache.hadoop.hbase.zookeeper.HQuorumPeer
The main function, and these main functions are the new one runnable Hmaster/hregionserver/quorumpeer, in the non-stop running ...


8.bin/hbase classpath Print Classpath


9.bin/hbase version Print HBase release information


10.bin/hbase CLASSNAME, this is very practical, all implementations of the main function of the class can be run through the script, such as the previous Hlog hfile hbck tool, essentially a quick call to this interface, Other classes that do not provide a quick way can also be called with this interface, such as the region merge call:
$HBASE _home/bin/hbase/org.apache.hadoop.hbase.util.merge




Summary of script usage:
1. Turn on the cluster, start-hbase.sh
2. Close the cluster, stop-hbase.sh
3. Turn on/off all regionserver, zookeeper,hbase-daemons.sh start/stop regionserver/zookeeper
4. Turn on/off single regionserver, zookeeper,hbase-daemon.sh start/stop regionserver/zookeeper
5. Turn on/off master hbase-daemon.sh start/stop Master, whether it becomes active master depends on whether there is currently active master
Two Advanced scripts
6.rolling-restart.sh can be used to scroll each restart
7.graceful_stop.sh Move server after all region, then stop/restart the server, can be used for version of the hot upgrade


Several details:
1. hbase-daemon.sh start Master and hbase-daemon.sh start Master--backup, the functions of these 2 commands, whether or not to be backup or active are controlled by Master's internal logic.


2. Stop-hbase.sh does not call hbase-daemons.sh stop regionserver to close regionserver, but calls hbase-daemons.sh stop zookeeper/ Master-backup to close ZK and backup master, close Regionserver actually calls the Hbaseadmin shutdown interface


3. $hbase_home/bin/hbase Stop master shuts down the entire cluster instead of a single master, using $hbase_home/bin/hbase-daemon.sh stop master only if you close a single master


4. $HBASE _home/bin/hbase Stop Regionserver/zookeeper cannot be tuned, the tune will go wrong, and no path will invoke this command, but it can be done by $hbase_home/bin/hbase start Regionserver/zookeeper to start RS or zk,hbase-daemon.sh call this command.