CentOS distributed environment HBase-0.96.0 Installation

1. Install zookeeper in the environment where distributed Hadoop is installed, continue to install zookeeper-3.4.5.

1. $ Wget http: // 124.2020.4.9/download/32878722/41763290/4/gz/165/152/13535668471__152/zookeeper-3.4.5.tar.gz
2. $ Tar-xf zookeeper-3.4.5.tar.gz
3. $ Cp zookeeper-3.4.5/conf/zoo_sample.cfg zookeeper-3.4.5/conf/zoo. cfg

$ wget http://124.202.164.9/download/32878722/41763290/4/gz/165/152/1353566847397_152/zookeeper-3.4.5.tar.gz$ tar -xf zookeeper-3.4.5.tar.gz$ cp zookeeper-3.4.5/conf/zoo_sample.cfg zookeeper-3.4.5/conf/zoo.cfg 

Modify the zookeeper configuration file:

1. $ Vim zookeeper-3.4.5/conf/zoo. cfg

$ vim zookeeper-3.4.5/conf/zoo.cfg

Make the following changes:

1. TickTime = 2000
2. DataDir =/home/ruifeng. shan/zookeeper
3. ClientPort = 2181
4. InitLimit = 5
5. SyncLimit = 2
6. Server.1 = master: 2888: 3888
7. Server.2 = node: 2888: 3888
8. Server.3 = slave: 2888: 3888

tickTime=2000  dataDir=/home/ruifeng.shan/zookeeper  clientPort=2181  initLimit=5  syncLimit=2  server.1=master:2888:3888  server.2=node:2888:3888  server.3=slave:2888:3888

Create a file myid in the dataDir directory you specified and add the specified server number to it. For example, you specify the following configuration server.1 = master: 2888: 3888 for the hostname named master, so its number should be 1. Distribute the zookeeper-3.4.5 to another machine. Modify the content of the corresponding myid file to the ID specified previously. For example, if node is 2 and slave is 3, the zookeeper service is started separately. Switch to the zookeeper-3.4.5 directory on each machine and run $ bin/zkServer. sh start to start the zookeeper service. 2. Install the hbase-0.96.0 in order to match the hadoop-2.2.0, the selected hbase version is 0.96.0

1. $ Wget http://mirror.bit.edu.cn/apache/hbase/hbase-0.96.0/hbase-0.96.0-hadoop2-bin.tar.gz
2. $ Tar-xf hbase-0.96.0-hadoop2-bin.tar.gz
3. $ Hbase-0.96.0 cd

$ wget http://mirror.bit.edu.cn/apache/hbase/hbase-0.96.0/hbase-0.96.0-hadoop2-bin.tar.gz$ tar -xf hbase-0.96.0-hadoop2-bin.tar.gz$ cd  hbase-0.96.0

Modify the configuration in conf:

1. Vim conf/hbase-env.sh

 vim conf/hbase-env.sh

Modify JAVA_HOME:

1. # The java implementation to use. Java 1.6 required.
2. Export JAVA_HOME =/usr/lib/jvm/java/jdk1.6.0 _ 37

# The java implementation to use.  Java 1.6 required.export  JAVA_HOME=/usr/lib/jvm/java/jdk1.6.0_37

Modify how HBase uses zookeeper:

1. # Tell HBase whether it shoshould manage it's own instance of Zookeeper or not.
2. Export HBASE_MANAGES_ZK = false

# Tell HBase whether it should manage it's own instance of Zookeeper or not.export HBASE_MANAGES_ZK=false

The default value of this parameter is true. Whether to allow hbase to manage zookeeper separately. If zookeeper has been installed on the machine to be installed, and zookeeper is shared by multiple platforms, set this parameter to false. If zookeeper is not installed on the cluster and hbase only uses zookeeper, set this parameter to true. Modify the hbase-site.xml file under its conf directory. The content is as follows:

1. <Configuration>
2. <Property>
3. <Name> hbase. rootdir </name>
4. <Value> hdfs: // master: 8089/hbase </value>
5. </Property>
7. <Property>
8. <Name> hbase. master </name>
9. <Value> hdfs: // master: 60000 </value>
10. </Property>
12. <Property>
13. <Name> hbase. cluster. distributed </name>
14. <Value> true </value>
15. </Property>
17. <Property>
18. <Name> hbase. zookeeper. property. clientPort </name>
19. <Value> 2181 </value>
20. </Property>
22. <Property>
23. <Name> hbase. zookeeper. quorum </name>
24. <Value> master, node, and slave </value>
25. </Property>
27. <Property>
28. <Name> hbase. zookeeper. property. dataDir </name>
29. <Value>/home/aaron/zookeeper </value>
30. </Property>
32. <Property>
33. <Name> hbase. client. caching. caching </name>
34. <Value> 200 </value>
35. </Property>
37. <Property>
38. <Name> hbase. balancer. period </name>
39. <Value> 300000 </value>
40. </Property>
42. <Property>
43. <Name> hbase. client. write. buffer </name>
44. <Value> 10485760 </value>
45. </Property>
47. <Property>
48. <Name> hbase. hregion. majorcompaction </name>
49. <Value> 7200000 </value>
50. </Property>
52. <Property>
53. <Name> hbase. hregion. max. filesize </name>
54. <Value> 67108864 </value>
55. <Description>
56. Maximum HStoreFile size. If any one of a column families 'hstorefiles has
57. Grown to exceed this value, the hosting HRegion is split in two. </description>
58. </Property>
60. <Property>
61. <Name> hbase. hregion. memstore. flush. size </name>
62. <Value> 1048576 </value>
63. <Description>
64. Memstore will be flushed to disk if size of the memstore
65. Exceeds this number of bytes. Value is checked by a thread that runs
66. Every hbase. server. thread. wakefrequency. </description>
67. </Property>
69. <Property>
70. <Name> hbase. server. thread. wakefrequency </name>
71. <Value> 30000 </value>
72. <Description> Time to sleep in between searches for work (in milliseconds ).
73. Used as sleep interval by service threads such as log roller. </description>
74. </Property>
76. </Configuration>

<configuration><property><name>hbase.rootdir</name><value>hdfs://master:8089/hbase</value></property><property><name>hbase.master</name><value>hdfs://master:60000</value></property><property><name>hbase.cluster.distributed</name><value>true</value></property><property><name>hbase.zookeeper.property.clientPort</name><value>2181</value></property><property><name>hbase.zookeeper.quorum</name><value>master,node,slave</value></property><property><name>hbase.zookeeper.property.dataDir</name><value>/home/aaron/zookeeper</value></property><property><name>hbase.client.scanner.caching</name><value>200</value></property><property><name>hbase.balancer.period</name><value>300000</value></property><property><name>hbase.client.write.buffer</name><value>10485760</value></property><property><name>hbase.hregion.majorcompaction</name><value>7200000</value></property><property><name>hbase.hregion.max.filesize</name><value>67108864</value><description>Maximum HStoreFile size. If any one of a column families' HStoreFiles hasgrown to exceed this value, the hosting HRegion is split in two.</description></property><property><name>hbase.hregion.memstore.flush.size</name><value>1048576</value><description>Memstore will be flushed to disk if size of the memstoreexceeds this number of bytes. Value is checked by a thread that runsevery hbase.server.thread.wakefrequency.</description></property><property><name>hbase.server.thread.wakefrequency</name><value>30000</value><description>Time to sleep in between searches for work (in milliseconds).Used as sleep interval by service threads such as log roller.</description></property></configuration>

Hbase. rootdir must be consistent with the value in fs. default. name in the hadoop core-site.xml file. The clock of the master and regionserver of hbase must be consistent. Otherwise, the following error will be reported when the regionserver is started: org. apache. hadoop. hbase. clockOutOfSyncException: org. apache. hadoop. hbase. clockOutOfSyncException: Server hadoop02, 60020,1374576225508 has been rejected; Reported time is too far out of sync with master. time difference of 62062623 ms> max allowed of ms is caused by a large Time error between the master and regionserver. Solution 1: Modify the following parameters

1. <Name> hbase. master. maxclockskew </name>
2. <Value> 180000 </value>
3. <Description> Time difference of regionserver from master </description>

<name>hbase.master.maxclockskew</name><value>180000</value><description>Time difference of regionserver from master</description>

This parameter defaults to 30 000 (30 seconds) solution 2: Use ntpdate to synchronize the cluster clock to modify the regionservers file configuration: this file is used to specify the regionserver machine, one line, generally, datanode is used as the regionserver:

1. $ Vim conf/regionservers

$ vim conf/regionservers

Masternodeslave configuration backup-masters:

1. $ Vim conf/backup-masters

$ vim conf/backup-masters

Master-backup configuration backup-masters can be automatically started after the master is down Note: to be compatible with the Hadoop-2.2.0, you need to replace the hadoop-common-2.1.0-beta.jar in the content lib package in the hbase lib package with the hadoop-2.2.0 under the hadoop-common-2.2.0.jar/share/hadoop/common directory. Start hbase: if each machine in the cluster has ssh password-less login, you can directly use the start-hbase.sh to start the entire cluster. Or start with the daemon script on each machine (switch to the hbase directory): To the master machine: bin/hbase-daemon.sh start master to the regionserver machine: bin/hbase-daemon.sh start regionserver check cluster normal: view master monitoring page: http: // master: 60010

HBase details: click here
HBase: click here

Hadoop + HBase cloud storage creation summary PDF

Regionserver startup failed due to inconsistent time between HBase nodes

Hadoop + ZooKeeper + HBase cluster configuration

Hadoop cluster Installation & HBase lab environment setup

HBase cluster configuration based on Hadoop cluster'

Hadoop installation and deployment notes-HBase full distribution mode installation

Detailed tutorial on creating HBase environment for standalone Edition