HBase 0.94.21 + ZooKeeper-3.4.6 distributed Installation

HBase 0.94.21 + ZooKeeper-3.4.6 distributed Installation

HBase 0.94.21 + ZooKeeper-3.4.6 distributed Installation

1: Configure hosts and ssh login without a password

Hadoop + HBase cloud storage creation summary PDF

Regionserver startup failed due to inconsistent time between HBase nodes

Hadoop + ZooKeeper + HBase cluster configuration

Hadoop cluster Installation & HBase lab environment setup

HBase cluster configuration based on Hadoop cluster'

Hadoop installation and deployment notes-HBase full distribution mode installation

Detailed tutorial on creating HBase environment for standalone Edition

2: Configure hbase

Decompress tar zxf hbase-0.94.21.tar.gz

Configure hbase-env.sh

[Bkjia @ bkjia02 hbase-0.94.21] $ cd conf
[Bkjia @ bkjia02 conf] $ cat hbase-env.sh
#
#/**
# * Copyright 2007 The Apache Software Foundation
#*
# * Licensed to the Apache Software Foundation (ASF) under one
# * Or more contributor license agreements. See the NOTICE file
# * Distributed with this work for additional information
# * Regarding copyright ownership. The ASF licenses this file
# * To you under the Apache License, Version 2.0 (
# * "License"); you may not use this file except T in compliance
# * With the License. You may obtain a copy of the License
#*
# * Http://www.apache.org/licenses/LICENSE-2.0
#*
# * Unless required by applicable law or agreed to in writing, software
# * Distributed under the License is distributed on an "as is" BASIS,
# * Without warranties or conditions of any kind, either express or implied.
# * See the License for the specific language governing permissions and
# * Limitations under the License.
#*/

# Set environment variables here.

# This script sets variables multiple times over the course of starting an hbase process,
# So try to keep things idempotent unless you want to take an even deeper look
# Into the startup scripts (bin/hbase, etc .)

# The java implementation to use. Java 1.6 required.
Export JAVA_HOME = $ HOME/jdk1.7.0 _ 45

# Extra Java CLASSPATH elements. Optional.
Export HBASE_CLASSPATH = $ HOME/hadoop/hadoop-1.2.1/conf

# The maximum amount of heap to use, in MB. Default is 1000.
# Export HBASE_HEAPSIZE = 1000

# Extra Java runtime options.
# Below are what we set by default. May only work with sun jvm.
# For more on why as well as other possible settings,
# See http://wiki.apache.org/hadoop/PerformanceTuning
Export HBASE_OPTS = "-XX: + UseConcMarkSweepGC"

# Uncomment one of the below three options to enable java garbage collection logging for the server-side processes.

# This enables basic gc logging to the. out file.
# Export SERVER_GC_OPTS = "-verbose: gc-XX: + PrintGCDetails-XX: + PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file (. gc) wocould still be generated in the HBASE_LOG_DIR.
# Export SERVER_GC_OPTS = "-verbose: gc-XX: + PrintGCDetails-XX: + PrintGCDateStamps-Xloggc: <FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0 _ 34 + and 1.7.0 _ 2 +.
# If FILE-PATH is not replaced, the log file (. gc) wocould still be generated in the HBASE_LOG_DIR.
# Export SERVER_GC_OPTS = "-verbose: gc-XX: + PrintGCDetails-XX: + PrintGCDateStamps-Xloggc: <FILE-PATH>-XX: + UseGCLogFileRotation-XX: numberOfGCLogFiles = 1-XX: GCLogFileSize = 512 M"

# Uncomment one of the below three options to enable java garbage collection logging for the client processes.

# This enables basic gc logging to the. out file.
# Export CLIENT_GC_OPTS = "-verbose: gc-XX: + PrintGCDetails-XX: + PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file (. gc) wocould still be generated in the HBASE_LOG_DIR.
# Export CLIENT_GC_OPTS = "-verbose: gc-XX: + PrintGCDetails-XX: + PrintGCDateStamps-Xloggc: <FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0 _ 34 + and 1.7.0 _ 2 +.
# If FILE-PATH is not replaced, the log file (. gc) wocould still be generated in the HBASE_LOG_DIR.
# Export CLIENT_GC_OPTS = "-verbose: gc-XX: + PrintGCDetails-XX: + PrintGCDateStamps-Xloggc: <FILE-PATH>-XX: + UseGCLogFileRotation-XX: numberOfGCLogFiles = 1-XX: GCLogFileSize = 512 M"

# Uncomment below if you intend to use the EXPERIMENTAL off heap cache.
# Export HBASE_OPTS = "$ HBASE_OPTS-XX: MaxDirectMemorySize ="
# Set hbase. offheapcache. percentage in hbase-site.xml to a nonzero value.

# Uncomment and adjust to enable JMX exporting
# See jmxremote. password and jmxremote. access in $ JRE_HOME/lib/management to configure remote password access.
# More details at: http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
#
# Export HBASE_JMX_BASE = "-Dcom. sun. management. jmxremote. ssl = false-Dcom. sun. management. jmxremote. authenticate = false"
# Export HBASE_MASTER_OPTS = "$ HBASE_MASTER_OPTS $ HBASE_JMX_BASE-Dcom. sun. management. jmxremote. port = 10101"
# Export HBASE_REGIONSERVER_OPTS = "$ HBASE_REGIONSERVER_OPTS $ HBASE_JMX_BASE-Dcom. sun. management. jmxremote. port = 10102"
# Export HBASE_THRIFT_OPTS = "$ HBASE_THRIFT_OPTS $ HBASE_JMX_BASE-Dcom. sun. management. jmxremote. port = 10103"
# Export HBASE_ZOOKEEPER_OPTS = "$ HBASE_ZOOKEEPER_OPTS $ HBASE_JMX_BASE-Dcom. sun. management. jmxremote. port = 10104"

# File naming hosts on which HRegionServers will run. $ HBASE_HOME/conf/regionservers by default.
# Export HBASE_REGIONSERVERS =$ {HBASE_HOME}/conf/regionservers

# File naming hosts on which backup HMaster will run. $ HBASE_HOME/conf/backup-masters by default.
# Export HBASE_BACKUP_MASTERS =$ {HBASE_HOME}/conf/backup-masters

# Extra ssh options. Empty by default.
# Export HBASE_SSH_OPTS = "-o ConnectTimeout = 1-o SendEnv = HBASE_CONF_DIR"

# Where log files are stored. $ HBASE_HOME/logs by default.
# Export HBASE_LOG_DIR =$ {HBASE_HOME}/logs

# Enable remote JDWP debugging of major HBase processes. Meant for Core Developers
# Export HBASE_MASTER_OPTS = "$ HBASE_MASTER_OPTS-Xdebug-Xrunjdwp: transport = dt_socket, server = y, suspend = n, address = 8070"
# Export HBASE_REGIONSERVER_OPTS = "$ HBASE_REGIONSERVER_OPTS-Xdebug-Xrunjdwp: transport = dt_socket, server = y, suspend = n, address = 8071"
# Export HBASE_THRIFT_OPTS = "$ HBASE_THRIFT_OPTS-Xdebug-Xrunjdwp: transport = dt_socket, server = y, suspend = n, address = 8072"
# Export HBASE_ZOOKEEPER_OPTS = "$ HBASE_ZOOKEEPER_OPTS-Xdebug-Xrunjdwp: transport = dt_socket, server = y, suspend = n, address = 8073"

# A string representing this instance of hbase. $ USER by default.
# Export HBASE_IDENT_STRING = $ USER

# The scheduling priority for daemon processes. See 'man nice '.
# Export HBASE_NICENESS = 10

# The directory where pid files are stored./tmp by default.
Export HBASE_PID_DIR =/home/bkjia/pids

# Seconds to sleep between slave commands. Unset by default. This
# Can be useful in large clusters, where, e.g., slave rsyncs can
# Otherwise arrive faster than the master can service them.
# Export HBASE_SLAVE_SLEEP = 0.1

# Tell HBase whether it shoshould manage it's own instance of Zookeeper or not.
Export HBASE_MANAGES_ZK = true

Modified location
# JDK installation directory
Export JAVA_HOME = $ HOME/jdk1.7.0 _ 45
# Hadoop configuration directory
Export HBASE_CLASSPATH = $ HOME/hadoop/hadoop-1.2.1/conf

And the last line

Export HBASE_MANAGES_ZK = true

# True: indicates that zookeeper is handed over to hbase management. When hbase is started, hbase in the hbase-site.xml is automatically started. zookeeper. all zookeeper instances in the quorum attribute # false: indicates that all zookeeper instances must be manually started when hbase is started.

Configure hbase-site.xml

[Bkjia @ bkjia02 conf] $ cat hbase-site.xml
<? Xml version = "1.0"?>
<? Xml-stylesheet type = "text/xsl" href = "configuration. xsl"?>
<! --
/**
* Copyright 2010 The Apache Software Foundation
*
* Licensed to the Apache Software Foundation (ASF) under one
* Or more contributor license agreements. See the NOTICE file
* Distributed with this work for additional information
* Regarding copyright ownership. The ASF licenses this file
* To you under the Apache License, Version 2.0 (
* "License"); you may not use this file except T in compliance
* With the License. You may obtain a copy of the License
*
* Http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* Distributed under the License is distributed on an "as is" BASIS,
* Without warranties or conditions of any kind, either express or implied.
* See the License for the specific language governing permissions and
* Limitations under the License.
*/
-->
<Configuration>
<Property>
<Name> hbase. rootdir </name>
<Value> hdfs: // bkjia001: 9000/user/bkjia/hbase </value>
</Property>
<Property>
<Name> hbase. cluster. distributed </name>
<Value> true </value>
</Property>
<Property>
<Name> hbase. master </name>
& Lt; value & gt; bkjia01: 60000 & lt;/value & gt;
</Property>
<Property>
<Name> hbase. zookeeper. quorum </name>
<Value> bkjia01, bkjia02 </value>
</Property>

</Configuration>

Configure regionservers

[Bkjia @ bkjia02 conf] $ cat regionservers
Bkjia01
Bkjia02

For more details, please continue to read the highlights on the next page:

1 2 Next Page