Ubuntu:glusterfs+hbase Installation Tutorials

HBase is typically installed on Hadoop HDFs, but it can also be installed on other distributed file systems that implement the Hadoop file interface. such as KFS.

Glusterfs is a clustered file system that can be extended to several peta-bytes.

It is a collection of various storage in InfiniBand RDMA or interconnect into a large parallel network file system. Storage can be made up of hardware x86_64server and Sata-ii and InfiniBand HBAs, regardless of commodity.

Glusterfs provides higher performance than Hadoop HDFs, so hbase can achieve higher performance. Install HBase on the glusterfs. is feasible. The scheme has been validated and adopted in this paper to keep it in mind.

Glusterfs is the core of the Scale-out storage solution Gluster, an open-source Distributed file system with strong scale-out capabilities that can support petabytes of storage capacity and handle thousands of of clients through expansion. The Glusterfs aggregates the physically distributed storage resources with TCP/IP or InfiniBand RDMA networks. Use a single global namespace to manage your data. The Glusterfs is based on a stackable user space design that provides superior performance for a variety of data loads.

I. Pre-conditions

The installation environment is virtual box VM, the operating system is Ubuntu Server 64 14.04. The installation user is bigdata.

1. Server List

192.168.8.108 bigdata1

192.168.8.110 Bigdata2

2. Change the hosts

sudo vi/etc/hosts

192.168.8.108 bigdata1

192.168.8.110 Bigdata2

3. SSH configuration

sudo su-root

passwd

Enter admin

Consent to root login with ssh password:

sudo vi/etc/ssh/sshd_config

# authentication: ? Logingracetime 120? #PermitRootLogin  without-password     #找到这里. Stare at it ? permitrootlogin  yes #改为yes    then restart SSH STRICTMODES  yes ???#service ssh Restart
#Tips: note here that/etc/init.d/ssh restart is unsuccessful in this way.

SSH configuration:

BIGDATA1:
#ssh-keygen-t rsa-p ""
#cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
#chmod ~/.ssh/authorized_keys

#scp ~/.ssh/authorized_keys [Email protected]:~/.ssh/

4. Java Installation Configuration

Download the latest version of Jdk64-bit (jdk-7u45-linux-x64.tar.gz)

Download http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

TAR-ZXVF jdk-7u51-linux-x64.tar.gz

sudo mv Jdk1.7.0_51/usr/share

environment variable Settings

sudo vi/etc/profile

On the last side of the file, add:

Export java_home=/usr/share/jdk1.7.0_51

Export JRE_HOME=/USR/SHARE/JDK1.7.0_51/JRE

Export path= $JAVA _home/bin: $JAVA _home/jre/bin: $PATH

Export classpath= $CLASSPATH:.: $JAVA _home/lib: $JAVA _home/jre/lib

Save, Source/etc/profile

5. Firewall Uninstall

Sudoapt-get Remove Iptables

Second, Glusterfs installation

1, installation Glusterfs

sudo apt-get install glusterfs-server

sudo glusterfsd–version

2. Create a folder

sudo mkdir-p/data/gluster

3. Create a volume

sudo gluster volume create Gluster_vol Replica 2 transport TCP Bigdata1:/data/gluster Bigdata2:/data/gluster

Note:

Delete Volume: sudo gluster volume delete Gluster_vol

Forced creation: Add force

4. Stop/Start Volume

sudo gluster volume stop Gluster_vol

sudogluster volume start Gluster_vol

5. View Volume Information

Gluster Volume Info

6. Mounting volume

sudo mount.glusterfs bigdata1:/gluster_vol/mnt/hbase

Mount.glusterfs Bigdata1:/gluster_vol/mnt/hbase

6. Quiz paper

df-h

BIGDATA1:

echo Test > Test1.txt

BIGDATA2:

Cat Test1.txt

At this point, Glusterfs successfully installed on Ubuntu server.

The following is the setup configuration for HBase.

Three, HBase installation

Proven by testing: In distributed mode, Glusterfs can replace Hadoop's HDFs as the underlying data store for HBase.
The configuration steps are as follows:
First of all, we introduce the cluster model under construction:
A total of 2 server,hostname are: Bigdata1 bigdata2 (each server can ssh without password login)
Hmaseter is: bigdata1
Hregionserver is: bigdata1
Zookeeper: Bigdata1 (HBase built-in zookeeper)
The HBase folder is created under the/mnt folder of the bigdata1, Bigdata2 server as the mount point of the Glusterfs (that is, the Glusterfs client), that is, each server has a/mnt/hbae/folder as

Glusterfs client, then let HBase write the data to this folder below.

Then change the configuration file for HBase: hbase-env.sh hbase-site.xml regionservers. These 3 profiles are all under the/usr/local/hbase/conf/folder

1. Download/install

wget http://apache.fayea.com/apache-mirror/hbase/stable/hbase-0.98.7-hadoop2-bin.tar.gz

Tar xvzfhbase-0.98.7-hadoop2-bin.tar.gz

sudo mvhbase-0.98.7-hadoop2/usr/local

sudoln-shbase-0.98.7-hadoop2 hbase

2. Configuration

Profile: hbase-env.sh (bigdata1, bigdata2 config file same)

Export java_home=/usr/share/jdk1.7.0_51//jdk Path

Export Hbase_manages_zk=true//zookeeper is built-in or self-erecting zookeeper (true for use built-in)

Configuration file: Hbase-site.xml
<configuration>
<property>
<name>hbase.rootdir</name>
<value>file:///mnt/hbase/</value>//And the path of the glusterfsclient corresponds
</property>
<property>
<name>hbase.cluster.distributed</name>//true for distributed mode, false for standalone mode
<value>true</value>
</property>
<property>
<name>hbase.master</name>
<value>bigdata1:60000</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>//zookeeper is 221
<value>bigdata1</value>
</property>
</configuration>

Configuration file regionservers//hregionserver server
Bigdata1

3. Implementation

#cd/usr/local/hbase/bin

#./start-hbase.sh//Open HBase cluster
#./hbase shell//Open hbase command
#hbase (main):001:0> status
2 servers, 0 dead, 0.7500 average load
The configuration was successful.

Four, test

#cd/usr/local/hbase/bin

HBase Shell

Create "test" "CF"

Reading the data from all tables on the BIGDATA2, assuming that the data in all the tables is read, indicates whether the underlying Glusterfs shares the data, Glusterfs can replace HDFs.

V. Conclusion

So far. This program Test success! Glusterfs is able to replace HDFs for higher performance.

Of course, further validation is required in the production environment.

Ubuntu:glusterfs+hbase Installation Tutorials