Ubuntu14.04 install distributed storage Sheepdog + ZooKeeper

Sheepdog is an open-source project developed by three Japanese researchers from NTT and provides distributed storage management. The sheepdog provides high-availability KVM to provide block-level storage volumes similar to clients in Amazon's electronic trading system (Elastic Block Storage virtual machines. Alibaba and other large companies in China are currently using this service. I. Sheepdog architecture diagram: using a fully symmetrical architecture with no central nodes, no single point of failure, linear expansion of storage capacity and performance; newly added nodes can be automatically added through simple configuration (IP: PORT), data is automatically implemented

Sheepdog is an open-source project developed by three Japanese researchers from NTT and provides distributed storage management. The sheepdog provides high-availability KVM to provide block-level storage volumes similar to clients in Amazon's electronic trading system (Elastic Block Storage virtual machines. Alibaba and other large companies in China are currently using this service.

1. Sheepdog Architecture

For example:

It adopts a fully symmetrical architecture with no central nodes and has no single point of failure. The storage capacity and performance can be linearly expanded;

New nodes can be automatically added (IP: PORT) through simple configuration, and data is automatically used to achieve load balancing;

Data can be automatically restored when a node fails;

Directly supporting QEMU/KVM applications;

II. Specific Data Storage Methods

For example:

VM data is stored using VDI Object and a block device is exposed to users;

Includes four Data objects: VDI, Data Object, attribute Object, and VM real-time status Data Object used for snapshot;

OBS is implemented using a small file of 4 MB, but it is easy to use this extension. For example, use a library to replace a small file of 4 MB;

Next, we will use the Sheepdog + Zookeeper method.

Environment Introduction:

System: Ubuntu 14.04

Software: Sheepdog, Zookeeper

This time we used the source code on github and compiled it into a deb package for installation.

1. Obtain the source code of sheepdog:

Root @ node1 :~ # Git clone https://github.com/sheepdog/sheepdog.git sheepdog-github

2. Switch to the most stable version 0.7.6:

Root @ node1 :~ # Cd sheepdog-github/

Root @ node1 :~ # Git checkout v0.7.6

4. Execute autogen. sh to generate configure. Before that, dependencies may need to be installed:

Root @ node1 :~ # Apt-get install autoconf libtool make pkg-config

Root @ node1 :~ #./Autogen. sh

5. Start compilation. sheepdog depends on corosync, zookeeper, and other cluster management tools. Because we make deb packages, we also rely on some debian tools:

Root @ node1 :~ # Apt-get install liburcu-dev corosync-dev zookeeper zookeeperd libzookeeper-mt-dev

Root @ node1 :~ #./Configure -- enable-zookeeper

Root @ node1 :~ # Apt-get install debhelper dh-autoreconf devscripts

Root @ node1 :~ # Make deb

6. Start to install sheepdog:

Root @ node1 :~ # Cd ..

Root @ node1 :~ # Dpkg-I sheepdog_0.7.6-1_amd64.deb

7. Now sheepdog has been installed. If you want to copy this deb package to another machine (node2) for installation, you must install the corresponding dependency package on that machine:

Root @ node1 :~ # Apt-get install zookeeperd libcfg4 libg4 libcpg4 libzookeeper-mt2 libcoroipcc4

Note: The libcfg4 package cannot be found on ubuntu14.04. Because libcfg4 is a package on 12.04, you need to add a package in/etc/apt/sources. list.

Deb http://cz.archive.ubuntu.com/ubuntu precise main

8. Configure zookeeper:

Generally, zookeeper is used as the sheepdog cluster management tool in the official environment. Related configurations are required. If you only use a single node for trial, you do not need to configure zookeeper.

Modify the myid file first. The number of nodes you configure is 1-255;

Root @ node1 :~ # Vi/etc/zookeeper/conf/myid

9. Modify the zoo. cfg file and fill in the information of each zookeeper node:

Root @ node1 :~ # Vi/etc/zookeeper/conf/zoo. cfg

Server.1 = 10.0.0.18: 2888: 3888

Server.2 = 10.0.0.19: 2889: 3889

I have two nodes and only two nodes. The format is server. myid = ip + port number;

The first port is used for information exchange among cluster members, and the second port is used for selecting a leader when the leader fails.

10. Restart zookeeper;

Root @ node1 :~ # Service zookeeper restart

11. Start sheepdog and mount the disk;

Root @ node1 :~ # Mkdir/mnt/sheepdog

Root @ node1 :~ # Mount-t ext4-o noatime, barrier = 0, user_xattr, data = writeback/dev/sdb1/mnt/sheepdog

Root @ node1 :~ # Useradd sheepdog

Root @ node1 :~ # Chown-R sheepdog: sheepdog/mnt/sheepdog

Root @ node1 :~ # Sheep/mnt/sheepdog-c zookeeper: 10.0.0.18: 2181, 10.0.0.19: 2181

Common Sheepdog commands

Set the number of replicas:

Dog cluster format -- copies = 3 // number of copies replicas

You can view the sheepdog node in either of the following ways:

CentOS 6 installation sheepdog Virtual Machine distributed storage http://www.linuxidc.com/Linux/2013-08/89109.htm

ZooKeeper cluster configuration http://www.linuxidc.com/Linux/2013-06/86348.htm

Use ZooKeeper to implement distributed shared lock http://www.linuxidc.com/Linux/2013-06/85550.htm

Distributed service framework ZooKeeper-manage data http://www.linuxidc.com/Linux/2013-06/85549.htm in distributed environment

ZooKeeper Cluster Environment Construction Practice http://www.linuxidc.com/Linux/2013-04/83562.htm

ZooKeeper server cluster environment configuration test http://www.linuxidc.com/Linux/2013-04/83559.htm

ZooKeeper cluster installation http://www.linuxidc.com/Linux/2012-10/72906.htm

For more information about Ubuntu, see Ubuntu special page http://www.linuxidc.com/topicnews.aspx? Tid = 2

This article permanently updates the link address: Http://www.linuxidc.com/Linux/2014-12/110352.htm