CentOS7 Combat Kubernetes Deployment

CentOS 7 Build Glusterfs

Lab Requirements:

4 machines installed Glusterfs form a cluster

The client stores the Docker registry in the file system.

4-node hard disk space does not synthesize a hard disk space, requiring each node to store a copy to ensure data security

Environmental planning

-----------------------------------------------------

Server

-----------------------------------------------------

node1:192.168.56.11 Host Name: GLUSTERFS1

-----------------------------------------------------

node2:192.168.56.12 Host Name: GLUSTERFS2

-----------------------------------------------------

node3:192.168.56.13 Host Name: GLUSTERFS3

-----------------------------------------------------

NODE4:192.168.56.14 Host Name: GLUSTERFS4

-----------------------------------------------------

Client

-----------------------------------------------------

192.168.56.20 Host Name: Client

-----------------------------------------------------

Os:bclinux Release 7.2 (Mars)

Preparation before the experiment:

All hosts shutdown firewall, SELinux Modify Hosts file, can parse each other

192.168.56.11 GLUSTERFS1

192.168.56.12 GLUSTERFS2

192.168.56.13 GLUSTERFS3

192.168.56.14 GLUSTERFS4

192.168.56.20 Client

。 SSH Trust (preferably host name)

-->SSH-KEYGEN-T RSA

-->ssh-copy-id-i/root/.ssh/id_rsa.pub [email protected]

-->ssh-copy-id-i/root/.ssh/id_rsa.pub [email protected]

-->ssh-copy-id-i/root/.ssh/id_rsa.pub [email protected]

-->ssh-copy-id-i/root/.ssh/id_rsa.pub [email protected]

-->ssh-copy-id-i/root/.ssh/id_rsa.pub [email protected]

First, installation

Service side

1. Install the Glusrerfs package on the glusterfs {1-4} node

。 Glusterfs:

http://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-api-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-cli-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-client-xlators-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-debuginfo-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-fuse-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-geo-replication-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-libs-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-rdma-3.7.17-1.el7.x86_64.rpm

wget https://download.gluster.org/pub/gluster/glusterfs/3.7/3.7.17/CentOS/epel-7.2/x86_64/ glusterfs-server-3.7.17-1.el7.x86_64.rpm

Installation:

-->yum Install Libibverbs librdmacm-y

-->wget http://dl.fedoraproject.org/pub/epel/6/x86_64//userspace-rcu-0.7.7-1.el6.x86_64.rpm

-->RPM-IVH userspace-rcu-0.7.7-1.el6.x86_64.rpm

-->yum Localinstall *.rpm-y

----------------------------------------------------------------------------------------------

Error:

Required: Liburcu-bp.so.1 () (64bit)

Error: Dependency detection failed:

Libibverbs.so.1 () (64bit) required by glusterfs-rdma-3.7.17-1.el7.x86_64

Libibverbs.so.1 (ibverbs_1.0) (64bit) required by glusterfs-rdma-3.7.17-1.el7.x86_64

Libibverbs.so.1 (ibverbs_1.1) (64bit) required by glusterfs-rdma-3.7.17-1.el7.x86_64

Librdmacm.so.1 () (64bit) required by glusterfs-rdma-3.7.17-1.el7.x86_64

Librdmacm.so.1 (rdmacm_1.0) (64bit) required by glusterfs-rdma-3.7.17-1.el7.x86_64

Liburcu-bp.so.1 () (64bit) required by glusterfs-server-3.7.17-1.el7.x86_64

Liburcu-cds.so.1 () (64bit) required by glusterfs-server-3.7.17-1.el7.x86_64

Workaround:

-->yum Install Libibverbs librdmacm-y

-->wget http://dl.fedoraproject.org/pub/epel/6/x86_64//userspace-rcu-0.7.7-1.el6.x86_64.rpm

-->RPM-IVH userspace-rcu-0.7.7-1.el6.x86_64.rpm

----------------------------------------------------------------------------------------------

3.USERSPACE-RCU:

-->wget http://dl.fedoraproject.org/pub/epel/7/x86_64/u/userspace-rcu-0.7.16-1.el7.x86_64.rpm

-->RPM-IVH serspace-rcu-0.7.16-1.el7.x86_64.rpm

Installation:

-->yum Localinstall *.rpm

。 To view the installed version

-->glusterfs-v

Glusterfs 3.7.17 built on Nov 3 2016 14:29:48

Repository Revision:git://git.gluster.com/glusterfs.git

Copyright (c) 2006-2013 Red Hat, Inc.

GlusterFS comes with absolutely NO WARRANTY.

It's licensed to under your choice of the GNU Lesser

General public License, version 3 or any later version (LGPLv3

or later), or the GNU general public License, version 2 (GPLV2),

In any cases as published by the Free software Foundation.

。 Start the Glusterfs service

-->systemctl Enable Glusterd.service

Created symlink From/etc/systemd/system/multi-user.target.wants/glusterd.service to/usr/lib/systemd/system/ Glusterd.service.

-->systemctl Start Gluterd.service

-->systemctl Status Glusterd.service

Glusterd.service-glusterfs, a clustered File-system server

Loaded:loaded (/usr/lib/systemd/system/glusterd.service; enabled; vendor preset:disabled)

Active:active (running) since Sun 2017-07-09 16:20:51 CST; 30s ago

process:4684 execstart=/usr/sbin/glusterd-p/var/run/glusterd.pid--log-level $LOG _level $GLUSTERD _options (code= Exited, status=0/success)

Main pid:4685 (Glusterd)

CGroup:/system.slice/glusterd.service

└─4685/usr/sbin/glusterd-p/var/run/glusterd.pid--log-level INFO

Jul 16:20:48 GLUSTERFS1 systemd[1]: Starting GlusterFS, a clustered File-system server ...

Jul 16:20:51 GLUSTERFS1 systemd[1]: Started GlusterFS, a clustered File-system server.

2. Configure the entire Glusterfs cluster on the GLUSTERFS1 node to join each node to the cluster

-->gluster Peer probe GLUSTERFS1

Peer Probe:success. Probe on localhost not needed

-->gluster Peer probe GLUSTERFS2

Peer Probe:success.

-->gluster Peer probe GLUSTERFS3

Peer Probe:success.

-->gluster Peer probe Glusterfs4

Peer Probe:success.

3. View node Status

-->gluster Peer Status

Number of Peers:3

Hostname:glusterfs2

Uuid:9149c992-e86c-436d-8da9-b2a7245289cb

State:peer in Cluster (Connected)

Hostname:glusterfs3

Uuid:dcbe562d-5bbd-4ee8-8855-c48428bba7ef

State:peer in Cluster (Connected)

Hostname:glusterfs4

uuid:677514f3-7c3c-4b8e-b93d-db3941485166

State:peer in Cluster (Connected)

4. Create a data store directory on Glusterfs{1-4}

-->mkdir-p/usr/local/share/models

5. Create a glusterfs disk on GLUSTERFS1

Attention:

Plus replica 4 is 4 nodes, each node is to store data once, is a data storage 4, each node one copy

If you do not add replica 4, the 4-node disk space is integrated into one hard disk,

-->gluster Volume Create models replica 4 glusterfs1:/usr/local/share/models glusterfs2:/usr/local/share/models Glusterfs3:/usr/local/share/models glusterfs4:/usr/local/share/models Force

Volume Create:models:success:please start the volume to access data

6. Start

-->gluster Volume start Models

Volume start:models:success

----------------------------------------------------------------------------

Client

1. Deploy the Glusterfs client and mount the Glusterfs file system

-->mkdir/usr/local/src/rpm

Method with server side (download RPM installation)

-->yum Install Libibverbs librdmacm-y

-->wget http://dl.fedoraproject.org/pub/epel/6/x86_64//userspace-rcu-0.7.7-1.el6.x86_64.rpm

-->RPM-IVH userspace-rcu-0.7.7-1.el6.x86_64.rpm

-->yum Localinstall *.rpm-y

It is best to install the client package as follows

-->RPM-IVH glusterfs-client-xlators-3.7.17-1.el7.x86_64.rpm glusterfs-fuse-3.7.17-1.el7.x86_64.rpm

-->mkdir-p/mnt/models

-->mount-t glusterfs-o ro glusterfs1:models/mnt/models/

2. View Effects

-->df-h

Df-ht

Filesystem Type Size used Avail use% mounted on

/dev/sda2 ext4 50G 1.5G 46G 4%/

Devtmpfs Devtmpfs 904M 0 904M 0%/dev

Tmpfs tmpfs 913M 0 913M 0%/dev/shm

Tmpfs tmpfs 913M 8.8M 904M 1%/run

Tmpfs tmpfs 913M 0 913M 0%/sys/fs/cgroup

/dev/sda6 ext4 35G 4.7G 29G 15%/var

/dev/sda3 ext4 9.8G 37M 9.2G 1%/Home

/DEV/SDA1 ext4 477M 94M 354M 21%/boot

Tmpfs tmpfs 183M 0 183M 0%/run/user/0

Glusterfs1:models fuse.glusterfs 50G 1.5G 46G 4%/mnt/models

Other Operations Commands

。 Delete Glusterfs disk

-->gluster volume stop models stops first

-->gluster Volume Delete Models

。 Uninstalling the Glusterfs disk

-->gluster Peer Detach Glusterfs4

。 ACL access control

-->gluster Volume set models auth.allow 192.168.56.*,10.0.1.*

Volume set:success

。 Add Glusterfs Node

-->gluster Peer probe Sc2-log5

-->gluster Peer probe Sc2-log6

-->gluster Volume Add-brick models sc2-log5:/data/gluster Sc2-log6:/data/gluster

。 Migrating Glusterfs Data

-->gluster Volume Remove-brick models sc2-log1:/usr/local/share/models sc2-log5:/usr/local/share/models start

-->gluster Volume Remove-brick models sc2-log1:/usr/local/share/models sc2-log5:/usr/local/share/models Status

-->gluster Volume Remove-brick models sc2-log1:/usr/local/share/models Sc2-log5:/usr/local/share/models Commit

。 Repairing Glusterfs data (in the case of Node 1 outage)

-->gluster Volume Replace-brick models sc2-log1:/usr/local/share/models sc2-log5:/usr/local/share/models Commit- Force

-->gluster Volume Heal Models full

CentOS7 Combat Kubernetes Deployment