CentOS7 install Ceph

CentOS7 install Ceph

1. installation environment

| ----- Node1 (mon, osd) sda is the system disk, and sdb and sdc are the osd disks.

|

| ----- Node2 (mon, osd) sda is the system disk, and sdb and sdc are the osd disks.

Admin ----- |

| ----- Node3 (mon, osd) sda is the system disk, and sdb and sdc are the osd disks.

|

| ----- Client

Ceph Monitors uses port 6789 for communication by default, and OSD uses port 6800: 7300 for communication by default.

2. Preparations (all nodes)

2.1 modify an IP address

Vim/etc/sysconfig/network-scripts/ifcfg-em1

IPADDR = 192.168.130.205

NETMASK = 255.255.255.0

GATEWAY = 192.168.130.2

2.2 disable the Firewall

Systemctl stop firewalld. service # stop firewall

Systemctl disable firewalld. service # disable firewall startup

Firewall-cmd -- state # view the firewall Status

2.3 modify yum Source

Cd/etc/yum. repos. d

Mv CentOS-Base.repo CentOS-Base.repo.bk

Wget http://mirrors.163.com/.help/CentOS6-Base-163.repo

Yum makecache

2.4 modify the time zone

Cp/usr/share/zoneinfo/Asia/Shanghai/etc/localtime

Yum-y install ntp

Systemctl enable ntpd

Systemctl start ntpd

Ntpstat

2.5 modify hosts

Vim/etc/hosts

192.168.130.205 admin

192.168.130.204 client

192.168.130.203 node3

192.168.130.202 node2

192.168.130.201 node1

2.6 install the epel repository, add the yum ceph repository, and update the software repository.

Install epel Repository

Rpm-vih http://mirrors.sohu.com/fedora-epel/7/x86_64/e/epel-release-7-2.noarch.rpm

Add a yum ceph Repository

Vim/etc/yum. repos. d/ceph. repo

[Ceph]

Name = Ceph noarch packages

Base url = http://mirrors.163.com/ceph/rpm-hammer/el7/x86_64/

Enabled = 1

Gpgcheck = 1

Type = rpm-md

Gpgkey = http://mirrors.163.com/ceph/keys/release.asc

2.7 install ceph-deploy and ceph (ceph is installed on all nodes, and ceph-deploy is only installed on the admin node)

Yum-y update & yum-y install -- release hammer ceph-deploy

3. Allow SSH login without a password (admin node)

3.1 generate an SSH key pair and press Enter when "Enter passphrase" is displayed. The password is blank:

Ssh-keygen

3.2. Copy the public key to all nodes.

Ssh-copy-id root @ node1

Ssh-copy-id root @ node2

Ssh-copy-id root @ node3

Ssh-copy-id root @ client

3.3 verify that you can log on without a password through SSH

Ssh node1

Ssh node2

Ssh node3

Ssh client

4. Create a Monitor (admin node)

4.1 create a monitor on node1, node2, and node3

Mkdir myceph

Cd myceph

Ceph-deploy new node1 node2 node3

4.2 modify the number of copies of osd and add the default size of osd pool = 2 to the end

Vim/etc/ceph. conf

Osd pool default size = 2

4.3 configure the initial monitor (s) and collect all keys

Ceph-deploy mon create-initial

5. Create An OSD (admin node)

5.1 list Disks

Ceph-deploy disk list node1

Ceph-deploy disk list node2

5.2. Clean the disk

Ceph-deploy disk zap node1: sdb

Ceph-deploy disk zap node1: sdc

Ceph-deploy disk zap node2: sdb

Ceph-deploy disk zap node2: sdc

Ceph-deploy disk zap node3: sdb

Ceph-deploy disk zap node3: sdc

5.3 prepare OSD

Ceph-deploy osd prepare node1: sdb

Ceph-deploy osd prepare node1: sdc

Ceph-deploy osd prepare node2: sdb

Ceph-deploy osd prepare node2: sdc

Ceph-deploy osd prepare node3: sdb

Ceph-deploy osd prepare node3: sdc

Ceph-deploy osd activate node1: sdb1

Ceph-deploy osd activate node1: sdc1

Ceph-deploy osd activate node2: sdb1

Ceph-deploy osd activate node2: sdc1

Ceph-deploy osd activate node3: sdb1

Ceph-deploy osd activate node3: sdc1

5.4 Delete OSD

Ceph osd out osd.3

Ssh node1 service ceph stop osd.3

Ceph osd crush remove osd.3

Ceph auth del osd.3 // Delete from Authentication

Ceph osd rm 3 // Delete

5.5 copy the configuration file and admin key to each node, so that you do not need to specify the monitor address and Ceph. client. admin. keyring each time you execute the ceph command.

Ceph-deploy admin node1 node2 node3

5.6 view Cluster health status

Ceph health

6. Configure the block device (client node)

6.1 create an image

Rbd create foo -- size 4096 [-m {mon-IP}] [-k/path/to/ceph. client. admin. keyring]

Rbd create foo -- size 4096-m node1-k/etc/ceph. client. admin. keyring

6.2 map an image to a block Device

Sudo rbd map foo -- name client. admin [-m {mon-IP}] [-k/path/to/ceph. client. admin. keyring]

Sudo rbd map foo -- name client. admin-m node1-k/etc/ceph. client. admin. keyring

6.3 create a file system

Sudo mkfs. ext4-m0/dev/rbd/foo

6.4 mount a File System

Sudo mkdir/mnt/ceph-block-device

Sudo mount/dev/rbd/foo/mnt/ceph-block-device

Cd/mnt/ceph-block-device

1. List storage pools

Ceph osd lspools

2. Create a storage pool

Ceph osd pool create pool-name pg-num pgp-num

Ceph osd pool create test 512 512

3. delete a storage pool

Ceph osd pool delete test -- yes-I-really-mean-it

4. Rename the storage pool

Ceph osd pool rename current-pool-name new-pool-name

Ceph osd pool rename test test2

5. View storage pool statistics

Rados df

6. Adjust the storage pool option value

Ceph osd pool set test size 3 sets the number of object copies

7. Obtain the storage pool option value

Ceph osd pool get test size get object copy count

1. Create a block device Image

Rbd create -- size {megabytes} {pool-name}/{image-name}

Rbd create -- size 1024 test/foo

2. List block device Images

Rbd ls

3. Retrieve Image Information

Rbd info {image-name}

Rbd info foo

Rbd info {pool-name}/{image-name}

Rbd info test/foo

4. Adjust the image size of a block Device

Rbd resize -- size 512 test/foo -- allow-shrink smaller

Rbd resize -- size 4096 test/foo increase

5. Delete Block devices

Rbd rm test/foo

Kernel module operations

1. Map Block devices

Sudo rbd map {pool-name}/{image-name} -- id {user-name}

Sudo rbd map test/foo2 -- id admin

If you enable cephx authentication, you also need to specify the key

Sudo rbd map test/foo2 -- id admin -- keyring/etc/ceph. client. admin. keyring

2. view the mapped Devices

Rbd showmapped

3. Cancel block device ing

Sudo rbd unmap/dev/rbd/{poolname}/{imagename}

Rbd unmap/dev/rbd/test/foo2