High-availability MySQL Cluster service with DRBD combined with Corosync

drbd:distributed replicated block Dvice distributed replication block device, which can be two host hard disk or partition into a mirrored device, similar to the principle of RAID1, but it will be the primary node of the data actively synchronized to the node from the network to mirror, When the primary node fails, the slave node becomes the primary node because it is a mirrored device, so the data is not lost. Corosync's role here is to make DRBD a resource for high-availability services through pacemaker, allowing for automatic switching between master and slave nodes. DRBD is a cost-saving solution for scenarios that require shared storage because of the use of their own hard disk devices between nodes.

first, the experimental planning:

1, This experiment uses two nodes Node1 and Node2, the system platform adopts Redhat 5.8 x86

The IP address of the Node1 is: 192.168.1.65

The IP address of the Node2 is: 192.168.1.66

The virtual VIP is: 192.168.1.60

2. Install the DRBD-related package, configure and start DRBD, and test if DRBD is working properly

3, install MySQL at each node, use mysql-5.5.42 to use common binary installation, and test login

4, installing Corosync and pacemaker, configuring DRBD and MySQL as resources for high-availability MySQL-based clusters

two, the experimental steps:

1, installation configuration DRBD

(1) Preparatory work

A. Modify the hostname and make sure it remains in effect after a reboot

Node1:

# sed-i ' [email protected]\ (hostname=\). *@\1node1.test.com '/etc/sysconfig/network# HOSTNAME node1.test.com

Node2:

# sed-i ' [email protected]\ (hostname=\). *@\1node2.test.com '/etc/sysconfig/network# HOSTNAME node2.test.com

B. Ensure the hostname resolution of two nodes is normal, modify the/etc/hosts file

192.168.1.65 node1.test.com node1192.168.1.66 node2.test.com node2

C. Configure SSH based on key authentication (dual-machine Trust)

Node1:

# ssh-keygen-t rsa# ssh-copy-id-i ~/.ssh/id_rsa.pub [email protected]

Node2:

# ssh-keygen-t rsa# ssh-copy-id-i ~/.ssh/id_rsa.pub [email protected]

d. Download and install the DRBD kernel module and the User Space tool, which uses the drbd8.3 version for http://mirrors.sohu.com/centos/5/extras/i386/RPMS/(here is centos5, For other versions please download it yourself) drbd83-8.3.8-1.el5.centos.i386.rpm (User Space Command line tool) kmod-drbd83-8.3.8-1.el5.centos.i686.rpm (DRBD kernel module)

Install separately on Node1 and Node2:

# yum-y--nogpgcheck localinstall drbd83-8.3.8-1.el5.centos.i386.rpm kmod-drbd83-8.3.8-1.el5.centos.i686.rpm

E. Copy the sample file for DRBD as a configuration file (Node1 on the finish)

# CP/USR/SHARE/DOC/DRBD83-8.3.8/DRBD.CONF/ETC

F. Configuring/etc/drbd.d/global-common.conf

global {        usage-count no;         # minor-count dialog-refresh disable-ip-verification}common {         protocol C;         handlers {                 pri-on-incon-degr  "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/ Notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f ";                  pri-lost-after-sb  "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/ Notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f ";                 local-io-error  "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/ Notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f ";                 #  fence-peer  "/usr/lib/drbd/crm-fence-peer.sh";                 # split-brain  "/usr/lib/drbd/notify-split-brain.sh root";                 #  out-of-sync  "/usr/lib/drbd/notify-out-of-sync.sh root";                 # before-resync-target  "/usr/lib/drbd/ snapshot-resync-target-lvm.sh -p 15 -- -c 16k ";                 # after-resync-target /usr/lib/drbd/unsnapshot-resync-target-lvm.sh;         }        startup {                  #wfc-timeout 120;                  #degr-wfc-timeout 120;         }        disk {                 on-io-error  detach;                # fencing resource-only;        }         net {                 cram-hmac-alg "SHA1";                 shared-secret  "Mydrbdlab";        }         syncer {                 rate 300m;        }}

G. Define a resource/etc/drbd.d/mydrbd.res, which reads as follows:

Resource MYDRBD {device/dev/drbd0;    Meta-disk internal;  DISK/DEV/VG_ROOT/MYDRBD;      On node1.test.com {address 192.168.1.65:7789;  } on node2.test.com {address 192.168.1.66:7789; }}

H. Copy the configuration files on the Node1 to Node2, keeping the configuration files of the two nodes consistent

# scp-r/etc/drbd.* node2:/etc

I. Initialize the resource and start the service on two nodes separately

# drbdadm CREATE-MD web# service DRBD start

J. Viewing the Startup status

# Drbd-overview 0:MYDRBD Connected secondary/secondary inconsistent/inconsistent C r----

At this point, two nodes are still in an inconsistent state, so we need to manually configure a node as the primary node, where the following command is configured on Node1:

# Drbdadm----Overwrite-data-of-peer primary MYDRBD

Check the status again:

# Drbd-overview 0:MYDRBD syncsource primary/secondary uptodate/inconsistent C r----[============> ...] sync ' ed:86.2% (17240/1024300) K delay_probe:25

Once the data has been synchronized, check again:

# Drbd-overview

0:MYDRBD Connected primary/secondary uptodate/uptodate C r----

K. Create a file system on the master node, only the master node can mount and read and write to the file system, from which the node cannot mount

# mkfs.ext3/dev/drbd0# mkdir/mydata# Mount/dev/drbd0/mydata

Test Copy the/etc/issue file to/mydata

# Cp/etc/issue/mydata

Unmount the file system on Node1, convert the DRBD role to secondary, convert Node2 to primary, test to see if the issue file exists

Node1:

# umount/mydata# Drbdadm Secondary MYDRBD

Node2:

# Drbdadm Primary mydrbd# mkdir/mydata# Mount/dev/drbd0/mydata

Ls/mydata see if the previously copied issue file exists, and if so, the DRBD configuration is complete.

2. Install the configuration on Node1 and Node2 mysql-5.5.42

Getting ready: Create MySQL user, MySQL group, create data directory

# groupadd-g 3306 mysql# useradd-g 3306-u 3306-s/sbin/nologin-m mysql# mkdir/mydata/data

(1) Download mysql-5.5.42 to local and unzip to/usr/local/

# Tar XF mysql-5.5.28-linux2.6-i686.tar.gz-c/usr/local

(2) Enter/usr/local/to create a soft connection

# cd/usr/local# LN-SV mysql-5.5.42-linux2.6-i686 mysql# cd MySQL

(3) Modify the permissions of the MySQL directory

# chown-r Root:mysql./*

(4) Initializing MySQL database

# scripts/mysql_install_db--user=mysql Datadir=/mydata/data

(5) Copy and modify MySQL configuration file and MySQL startup script

# CP support-files/my-large.cnf/etc/my.cnf# CP Support-files/mysql.server/etc/init.d/mysqld

# VIM/ETC/MY.CNF Add the following content DataDir =/mydata/data/innodb_file_per_table = 1

(6) test start MySQL, and create a test database TestDB (start DRBD before testing, test on two nodes separately)

# service mysqld start# mysql mysql> create database testdb; Mysql>\q

(7) test complete, close the MySQL service and make it boot without booting

# service Mysqld stop# chkconfig mysqld off# chkconfig mysqld--list

Make sure that the two-node MySQL service is configured to complete.

3. Install configuration Corosync, configure the MySQL high-availability service based on DRBD

(1) Download install Corosync and pacemaker, (32-bit) http://clusterlabs.org/rpm/epel-5/i386/

Required installation package: Cluster-glue,cluster-glue-libs,heartbeat, Resource-agents,corosync,heartbeat-libs, pacemaker, Corosynclib, LIBESMTP, Pacemaker-libs (The following configuration is done on Node1)

# yum-y--nogpgcheck localinstall/root/cluster/*.rpm

(2) Copy the template configuration file to/etc/corosync/and make the corresponding changes

# cd/etc/corosync# CP corosync.conf.example corosync.conf

# vim corosync.conf #修改如下 secauth:on bindnetaddr:192.168.1.0# Add the following service {ver:0 name:pacemaker}aisexec {User: Root Group:root}

(3) key file to generate communication between nodes

# Corosync-keygen

(4) Copy corosync.conf and Authkey to Node2

# scp-p corosync.conf Authkey node2:/etc/corosync/

(5) The directory where the Corosync log is created

# mkdir/var/log/cluster# ssh node2 ' mkdir/var/log/cluster '

(6) Try to start the Corosync on Node1

#/etc/init.d/corosync Start

To see if the Corosync engine starts properly:

# grep -e  "Corosync cluster engine"  -e  "Configuration file"  /var/ log/cluster/corosync.logjul 28 12:17:28 corosync [main  ] corosync  cluster engine  (' 1.2.7 '):  started and ready to provide service. jul 28 12:17:28 corosync [main  ] successfully read main  configuration file  '/etc/corosync/corosync.conf '. jul 28 22:32:06 corosync [main  ] corosync cluster engine  exiting with status 0 at main.c:170.jul 28 22:32:46 corosync [ main  ] corosync cluster engine  (' 1.2.7 '):  started and ready  to provide service. jul 28 22:32:46 corosync [main  ] successfully read main  configuration file  '/etc/corosync/corosync. conf '. 

To view the initialization of a member node notification if it is issued normally:

# grep Totem/var/log/cluster/corosync.logjul 12:17:28 corosync [TOTEM] Initializing Transport (UDP/IP). Jul 12:17:28 Corosync [TOTEM] Initializing transmit/receive security:libtomcrypt sober128/sha1hmac (mode 0). Jul 12:17:29 Corosync [TOTEM] The network interface [192.168.1.65] is now up. Jul 12:17:30 Corosync [TOTEM] Process pause detected for 913 MS, flushing membership messages. Jul 12:17:30 Corosync [TOTEM] A processor joined or left the membership and A new membership was formed.

Check the startup process for any errors that occur:

# grep ERROR:/var/log/cluster/corosync.log | Grep-v unpack_resources

To see if the pacemaker starts normally:

# grep Pcmk_startup/var/log/cluster/corosync.logjul 12:17:30 corosync [PCMK] Info:pcmk_startup:CRM:InitializedJul 12:17:30 Corosync [PCMK] logging:initialized pcmk_startupjul 12:17:30 corosync [PCMK] Info:pcmk_startup:Maxi Mum core file size Is:4294967295jul 12:17:30 corosync [PCMK] Info:pcmk_startup:service:9jul 12:17:30 corosync [PCMK] info:pcmk_startup:Local hostname:node1.test.comJul 22:32:49 corosync [PCMK] Info:pcmk_startup:CRM:Init Ialized

If the Corosync and pacemaker on the node1 are no problem, then you can start the corresponding service on the Node2

# ssh Node2 '/etc/init.d/corosync start '

(7) View Corosync status

# crm_mon============last Updated:fri Jul 22:52:48 2015stack:openaiscurrent dc:node1.test.com-partition with Quoru MVERSION:1.0.12-UNKNOWN2 Nodes configured, 2 expected Votes0 Resources configured.============online: [node1.hpf.com No De2.hpf.com]

(8) Disable Stonith, there is no stonith device, so you need to disable

# CRM Configure Property Stonith-enabled=false

(9) Configure default policies that do not have a legal number of votes

# CRM Configure Property No-quorum-policy=ignore

(10) Defining resources

A. Defining DRBD Master-Slave resources

# CRMCRM (Live) # CONFIGURECRM (live) configure# primitive MYSQLDRBD OCF:HEARTBEAT:DRBD params drbd_resource=mydrbd op Start timeout=240 op Stop timeout=100 op monitor interval=20 role=master timeout=30 op monitor interval=30 role=slave time Out=30crm (Live) configure# Ms MS_MYSQLDRBD MYSQLDRBD Meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 Notify=truecrm (Live) configure# Verifycrm (live) configure# commit

B. Defining resources to automatically mount file systems

CRM (Live) # CONFIGURECRM (live) configure# primitive mystore ocf:heartbeat:Filesystem params device=/dev/drbd0 directory =/mydata fstype=ext3 op start timeout=60 op stop timeout=60

C. Defining resource constraints, location constraints, and order constraints

CRM (Live) configure#

CRM (Live) configure#

D. Defining MySQL Resources

CRM (Live) configure#

E. Defining VIPs

CRM (Live) configure# primitive Myip ocf:heartbeat:IPaddr params ip=192.168.1.60 nic=eth0 CIDR_NETMASK=255.255.255.0CRM (live) configure# colocation myip_with_ms_mysqldrbd inf:ms_mysqldrbd:Master myipvip and DRBD master node together, here the boot order does not matter CRM (live) configure# Verifycrm (live) configure# commit

At this point, the MySQL high-availability cluster based on DRBD and Corosync is complete, and the performance of the master-slave test service can be switched.

# CRM node standby# CRM node online

High-availability MySQL Cluster service with DRBD combined with Corosync