MFS High Availability

Use:
Solve the single point problem of MFS Master, as well as other standard configuration methods that require a highly available environment

Planning:
Using DRBD to realize the disaster capacity of the main standby machine, heartbeat do heartbeat monitoring, pacemaker realize service (Resource) switching and control, etc.

Describe:
The DRBD dual master mode has higher requirements for network and configuration, and is not used in this structure;
DRBD requires a partition to be emptied and cannot be formatted
Switching and failover of service can be realized by heartbeat alone;
Other components in the HA project are required, such as glue and resourceagent.

software:
DRBD: http://oss.linbit.com/ drbd/
DRBD 8.3.9: drbd-8.3.9.tar.gz

Heartbeat 3.0.4:  HEARTBEAT-3-0-STABLE-3.0.4.TAR.BZ2
Cluster Glue 1.0.7: glue-1.0.7.tar.bz2
Resource Agents 1.0.3:  AGENTS-1.0.3.TAR.BZ2
Pacemaker 1.0.5:  MFS Download: http://www.moosefs.org/index.php/download.html
Moosefs 1.6.20: mfs-1.6.20-2.tar.gz

Environment:
mfsmaster:192.168.1.1
mfsbackup:192.168.1.2
vip:192.168.1.10

Installation:
Set up hosts:
# vi/etc/hosts
192.168.1.1 Mfs.master
192.168.1.2 mfs.backup
Drbd:
# wgethttp://oss.linbit.com/drbd/8.3/drbd-8.3.9.tar.gz
# tar zxvf drbd-8.3.9.tar.gz
# CD drbd-8.3.9
#./configure--PREFIX=/USR/LOCAL/DRBD--with-km
# make && make install
# vi/usr/local/drbd/etc/drbd.d/global_common.conf
Syncer {
# rate after al-extents Use-rle cpu-mask verify-alg csums-alg
Rate 100M;
}
# Vi/usr/local/drbd/etc/drbd.d/mfs.res

Resource MFS {
device/dev/drbd0;
Disk/dev/lvm/mfsdata;
Meta-disk internal;
On Mfs.master {
Address 192.168.1.1:7789;
}

On mfs.backup {
address 192.168.1.2:7789;
  }
}
# cp/usr/local/drbd/etc/rc.d/init.d/drbd/etc/init.d/
# Insmod/lib/modules/2.6.18-8.el5/kernel/drivers/block/drbd.ko
# modprobe DRBD
# chkconfig--add DRBD
# chkconfig--level + DRBD on
# service DRBD start
# Drbdadm CREATE-MD All
# mkfs.ext3/dev/drbd0
# Vi/etc/fstab
/dev/drbd0/mfsmeta ext3 defaults,noatime,nodiratime 0 0
The same configuration as above master standby, check the synchronization status:
# CAT/PROC/DRBD
GIT-HASH:1C3B2F71137171C1236B497969734DA43B5BEC90 build by [email protected], 2010-12-20 19:19:37
0:cs:connected ro:primary/secondary ds:uptodate/uptodate C r-----
ns:89190240 nr:613604 dw:89803844 dr:620461 al:45275 bm:5 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Perform on the host:
# drbdsetup/dev/drbd0 Primary-o
# Mount/mfsmeta
Now that the DRBD installation is complete, you can do some testing according to the documentation.

MFS:
# wgetHttp://pro.hit.gemius.pl/hitredir/id=0sWa0S8ft4sTAHF1bGAAEZPcP3ziyq7f9SdhoQf7oeT.c7/url=moosefs.org/tl_ Files/mfscode/mfs-1.6.20-2.tar.gz
# tar zxvf mfs-1.6.20-2.tar.gz
# CD mfs-1.6.20
#./configure--prefix=/usr/local/mfs
# make
# make install
# vi/usr/local/mfs/etc/mfsmaster.cfg
Data_path =/mfsdata/metalog
# Mkdir/mfsmeta/metalog
# chown Nobody.nobody/mfsmeta/metalog
Set up temporary hosts to test MFS boot:
# vi/etc/hosts
192.168.1.1 Mfsmaster
#/usr/local/mfs/sbin/mfsmaster Start
The MFS master installation is complete, other Chunkserver installation and detailed configuration, please refer to the documentation.

HA:  
configuration:
# Export prefix=/usr
# Export Cluster_user=hacluster

# getent passwd ${cluster_user} >/dev/null | | useradd-r-G ${cluster_ GROUP}-d/var/lib/heartbeat/cores/hacluster-s/sbin/nologin-c "Cluster user" ${cluster_user}

GLUE:
# Wget-o cluster-glue.tar.bz2 http://hg.linux-ha.org/glue/archive/tip.tar.bz2
# cd reusable-cluster-components-*
Span style= "Color:rgb (0,85,255);" >#./autogen.sh &&./configure--prefix= $PREFIX  --with-daemon-user=${cluster_user}--with-daemon-group =${cluster_group}
# make
# make install

Resource Agent:
# Wget-o resource-agents.tar.bz2 http://hg.linux-ha.org/agents/archive/tip.tar.bz2
# tar JXVF resource-agents.tar.bz2
# CD cluster-resource-agents-*
#./autogen.sh &&./configure--prefix= $PREFIX
# make
# make Install

Heartbeat:
# Wget-o heartbeat.tar.bz2http://hg.linux-ha.org/dev/archive/tip.tar.bz2
# tar jxvf heartbeat.tar.bz2
# CD heartbeat-*
#./bootstrap &&./configure--prefix= $PREFIX
# make
# Make Install
# CP DOC/HA.CF $PREFIX/etc/ha.d/
# CP Doc/authkeys $PREFIX/etc/ha.d/
# chmod 0600 $PREFIX/etc/ha.d/authkeys
Configuration heartbeat:
# VI $PREFIX/ETC/HA.D/HA.CF
Debugfile/opt/logs/heartbeat/ha-debug
Logfile/opt/logs/heartbeat/ha-log
Logfacility local0
keepalive 2
Deadtime 30
Warntime 10
Initdead 120
Udpport 694
Ucast eth0 192.168.1.2
Auto_failback on
Ping 192.168.1.254
Respawn Hacluster/usr/lib64/heartbeat/ipfail
CRM on
# Service Heartbeat Start
Please refer to the documentation for specific configuration instructions.

Pacemaker:
# Wget-o pacemaker.tar.bz2http://hg.clusterlabs.org/pacemaker/stable-1.0/archive/tip.tar.bz2
# tar jxvf pacemaker.tar.bz2
# CD pacemaker-1-0-*
#./autogen.sh &&/configure--prefix= $PREFIX--with-lcrso-dir= $LCRSODIR
# make
# make install
# ldconfig-v
# CRM
CRM (Live) #Configure node Mfs.master
CRM (Live) #Configure node Mfs.backup
CRM (Live) #Configure primitive MFSMASTER_DRBD OCF:LINBIT:DRBD params drbd_resource= "MFS" drbdconf= "/usr/local/drbd/etc/ drbd.conf "Meta migration-threshold=" 10 "
CRM (Live) #Configure monitor MFSMASTER_DRBD 30s:20s
CRM (Live) #Configure primitive Mfsmaster_fs ocf:heartbeat:Filesystem params device= "/dev/drbd0" directory= "/mfsmeta" fstype= "Ext3"
CRM (Live) #Configure primitive Mfsmaster_vip OCF:HEARTBEAT:IPADDR2 params ip= "192.168.1.10" nic= "eth0:0"
CRM (Live) #Configure primitive Mfsmaster Lsb:mfsmaster
CRM (Live) #Configure monitor Mfsmaster 30s:30s
CRM (Live) #Configure group Mfsmaster_group MFSMASTER_VIP Mfsmaster_fs Mfsmaster
CRM (Live) #Configure Ms Mfsmaster_drbd_ms MFSMASTER_DRBD meta master-max= "1" master-node-max= "1" clone-max= "2" Clone-node-max = "1" notify= "true"
CRM (Live) #Configure colocation Mfsmaster_colo inf:mfsmaster_group mfsmaster_drbd_ms:master
CRM (Live) #Configure Order Mfsmaster_order Inf:mfsmaster_drbd_ms:promote Mfsmaster_group:start
CRM (Live) #Configure Property $id = "Cib-bootstrap-options" expected-quorum-votes= "2" stonith-enabled= "false" No-quorum-policy= "Ignore" start-failure-is-fatal= "false"
CRM (Live) #Configure Commit
CRM (Live) #quit

Set VIP to Mfsmaster:
# vi/etc/hosts
192.168.1.10 Mfsmaster
You can use Crm_mon to view resource status, if normal should be similar:
# Crm_mon

 ============
last Updated:sun Jan 14:01:53:
stack:heartbeat
current DC:mfs.backup (985860ea-ae2b-4490-b7e9-19f902321358)-partition with quorum
VERSION:1.0.10-B0266DD5FFA9C51377C68B1F29D6BC84367F51DD
2 Nodes Configured, 2 expected votes

============
Online: [Mfs.master Mfs.backup]

Resource Group:mfsmaster_group
Mfsmaster_vip (OCF::HEARTBEAT:IPADDR2): Started mfs.master
Mfsmaster_fs (ocf::heartbeat:filesystem): Started mfs.master
Mfsmaster (lsb:mfsmaster): Started mfs.master
Master/slave Set:mfsmaster_drbd_ms
Masters: [Mfs.master]
slaves: [Mfs.backup]

At this point, MFS's high-availability environment is built and can be tested using CRM node Stndby and CRM node online.

MFS High Availability