Build a high-availability Web Cluster Based on Corosync + Pacemaker + NFS

1. experiment environment:

Node1: 192.168.1.17RHEL5.8 _ 32bit, web server)

Node2: 192.168.1.18RHEL5.8 _ 32bit, web server)

NFS: 192.168.1.19RHEL5.8 _ 32bit, nfs server)

VIP: 192.168.1.20webip)

2. Preparations

<1> Configure the Host Name

The node name is parsed using/etc/hosts. The node name must be consistent with the execution result of the uname-n command.

Node1:

# hostname node1.ikki.com# vim /etc/sysconfig/networkHOSTNAME=node1.ikki.com

Node2:

# hostname node1.ikki.com# vim /etc/sysconfig/networkHOSTNAME=node2.ikki.com

<2> Configure nodes for ssh key-based communication

Node1:

# ssh-keygen -t rsa# ssh-copy-id -i ~/.ssh/id_rsa.pub root@node2

Node2:

# ssh-keygen -t rsa# ssh-copy-id -i ~/.ssh/id_rsa.pub root@node1

<3> Configure Communication Between Nodes Based on host names

Node1 & Node2:

# vim /etc/hosts192.168.1.17   node1.ikki.com node1192.168.1.18   node2.ikki.com node2

<4> Configure Time Synchronization for each node

Node1 & Node2:

# crontab -e*/5 * * * *     /sbin/ntpdate 202.120.2.101 &> /dev/null

3. Install corosync and pacemaker nodes)

<1> dependent rpm package:

Libibverbs, librdmacm, lm_sensors, libtool-ltdl, openhpi-libs, openib, perl-TimeDate, libnes

<2> download the software package to a local directory such as/root/cluster ):

# cd /root/cluster# lscluster-glue-1.0.6-1.6.el5.i386.rpmcluster-glue-libs-1.0.6-1.6.el5.i386.rpmcorosync-1.2.7-1.1.el5.i386.rpmcorosynclib-1.2.7-1.1.el5.i386.rpmheartbeat-3.0.3-2.3.el5.i386.rpmheartbeat-libs-3.0.3-2.3.el5.i386.rpmlibesmtp-1.0.4-5.el5.i386.rpmpacemaker-1.1.5-1.1.el5.i386.rpmpacemaker-libs-1.1.5-1.1.el5.i386.rpmresource-agents-1.0.4-1.1.el5.i386.rpm

<3> install the local software package and dependency package:

# cd /root/cluster# yum -y --nogpgcheck localinstall *.rpm

4. Configure corosync

Node1:

# Cd/etc/corosync # cp corosync. conf. example corosync. conf # vim corosync. conf # Add the following content: service {ver: 0 name: pacemaker # use_mgmtd: yes} aisexec {user: root group: root} # vim corosync. conf # modify the following content: bindnetaddr: 192.168.1.0 # network address secauth: on # enable to_syslog: no # disable system log records and use separate logfile records) threads: 2 # set the number of threads

Generate the authentication key file used for inter-node communication:

# corosync-keygen

Copy corosync. conf and authkey to Node2:

# scp -p corosync.conf authkey  node2:/etc/corosync/

The directory where the logs generated by corosync are created for the two nodes respectively:

# mkdir /var/log/cluster# ssh node2  'mkdir /var/log/cluster'

5. Start and check the service

Node1:

# /etc/init.d/corosync start

Check whether the corosync engine is properly started:

# grep -e "Corosync Cluster Engine" -e "configuration file" /var/log/cluster/corosync.logSep 16 18:59:29 corosync [MAIN  ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.Sep 16 18:59:29 corosync [MAIN  ] Successfully read main configuration file '/etc/corosync/corosync.conf'.Sep 16 19:28:26 corosync [MAIN  ] Corosync Cluster Engine exiting with status 0 at main.c:170.Sep 16 19:54:14 corosync [MAIN  ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.Sep 16 19:54:14 corosync [MAIN  ] Successfully read main configuration file '/etc/corosync/corosync.conf'.

Check whether the initialization member node notification is normal:

# grep  TOTEM  /var/log/cluster/corosync.logSep 16 18:59:29 corosync [TOTEM ] Initializing transport (UDP/IP).Sep 16 18:59:29 corosync [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).Sep 16 18:59:29 corosync [TOTEM ] The network interface [192.168.1.17] is now up.Sep 16 18:59:29 corosync [TOTEM ] A processor joined or left the membership and a new membership was formed.

Check whether errors occur during startup:

# grep ERROR: /var/log/cluster/corosync.log | grep -v unpack_resources

Check whether pacemaker is started properly:

# grep pcmk_startup /var/log/cluster/corosync.logSep 16 18:59:29 corosync [pcmk  ] info: pcmk_startup: CRM: InitializedSep 16 18:59:29 corosync [pcmk  ] Logging: Initialized pcmk_startupSep 16 18:59:29 corosync [pcmk  ] info: pcmk_startup: Maximum core file size is: 4294967295Sep 16 18:59:29 corosync [pcmk  ] info: pcmk_startup: Service: 9Sep 16 18:59:29 corosync [pcmk  ] info: pcmk_startup: Local hostname: node1.ikki.com

If the above check is normal, you can start corosync on Node2 to start Node2 remotely on node1. do not directly start Node2)

# ssh node2 -- /etc/init.d/corosync start

View the startup status of the cluster node:

# crm status============Last updated: Tue Sep 17 23:39:11 2013Stack: openaisCurrent DC: node1.ikki.com - partition with quorumVersion: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f2 Nodes configured, 2 expected votes0 Resources configured.============Online: [ node1.ikki.com node2.ikki.com ]

View the related processes started by corosync:

# ps auxfroot     13200  0.6  0.7  86880  3952 ?        Ssl  12:29   4:06 corosyncroot     13208  0.0  0.4  11724  2104 ?        S    12:29   0:00  \_ /usr/lib/heartbeat/stonithd101      13209  0.0  0.7  12872  3820 ?        S    12:29   0:01  \_ /usr/lib/heartbeat/cibroot     13210  0.0  0.4   6572  2156 ?        S    12:29   0:00  \_ /usr/lib/heartbeat/lrmd101      13211  0.0  0.3  12060  2040 ?        S    12:29   0:00  \_ /usr/lib/heartbeat/attrd101      13212  0.0  0.5   8836  2900 ?        S    12:29   0:00  \_ /usr/lib/heartbeat/pengine101      13213  0.0  0.6  12280  3112 ?        S    12:29   0:02  \_ /usr/lib/heartbeat/crmd

6. Configure the cluster to disable the stonith device.

Corosync enables stonith by default, but the current experiment environment does not have the corresponding stonith device, so stonith needs to be disabled:

# crm configure property stonith-enabled=false

View the current configuration information:

# crm configure shownode node1.ikki.comnode node2.ikki.comproperty $id="cib-bootstrap-options" \        dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \        cluster-infrastructure="openais" \        expected-quorum-votes="2" \        stonith-enabled="false" \

7. Add an IP address to the cluster. Resource webip ):

Node1:

# crm configure primitive webip ocf:heartbeat:IPaddr params ip=192.168.1.20

View the resource startup status:

# crm status ============Last updated: Tue Sep 17 23:48:10 2013Stack: openaisCurrent DC: node1.ikki.com - partition with quorumVersion: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f2 Nodes configured, 2 expected votes1 Resources configured.============Online: [ node1.ikki.com node2.ikki.com ] webip  (ocf::heartbeat:IPaddr):        Started node1.ikki.com

Check whether the webip is effective:

# ifconfigeth0:0    Link encap:Ethernet  HWaddr 08:00:27:F1:60:13          inet addr:192.168.1.20  Bcast:192.168.1.255  Mask:255.255.255.0          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

8. Disable the quorum for the cluster configuration.

Node2:

Stop the corosync service on Node1:

# ssh node1 -- /etc/init.d/corosync stop

View the cluster working status:

# crm status============Last updated: Tue Sep 17 23:49:41 2013Stack: openaisCurrent DC: node2.ikki.com - partition WITHOUT quorumVersion: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f2 Nodes configured, 2 expected votes1 Resources configured.============Online: [ node2.ikki.com ]OFFLINE: [ node1.ikki.com ]

In the dual-node cluster environment, the legal votes cannot take effect. When Node1 is offline, webip resources cannot be transferred to node2. therefore, you need to disable quorum:

# crm configure property no-quorum-policy=ignore

View the cluster status again:

# crm status============Last updated: Tue Sep 17 23:51:27 2013Stack: openaisCurrent DC: node2.ikki.com - partition with quorumVersion: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f2 Nodes configured, 2 expected votes1 Resources configured.============Online: [ node1.ikki.com node2.ikki.com ] webip  (ocf::heartbeat:IPaddr):        Started node2.ikki.com

Start the corosync service on Node1:

# ssh node1 -- /etc/init.d/corosync start

Specify the default viscosity value for the resource:

# crm configure rsc_defaults resource-stickiness=100

9. configure a high-availability Web cluster with the active/passive model

<1> install httpd on each node and provide the test page

<2> Add a web service resource (httpd) to the cluster ):

# crm configure primitive httpd lsb:httpd

View the resource activation status:

# crm status============Last updated: Tue Sep 17 23:54:36 2013Stack: openaisCurrent DC: node2.ikki.com - partition with quorumVersion: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f2 Nodes configured, 2 expected votes2 Resources configured.============Online: [ node1.ikki.com node2.ikki.com ] webip  (ocf::heartbeat:IPaddr):        Started node1.ikki.com httpd  (lsb:httpd):    Started node2.ikki.com

<3> Configure resource constraints:

# crm configure colocation httpd-with-ip INFINITY: httpd webip

<4> Configure the resource sequence: webip and httpd ):

# crm configure order httpd-after-ip mandatory: webip httpd

<5> Configure cluster location constraints:

# crm configure location prefer-node1 httpd rule 200: #uname eq node1.ikki.com

10. Build an NFS server

NFS:

# mkdir -p /web/htdocs# vim /etc/exports/web/htdocs     192.168.1.0/24(ro)# exportfs -rav

11. Add webstore resources provided by nfs for the cluster and configure Constraints

Node1:

<1> Add a webstore Resource

# crm configure primitive webstore ocf:heartbeat:Filesystem params device=192.168.1.19:/web/htdocs directory=/var/www/html fstype=nfs op start timeout=60 op stop timeout=60

<2> set location Constraints

# crm configure colocation httpd_with_webstore inf: httpd webstore

<3> set order constraints

# crm configure order webstore_before_httpd mandatory: webstore httpd

<4> set order constraints using crm interactive commands)

# Crm (live) configure # edit Delete the previously defined constraint order httpd_after_ip inf: webip httpd # crm (live) configure # order webstore_after_ip inf: webip webstore # crm (live) configure # verify # crm (live) configure # commit

12. view the cluster configuration overview and Resource Status

<1> View cluster configurations

# crm configure shownode node1.ikki.com \        attributes standby="off"node node2.ikki.comprimitive httpd lsb:httpd \        meta target-role="Started"primitive webip ocf:heartbeat:IPaddr \        params ip="192.168.1.20" \        meta target-role="Started"primitive webstore ocf:heartbeat:Filesystem \        params device="192.168.1.19:/web/htdocs" directory="/var/www/html" fstype="nfs" \        op start interval="0" timeout="60" \        op stop interval="0" timeout="60" \        meta target-role="Started"location perfer_node1 httpd \        rule $id="perfer_node1-rule" 200: #uname eq node1.ikki.comcolocation httpd_with_webip inf: httpd webipcolocation httpd_with_webstore inf: httpd webstoreorder webstore_after_ip inf: webip webstoreorder webstore_before_httpd inf: webstore httpdproperty $id="cib-bootstrap-options" \        dc-version="1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f" \        cluster-infrastructure="openais" \        expected-quorum-votes="2" \        stonith-enabled="false" \        no-quorum-policy="ignore" \        last-lrm-refresh="1379355508"

<2> View Resource Status

# crm status============Last updated: Tue Sep 17 23:58:35 2013Stack: openaisCurrent DC: node2.ikki.com - partition with quorumVersion: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f2 Nodes configured, 2 expected votes3 Resources configured.============Online: [ node1.ikki.com node2.ikki.com ] webip  (ocf::heartbeat:IPaddr):        Started node1.ikki.com httpd  (lsb:httpd):    Started node1.ikki.com webstore       (ocf::heartbeat:Filesystem):    Started node1.ikki.com

This article from the "Don't dead birds a Hui" blog, please be sure to keep this source http://phenixikki.blog.51cto.com/7572938/1304999