Heartbeat implementing a Web server for high availability

First, Heartbeat overview:

　　How Heartbeat works:The core of heartbeat consists of two parts, the heartbeat monitoring part and the resource takeover part, heartbeat monitoring can be carried out through the network link and the serial port, and support redundant links, they send each other a newspaper Greek tell each other their current state, If the message is not received by the other party within the specified time, then it is considered to be invalid, then the resource takeover module should be started to take over the resources or services running on the other host.

The heartbeat-3.x version is then divided into 4 modules, which can be installed from the official website:
Http://www.linux-ha.org/wiki/Downloads Download gets:
The current versions are:
Clusterlabs-resource-agents-v3.9.2-0-ge261943.tar.gz #集群实验资源代理
HEARTBEAT-3-0-7E3A82377FA8.TAR.BZ2 # Heartbeat Main Package
PACEMAKER-1.1.9-1512.EL6.SRC.RPM # Pacemaker
REUSABLE-CLUSTER-COMPONENTS-GLUE--GLUE-1.0.9.TAR.BZ2 #可重复使用的群集组件

Port number: 694[[email protected] ~]# cat/etc/services | grep 694ha-cluster 694/tcp # Heartbeat ha-clusterha-cluster 694/udp # Heartbeat Ha-cluster

Extensions: who manages port definitions for TCP/UDP public services
IANA refers to the organization (Internet Assigned Numbers Authority), the Internet number assignment. Responsible for IP address assignment planning and port definition for TCP/UDP public services.
All IANA tasks can be broadly divided into three types:
First, the domain name. The IANA manages DNS domain name roots and. int,.arpa domain names, as well as IDN (Internationalized Domain name) resources.
Second, digital resources. The IANA coordinates the global IP and as (autonomous system) numbers and provides them to the regional Internet registries.
Note: As autonomous system number, is the number in the BGP routing protocol.
Third, the agreement allocation. The IANA manages the protocol numbering system together with the standardization organizations.
Official website: http://www.iana.org/

Second, actual combat: using Heartbeat to achieve a Web server high-availability 1.Heartbeat topology map

XUEGOD63 main web
Xuegod64 from the Web
Xuegod62 NFS

2. Preparatory work: XUEGOD63,XUEGOD64

1. Modify the hostname, and permanently enter # VIM/ETC/SYSCONFIG/NETWORKHOSTNAME=XUEGOD63.CN2. Parsing  # vim/etc/hosts192.168.1.63   xuegod63.cn192.168.1.64   

3. Configuring XUEGOD62 for NFS server, providing storage resources

[Email protected] ~]# yum-y install nfs-utils [[email protected] ~]# Mkdir/wwwdir[[email protected] ~]# echo ' Heartbeat Web Test ' >/wwwdir/index.html[[email protected] ~]# cat! $cat/wwwdir/index.htmlheartbeat Web Test[[email protected] ~  ]# Vim/etc/exports[[email protected] ~]# cat! $cat/etc/exports/wwwdir192.168.1.0/24 (rw) [[email protected] ~]# chmod 777 -r/wwwdir/[[email protected] ~]# ll-d/wwwdir/drwxrwxrwx 2 root root Jul  3 15:19/wwwdir/[[email protected] ~]# Systemctl start Nfs[[email protected] ~]# systemctl enable nfscreated symlink from/etc/systemd/system/multi-user.target . Wants/nfs-server.service To/usr/lib/systemd/system/nfs-server.service. [Email protected] ~]# showmount-eexport list for Xuegod70.cn:/wwwdir 192.168.1.0/24

4.XUEGOD63,64 Test NFS Storage mount and install httpd Web server (basic operation is consistent)

[[email protected] ~]# yum install nfs-utils httpd-y[[email protected] ~]# systemctl start nfs[[email  Protected] ~]# showmount-e 192.168.1.62Export list for 192.168.1.62:/wwwdir 192.168.1.0/24[[email protected] ~]# Mount-t NFS 192.168.1.62mount: 192.168.1.62[[email protected not found in/etc/fstab] ~]# mount-t NFS 192.168.1.62:/wwwdi   R/var/www/html/[[email protected] ~]# df-h file system capacity used available% mount point/dev/sda3 19G 1.9G                  17G 11%/devtmpfs 479M 0 479M 0%/devtmpfs 489M 0 489M 0%/dev/shmtmpfs               489M 6.7M 482M 2%/runtmpfs 489M 0 489M 0%/sys/fs/cgroup/dev/sr0    4.3G 4.3G 0 10%/mnt/dev/sda1 497M 113M 385M 23%/boottmpfs 98M 0 98M 0%/run/user/0192.168.1.62:/wwwdir 19G 1.5G 18G 8%/var/www/html[[email protected] ~]# vim/etc/httpd/conf/ httpd.conf [[Email protectED] ~]# systemctl Restart httpd[[email protected] ~]# systemctl enable httpdcreated symlink From/etc/systemd/system /multi-user.target.wants/httpd.service To/usr/lib/systemd/system/httpd.service. [[email protected] ~]# elinks--dump 192.168.1.62 #测试挂载情况 Heartbeat Web test# Unload resources: These resources are loaded directly by Heartbeat [[EMAIL&NB Sp;protected] ~]# umount/var/www/html/[[email protected] ~]# systemctl stop httpd[[email protected] ~]# Systemctl Disable httpd

5.XUEGOD63 Installation Heartbeat

Heartbeat compiling steps under Ceontos7

Download: http://www.linux-ha.org/wiki/Downloads, download the latest heartbeat version from the official:
Heartbeat 3.0.6
Cluster Glue 1.0.12
Resource Agents 3.9.6

1. Configure the underlying environment (configure the network yum source) Install the dependency package #yum install-y bzip2 bzip2-devel gcc gcc-c++ autoconf automake libtool e2fsprogs-devel Glib2-dev El libxml2 libxml2-devel libtool-ltdl-devel asciidoc libuuid-devel docbook If Ltdl package installation is unsuccessful, download RPM package installation: #rpm-IVH Libtool-ltdl-devel-2.4.2-22.el7_3.x86_64.rpm2. Compiling Cluster glue# tar jxf reusable-cluster-components-glue.tar.bz2# CD reusable-cluster-components-glue--0a7add1d9996/# groupadd haclient Create user group # useradd-g haclient hacluster Create user #./autogen . SH #./configure--prefix=/usr/local/heartbeat/# make && make install3. Compiling resource agents# tar-zxf Resource-agen ts-3.9.6.tar.gz# CD resource-agents-3.9.6#/autogen.sh#./configure--prefix=/usr/local/heartbeat-- With-daemon-user=hacluster--with-daemon-group=haclient--enable-fatal-warnings=no LIBS= '/lib64/libuuid.so.1 ' # make# make install4. Compiling heartbeat# tar-jxvf heartbeat-3.0.6.tar.bz2# CD heartbeat-3-0-958e11be8686/#./bootstrap# Export cflags= "$CFLAGS-i/usr/local/heartbeat/include-l/usr/local/heartbeat/lib" #./confiGure--prefix=/usr/local/heartbeat--with-daemon-user=hacluster--with-daemon-group=haclient-- Enable-fatal-warnings=no libs= '/lib64/libuuid.so.1 ' # make # make INSTALL5. Build configuration file: # cd/usr/local/heartbeat/etc/ha.d/# Cp/root/heartbeat-3-0-958e11be8686/doc/{ha.cf,haresources,authkeys}. # chkconfig--add heartbeat# chkconfig Heartbeat on# chmod 600/usr/local/heartbeat/etc/ha.d/authkeys# mkdir-pv/usr/local/heartbeat/usr/lib/ocf/lib/ heartbeat/# cp/usr/lib/ocf/lib/heartbeat/ocf-*/usr/local/heartbeat/usr/lib/ocf/lib/heartbeat/## ln-sv/usr/local/ heartbeat/lib64/heartbeat/plugins/*/usr/local/heartbeat/lib/heartbeat/plugins/

6, Heartbeat configuration heartbeat configuration is mainly related to HA.CF, haresources, Authkeys these three files. HA.CF: Master profile, Haresource: Used to configure the service for heartbeat hosting, Authkey: is used to specify the authentication method for heartbeat.(1) Configuring the HA.CF----master configuration file# VIM/USR/LOCAL/HEARTBEAT/ETC/HA.D/HA.CF (Note: Front number is line number) Debugfile/var/log/ha-debug # #用于记录heartbeat的调试信息29 Logfi Le/var/log/ha-log # #用于记录heartbeat的日志信息34 Logfacilitylocal0 # #设置heartbeat的日志, here's the system log 48 KeepAlive 2 # #设定心跳 (monitoring) time is 2 seconds Deadtime # #指定若备用节点在30秒内未收到主节点心跳信号, take over the primary server resources warn         Time # #指定心跳延迟的时间为10秒, in 10 seconds the node cannot receive the heartbeat of the primary node, that is, the log is written to warning logs, but does not switch service Initdead 60 # #系统启动或重启后预留的忽略时间段, at least twice times the value of Deadtime udpport 694 # UDP port used by #广播/unicast communication #bcast ENS32 # Linu X # #使用网卡eno32发送心跳检测113 #mcast eth0 225.0.0.1 694 1 0 # #采用网卡eth0的Udp多播来组织心跳, typically in alternate nodes bcast, Ucast and Mcast represent broadcast, single                 Broadcast and multicast, is the way to organize the heartbeat, choose its 1121 ucast ens32 192.168.1.64 # #采用网卡eth32的udp单播来组织心跳, followed by the IP address of the two-machine IP address 157 auto_failback on # #定义当主节点恢复后 If the service is automatically cut back to 211 node xuegod63.cn # #主节点名称212 node xuegod64.cn # #备用节点名   Called the 192.168.1.1 Ping             # #通过ping网关检测心跳是否正常, used only to test the network 253 respawn Hacluster/usr/local/heartbeat/libexec/heartbeat/ipfail # #指定和heartbea     T start together, close the process 259 Apiauth ipfail gid=haclient uid=hacluster # #设置启动IPfail的用户和组 (2)  Configuring haresources-----resource Files The haresources file is used to specify the master node, cluster IP, subnet mask, broadcast address, and startup service cluster resources for a dual-machine system, each line of the file can contain one or more resource script names, separated by spaces between resources,
Using two colons between the parameters, the master node and the resource file haresources in the backup node are exactly the same as the general format: Node-name network <resource-group>node-name represents the host name of the master node, Must match the node name specified in the ha.cf file. The network is used to set the IP address, subnet mask, and device identity of the cluster. The resource-group is used to specify services that need to be heartbeat hosted (that is, these services can be started and shut down by Heartbeat). Note: The IP address specified here is the IP address of the cluster external service if you want to host these services, you must write the service into a script that can be started or closed via start/stop and placed in the/etc/init.d/or/etc/ha.d/resource.d/directory. Heartbeat will either automatically go to the/ETC/INIT.D or/ETC/HA.D/RESOURCE.D directory under the script name to find the appropriate script to start or close the operation. Example: # vim/usr/local/heartbeat/etc/ha.d/haresources (Note: The preceding number is the line number) xuegod63.cn Ipaddr::192.168.2.111/24/ens32 Filesystem::192.168.2.62:/wwwdir::/var/www/html::nfs httpdxuegod71.cn ipaddr::192.168.10.111/24/ens33 Filesystem: : 192.168.10.70:/wwwdir::/var/www/html::nfs httpd Note: xuegod63.cn is the host name of the primary server and does not need to be modified on XUEGOD64. This will add a resource to this host by default. When the xuegod63 is broken, XUEGOD64 will take over again. Ipaddr::192.168.1.111/24/eth32 #指定VIP及绑定到哪个网卡上Filesystem:: 192.168.1.62:/wwwdir::/var/www/html::nfs # Specifies the storage httpd #指定要启动的服务 that you want to mount.     This service must be under/ETC/INIT.D or/USR/LOCAL/HEARTBEAT/ETC/HA.D/RESOURCE.D directory(3) Configuring Authkeys-----Heartbeat im Key Verification file#vim/usr/local/heartbeat/etc/ha.d/authkeys auth MD5 hello! Note: The auth after filling the serial number, can be filled in, but the second line must be the ordinal name, and then for authentication method, support three kinds (CRC MD5 SHA1 ) mode validation, and, most of all, the custom key. What kind of verification should I choose? If the heartbeat is running on top of a secure network, such as the crossover line in this example, you can use the CRC, which is the least expensive approach from a resource perspective. If the network is not secure, but you also want to reduce CPU usage, use MD5.
Finally, if you want to get the best certification, regardless of CPU usage, use SHA1, which is the hardest to crack among the three.(4) Writing httpd startup scripts# vim/usr/local/heartbeat/etc/ha.d/resource.d/httpd #!/bin/bash/bin/systemctl $ httpd # chmod 755!$⑸ copy config file to standby # scp-rp/usr/local/heartbeat/etc/ha.d/* [Email protected]:/usr/local/heartbeat/etc/ha.d/

Third, test 1. Manually load VIP 192.168.1.111 to Ens32

[Email protected] ~]# Cd/usr/local/heartbeat/etc/ha.d/resource.d[[email protected] resource.d]#./ipaddr 192.168.1.111/24/ens32 startipaddr[7116]: INFO:  successinfo:  Success view Vip:[[email protected] ~]# ifconfig  The IP of the #可以看到 ens32:1 is: 192.168.1.111

2. Manually load NFS storage resources to/var/www/html (load requires fuser support, yum install psmisc-y)

[[email protected] resource.d]#./filesystem 192.168.1.62:/wwwdir/var/www/html/nfs start filesystem[23567]: INFO:  Successinfo:  Success [[email protected] ~]# Ls/var/www/html/index.html/var/www/html/index.html[[email protected] ~]# DF-  h ... 192.168.2.62:/wwwdir    18G  1.1G   17G    6%/var/www/html manual start httpd Service [[email protected] ~]# Systemctl Start HTTPD test opens the main service page http://192.168.1.63

Configuring Heartbeat on 3.xuegod64

Modify file permissions: [[email protected] ~]# chmod 600/usr/local/heartbeat/etc/ha.d/authkeys  #此文件必须是600权限 Otherwise heartbeat start does not successfully modify the unicast address: [[email protected] ~]# vim/usr/local/heartbeat/etc/ha.d/ha.cf change: Ucast ens32 192.168.1.64: Ucast ens32 192.168.1.63 mount requires fuser support, yum install Psmisc-y

4. Two Machine start heartbeat service

[[email protected] ~]#/etc/init.d/heartbeat restart[[email protected] ~]#/etc/init.d/heartbeat Restart view port number: [[email] Protected] ~]# Netstat-antup | grep 694udp        0      0 0.0.0.0:694        0.0.0.0:*           6895/HEARTBEAT:WR View port number: [[email protected] ~]# Netstat-antup | grep 694udp        0      0 0.0.0.0:694        0.0.0.0:*           6895/heartbeat:wri

5. View the cluster

View cluster resources in XUEGOD63: [[email protected] ~]# ifconfig  #查看VIP加载ens32:1: Flags=4163<up,broadcast,running,multicast >  MTU        inet 192.168.1.111  netmask 255.255.255.0  broadcast 192.168.1.255 ether 00:0c:29        : F0:FC:2C  Txqueuelen  (Ethernet) [[email protected] ~]# Df-h[[email protected] ~]# DF-  H #查看挂载 ... 192.168.2.62:/wwwdir    18G  1.1G   17G    6%/var/www/html[[email protected] ~]# netstat-anput | grep httpd   #查看httpdxuegod64上查看, no task resources: [[email protected] ~]# ifconfig[[email protected] ~]# Df-h[[email protected] ~]# Netstat-anput | grep httpd passes the VIP test: When both hosts are turned on, all requests go to the xuegod63. Visit: http://192.168.1.111/Normal

6. Fault setting

When the xuegod63 is dropped, the NIC is turned off, wait 30 seconds, all requests go to XUEGOD64 [[email protected] ha.d]# ifdown ens32[[email protected] ~]#    Ifconfig ... eth0:0 Link encap:ethernet HWaddr 00:0c:29:48:80:95 inet addr:192.168.1.111 bcast:192.168.1.255 mask:255.255.255.0[ [email protected] ~]# df-h ... 192.168.1.62:/wwwdir 9.7G 3.4G 5.8G 37%/var/www/html[[email protected] ~] #systemctl status httpdhttpd (PID 6375 ) is running...xuegod63 on the ENS32 network card: [[email protected] ~]# ifup ens32 Wait 30 seconds, view: [[email protected] ~]# df-h ... 192.168.1.62:/wwwdir 9.7G 3.4G 5.8G 37%/var/www/html #已经加载了httpd资源 [[email protected] ~]# systemctl status htt PDHTTPD (PID 27097) is running ... [[email protected] ~]# ifconfig ...    eth0:0 Link encap:ethernet HWaddr 00:0c:29:12:ec:1e inet addr:192.168.1.111 bcast:192.168.1.255 mask:255.255.255.0 The source has been cut back over. View release information on XUEGOD64: [[email protected] ~]# ifconfig #查看不到ens32:0 192.168.1.111 This IP address [[email protected] ~]# DF -hfilesystem           Size used Avail use% mounted on/dev/sda2 9.7G 3.7G 5.5G 41%/tmpfs 569M 0 569M 0%/dev/shm/dev/sda1 194M 28M 157M 15%/boot/dev/sr0 3.4G 3.4G 0 100%/mnt[[email  protected] ~]# service httpd STATUSHTTPD is stopped

Extended:
Heartbeat self--ipfail tool for switching off the network
　　The principle of switching ipfail broken network

About Ipfail The principle of the switch is very simple, first heartbeat to determine whether their network is normal is actually through ping an IP, if you can ping the pass, that the network is through, if the ping does not pass, the network is broken, or the main server NIC is broken, Then perform the toggle action.

Ping a group's ipfail configuration:

Ping_group group1 172.16.103.254 172.16.103.212

Heartbeat implementing a Web server for high availability