Heartbeat+ldirectord+lvs NAT

Ldirectord to manage the LVS, you can regularly check the back end Realserver, found that after the failure of automatic removal, after the restoration automatically added LVS table, let us feel it.

LVS End:

Node1

eth0:192.168.3.124

eth0:192.168. 8. 124 for Heartbeat Heartbeat

Node2

eth0:192.168.3.126

eth0:192.168. 8. 126 for Heartbeat Heartbeat

Realserver End:

Web1

eth0:192.168.3.128

eth0:0:192.168.8.128

Web2

eth0:192.168.3.129

eth0:0:192.168.8.129

First, it is recommended to configure the LVS before you continue to configure the other parts.

Configuring LVS Nat mode on Node1 and Node2, taking Node1 as an example

Note: First on the Node1 to build a VIP to do the LVS test, test error and then delete, and then use heartbeat VIP, this can give us to reduce the error rate

LVS Terminal Node1 Configuration:

Ifconfig eth0:1 192.168.3.233 netmask 255.255.255.0

Ipvsadm-a-T 192.168.3.233:80-s WRR

Ipvsadm-a-T 192.168.3.233:80-r 192.168.8.128:80-g

Ipvsadm-a-T 192.168.3.233:80-r 192.168.8.129:80-g

Realserver End WEB1,WEB2 Configuration:

IP route add 192.168.3.0 via 192.168.3.233

Note: Point the Realserver gateway to the VIP 192.168.2.233

After the configuration is complete, test:

[Email protected] ha.d]# Curl 192.168.3.233/1.html

hello,192.168.3.128

[Email protected] ha.d]# Curl 192.168.3.233/1.html

hello,192.168.3.129

The above instructions LVS configuration success, Cancel Eth0:1 can, Ifdown eth0:1, need to do the same configuration on Node2.

Two. Configure Ldirectord

RPM-QL Heartbeat-ldirectord

Cp/usr/share/doc/ldirectord-1.0.4/ldirectord.cf/etc/ha.d

Vim/etc/ha.d/ldirectord.cf

# Global Directiveschecktimeout=20 The time interval for error #判定real server. checkinterval=10 #指定ldirectord在两次检查之间的间隔时间. FALLBACK=127.0.0.1:80 #当所有的real The address of the Web service redirection when the server node does not work. Autoreload=yes #是否自动重载配置文件, when you choose Yes, the configuration file changes to automatically load configuration information. Logfile= "/var/log/ldirectord.log" #设定ldirectord日志输出文件路径. Quiescent=no #当选择no时, if a node does not respond during the Checktimeout set time period, Ldirectord will remove the real server directly from the LVS routing table, at which point the existing client connection will be interrupted and the LV s loses all connection tracking records and persistent connection templates, if select Yes, when a real server fails, Ldirectord sets the weight of the failed node to 0, the new connection will not be reachable, but the node is not purged from the LVS routing table. The connection trace record and the program connection template remain on the director at the same time. # Sample for an HTTP virtual servicevirtual=192.168.3.233:80 #指定虚拟的IP地址和端口号, note that the line following the virtual line must be real=192.168.8.128:   MASQ #指定Real Server server address and port, while setting the LVS operating mode, using Gate to represent Dr Mode, Ipip represents TUNL mode, MASQ represents NAT mode. real=192.168.8.129:80 masq fallback=127.0.0.1:80 masq service=http #指定服务的类型, here is load balancing the HTTP service   。 request= the "ipvsadm.html" #ldirectord将根据指定的Real the server address, combining the request path given by this option, sending an access request, checking that the service on the real server is working properly, and ensuring thatThe address given in the page is accessible, otherwise Ldirectord will mistakenly think that this node has failed, error monitoring phenomenon.   receive= "Test OK" #指定请求和应答字串.   SCHEDULER=WRR #指定调度算法, here is the RR (round call) algorithm.   Protocol=tcp #指定协议的类型, LVS supports TCP and UDP protocols.   Checktype=negotiate #指定Ldirectord的检测类型, default is negotiate. Checkport=80 #指定监控的端口号. # virtualhost=www.gaojf.com #虚拟服务器的名称, casually designated.

Need to set ipvsadm.html on WEB1,WEB2

echo "Test OK" > ipvsadm.html

Node1 on the Ldirectord configuration, copy to Node2 on a copy.

Three. Configure Heartbeat

Heartbeat configuration See the previous blog post "Heartbeat to implement Nginx hot standby", in this only change/etc/ha.d/haresources resources, others do not need to move. As follows:

Nod1 End/ETC/HA.CF

Logfile/var/log/ha-loglogfacilitylocal0keepalive 2deadtime 30warntime 10initdead 120udpport694ucast eth0 192.168.8.126auto_failback onnodeusvr-124.cityre.cnnodeusvr-126.cityre.cnping 192.168.3.1respawn hacluster/usr/ Lib64/heartbeat/ipfail

Node2 End/ETC/HA.CF

Logfile/var/log/ha-loglogfacilitylocal0keepalive 2deadtime 30warntime 10initdead 120udpport694ucast eth0 192.168.8.124auto_failback onnodeusvr-124.cityre.cnnodeusvr-126.cityre.cnping 192.168.3.1respawn hacluster/usr/ Lib64/heartbeat/ipfail

Node1 and Node2 end of the/etc/ha.d/haresources

usvr-124.cityre.cn Ipaddr::192.168.3.233/24/eth0 LDIRECTORD::LDIRECTORD.CF

Note: You need to write each other's host in Node1 and Node2

192.168.3.124 usvr-124.cityre.cn

192.168.3.126 usvr-126.cityre.cn

Note: The VIP here is our LVS VIP, so as to achieve the VIP drift.

Four. Testing

Heartbeat service heartbeat Start after configuration is complete

1. See if the VIP is started

[[Email protected] ha.d]# IP A
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 16436 qdisc noqueue State UNKNOWN
Link/loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host Lo
INET6:: 1/128 Scope Host
Valid_lft Forever Preferred_lft Forever
2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc MQ State up Qlen 1000
Link/ether 00:50:56:89:a2:16 BRD FF:FF:FF:FF:FF:FF
inet 192.168.3.124/24 BRD 192.168.3.255 Scope Global eth0
inet 192.168.8.124/24 BRD 192.168.8.255 Scope Global eth0:0
Inet6 FE80::250:56FF:FE89:A216/64 Scope link
Valid_lft Forever Preferred_lft Forever
[[Email protected] ha.d]# IP A
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 16436 qdisc noqueue State UNKNOWN
Link/loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host Lo
INET6:: 1/128 Scope Host
Valid_lft Forever Preferred_lft Forever
2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc MQ State up Qlen 1000
Link/ether 00:50:56:89:a2:16 BRD FF:FF:FF:FF:FF:FF
inet 192.168.3.124/24 BRD 192.168.3.255 Scope Global eth0
inet 192.168.8.124/24 BRD 192.168.8.255 Scope Global eth0:0
inet 192.168.3.233/24 BRD 192.168.3.255 Scope Global secondary eth0
Inet6 FE80::250:56FF:FE89:A216/64 Scope link
Valid_lft Forever Preferred_lft Forever

2. View the LVS table

[Email protected] ha.d]# ipvsadm-l-N
IP Virtual Server version 1.2.1 (size=4096)
Prot Localaddress:port Scheduler Flags
Remoteaddress:port Forward Weight activeconn inactconn
TCP 192.168.3.233:80 WRR
-192.168.8.128:80 MASQ 1 0 0
-192.168.8.129:80 MASQ 1 0 0

3. Stop Nginx on the web1 side for about 20 seconds and continue to view the LVS table

[Email protected] ha.d]# ipvsadm-l-N
IP Virtual Server version 1.2.1 (size=4096)
Prot Localaddress:port Scheduler Flags
Remoteaddress:port Forward Weight activeconn inactconn
TCP 192.168.3.233:80 WRR
-192.168.8.129:80 masq 1 0 0

The discovery of the Node1 end 192.168.8.128:80 has been clear from the LVS table, indicating that Ldirectord is working.

4. Reboot the Web1-side nginx to continue viewing the LVS table

[Email protected] ha.d]# ipvsadm-l-N
IP Virtual Server version 1.2.1 (size=4096)
Prot Localaddress:port Scheduler Flags
Remoteaddress:port Forward Weight activeconn inactconn
TCP 192.168.3.233:80 WRR
-192.168.8.128:80 MASQ 1 0 0
-192.168.8.129:80 MASQ 1 0 0

Discover that Node1 is re-added to the LVS table.

5. Stop all WEB1,WEB2-side nginx and continue to view the LVS table

[Email protected] nginx1.6]# ipvsadm-l-N
IP Virtual Server version 1.2.1 (size=4096)
Prot Localaddress:port Scheduler Flags
Remoteaddress:port Forward Weight activeconn inactconn
TCP 192.168.3.233:80 WRR
- 127.0.0.1:80 Local 1 0 0

When all LVs realserver nodes are all down, there will be node1 on the 127.0.0.1:80 boot, when accessing the VIP

Curl 192.168.3.233/1.html will appear "page is being maintained" to improve the user experience.

6. If Node1 down, Node2 will take over VIP

Service Heartbeat stop on Node1

View IP on Node2

[[Email protected] ha.d]# IP A
1:lo: <LOOPBACK,UP,LOWER_UP> MTU 16436 qdisc noqueue State UNKNOWN
Link/loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host Lo
INET6:: 1/128 Scope Host
Valid_lft Forever Preferred_lft Forever
2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc MQ State up Qlen 1000
Link/ether 00:50:56:89:91:a0 BRD FF:FF:FF:FF:FF:FF
inet 192.168.3.126/24 BRD 192.168.3.255 Scope Global eth0
inet 192.168.8.126/24 BRD 192.168.8.255 Scope Global eth0:0
inet 192.168.3.233/24 BRD 192.168.3.255 Scope Global secondary eth0
Inet6 FE80::250:56FF:FE89:91A0/64 Scope link
Valid_lft Forever Preferred_lft Forever

7. View the LVS table on Node2

[Email protected] ha.d]# ipvsadm-l-N
IP Virtual Server version 1.2.1 (size=4096)
Prot Localaddress:port Scheduler Flags
Remoteaddress:port Forward Weight activeconn inactconn
TCP 192.168.3.233:80 WRR
-192.168.8.128:80 MASQ 1 0 0
-192.168.8.129:80 MASQ 1 0 0

Summarize:

1.node1 and Node2 on the LVS will not run simultaneously, when the VIP on the Node1, LVS on the Node2 is stopped, when the VIP on the Node2, the LVS on the Node1 is stopped.

2.ldirectord does not need to be paired, we can also use it separately to manage LVS, which means you do not have to add Ldirectord to the heartbeat resource file.

Heartbeat+ldirectord+lvs NAT