OpenStack Controller HA test environment build record (ii)--Configure Corosync and pacemaker

Corosync.conf please back up and edit:
# vi/etc/corosync/corosync.conf

Totem {
Version:2

token:10000
Token_retransmits_before_loss_const:10
Secauth:off
Rrp_mode:active

interface {
ringnumber:0
bindnetaddr:10.0.0.0
Broadcast:yes
mcastport:5405
Ttl:1
}
Transport:udpu
}

nodelist {
Node {
ring0_addr:10.0.0.12
}
Node {
ring0_addr:10.0.0.13
}
}

Logging {
Fileline:off
To_stderr:no
To_logfile:yes
LogFile:/var/log/cluster/corosync.log
To_syslog:yes
Debug:off
Timestamp:on
Logger_subsys {
Subsys:quorum
Debug:off
}
}

Quorum {
Provider:corosync_votequorum
Expected_votes:2
Wait_for_all:1
Last_man_standing:1
last_man_standing_window:10000
}

Each node is edited corosync.conf.
Start the Corosync service on two nodes:
# Systemctl Enable Corosync.service
# Systemctl Start Corosync.service


View the status of the Corosync service on either node:
# Corosync-cmapctl Runtime.totem.pg.mrp.srp.members
Runtime.totem.pg.mrp.srp.members.167772172.config_version (U64) = 0
Runtime.totem.pg.mrp.srp.members.167772172.ip (str) = R (0) IP (10.0.0.12)
Runtime.totem.pg.mrp.srp.members.167772172.join_count (u32) = 1
Runtime.totem.pg.mrp.srp.members.167772172.status (str) = joined
Runtime.totem.pg.mrp.srp.members.167772173.config_version (U64) = 0
Runtime.totem.pg.mrp.srp.members.167772173.ip (str) = R (0) IP (10.0.0.13)
Runtime.totem.pg.mrp.srp.members.167772173.join_count (U32) = 2
Runtime.totem.pg.mrp.srp.members.167772173.status (str) = joined
167772172 is the member ID, its IP is 10.0.0.12, the state is joined;
167772173 is the member ID, its IP is 10.0.0.13, the state is joined;
Corosync Service status is correct.


Start the Pacemaker service on two nodes:
# Systemctl Enable Pacemaker.service
# Systemctl Start Pacemaker.service


To view the service startup status:
# Crm_mon
Last Updated:tue Dec 8 03:58:37 2015
Last Change:tue Dec 8 03:58:27 2015
Stack:corosync
Current Dc:controller3 (167772173)-Partition with quorum
Version:1.1.12-a14efad
2 Nodes configured
0 Resources configured


Online: [Controller2 Controller3]
Pacemaker service is in good condition. If the state is not good, there will be a "brain crack" phenomenon. That is, in Controller2 and Controller3 respectively run Crm_mon, see the current DC is not unified, but the respective itself. One possible cause of this problem is that the firewall is turned on.


Execute the "CRM Configure" command on either node:
# CRM Configure Property no-quorum-policy= "Ignore" pe-warn-series-max= "$" pe-input-series-max= "1000" pe-error-series-max= "cluster-recheck-interval=" 5min "
The default voting rules suggest that the number of nodes in the cluster is odd and not less than 3. When the cluster has only 2 nodes, of which 1 nodes are broken, the cluster resources are not transferred because the default voting rules are not met, and the cluster is still not available as a whole. No-quorum-policy= "Ignore" solves this two-node problem, but is not used in a production environment. In other words, the production environment is still at least 3 nodes.
Pe-warn-series-max, Pe-input-series-max, Pe-error-series-max represent the log depth.
Cluster-recheck-interval is the frequency at which the node is re-examined.

Disable Stonith:
# CRM Configure Property Stonith-enabled=false
Stonith is a physical device that accepts instructions for power outages, and the test environment does not have this device, and if you do not turn off this option, the Execute CRM command always contains its error message.

To view the configuration:
# CRM Configure Show


Corosync and pacemaker status is correct, you can create VIP resources. My VIP is "10.0.0.10":
# CRM Configure primitive Myvip OCF:HEARTBEAT:IPADDR2 params ip= "10.0.0.10" cidr_netmask= "OP monitor interval=" 30s "

------------------------------------------------------------------------------------------------------
/etc/corosync/corosync.conf
The token meaning in the file is:
The token value specifies the time, in milliseconds, during which the Corosync token are expected to be transmitted around The ring. When this timeout expires, the token was declared lost, and after Token_retransmits_before_loss_const lost tokens, the non- Responding processor (cluster node) is declared dead. In other words, tokenxtoken_retransmits_before_loss_const are the maximum time a node is allowed to not respond to Cluste R messages before being considered dead. The default for token is a milliseconds (1 second), with 4 allowed retransmits. These defaults is intended to minimize failover times, but can cause frequent ' false alarms ' and unintended failovers in Case of the short network interruptions. The values used here is safer, albeit with slightly extended failover times.

There are two ways of node communication, one is multicast and one is unicast. I used unicast in my test, which specifies the nodelist.


Multicast mode configuration:
Totem {
Version:2

# time (in MS) to wait for a token (1)
token:10000

# How many token retransmits before forming a new
# Configuration
Token_retransmits_before_loss_const:10

# Turn off the virtual synchrony filter
Vsftype:none

# Enable Encryption (2)
Secauth:on

# How many threads to use for encryption/decryption
threads:0

# This specifies the redundant ring protocol, which could be
# None, active, or passive. (3)
Rrp_mode:active

# The following is a two-ring multicast configuration. (4)
interface {
ringnumber:0
bindnetaddr:192.168.42.0
mcastaddr:239.255.42.1
mcastport:5405
}
interface {
Ringnumber:1
bindnetaddr:10.0.42.0
mcastaddr:239.255.42.2
mcastport:5405
}
}

AMF {
Mode:disabled
}

Service {
# Load The Pacemaker Cluster Resource Manager (5)
Ver:1
Name:pacemaker
}

aisexec {
User:root
Group:root
}

Logging {
Fileline:off
To_stderr:yes
To_logfile:no
To_syslog:yes
Syslog_facility:daemon
Debug:off
Timestamp:on
Logger_subsys {
Subsys:amf
Debug:off
Tags:enter|leave|trace1|trace2|trace3|trace4|trace6
}}
1. The result of multiplying tokens and token_retransmits_before_loss_const determines the failover time of the cluster. The token unit is in milliseconds. If a node exceeds $ (token*token_retransmits_before_loss_const) is not responding, the node is considered dead.
2. If the Secauth option is enabled, communication between nodes is encrypted with a 128-bit key, and the key is stored in/etc/corosync/authkey, which can be generated using Corosync-keygen.
3, Corosync configuration requires redundant network (with more than one network interface), must adopt the RRR mode,
Note the following recommended interface configurations:
3.1, the configuration of each interface must have a unique ringnumber, and the starting value is 0.
3.2, BINDNETADDR is the IP address network segment that you want to bind,
3.3, multicast address mcastaddr can not be reused across the boundaries of the cluster, that is, there are never two separate clusters using the same multicast group address. The address of the multicast group must follow the RfC 2365, "Administratively Scoped IP multicast"
3.4, firewall configuration aspect, Corosync only need UDP communication, use Mcastport (receive) and Mcastport-1 (send).
4, pacemaker services can be declared in the corosync.conf, or in the/etc/corosync/service.d/pacemaker can be declared.
Note: If you are Corosync (version 2) under Ubuntu 14.04, you need to annotate the command in the service stanza to start pacemaker. Additionally, you need to be aware of the boot order of Corosync and pacemaker, which you need to specify manually:
# UPDATE-RC.D Pacemaker Start 20 2 3 4 5. Stop 00 0 1 6.
/etc/corosync/uidgid.d/pacemaker must be added:
Uidgid {
Uid:hacluster
Gid:haclient
}
5. Note that corosync.conf and authkey on all nodes should be kept in sync.

6, the Service node, ver:1 indicates that Corosync does not automatically start pacemaker. If you want to corosync auto-start pacemaker, set ver to 0. Since CENTOS7 does not have a/etc/rc.d/init.d/pacemaker script (pacemaker service can be set with systemctl command in Centos7), I do not have this section in my configuration. You can create/etc/rc.d/init.d/pacemaker manually, the same way you would create a/etc/rc.d/init.d/haproxy script in the next article about Haproxy.


Unicast mode configuration:
Totem {
#...
interface {
ringnumber:0
bindnetaddr:192.168.42.0
Broadcast:yes (1)
mcastport:5405
}
interface {
Ringnumber:1
bindnetaddr:10.0.42.0
Broadcast:yes
mcastport:5405
}
TRANSPORT:UDPU (2)
}

nodelist {(3)
Node {
ring0_addr:192.168.42.1
ring1_addr:10.0.42.1
Nodeid:1
}
Node {
ring0_addr:192.168.42.2
ring1_addr:10.0.42.2
Nodeid:2
}
}

1. If the broadcast parameter is set to Yes, the broadcast address is used for communication. If the broadcast parameter is used, do not use the MCASTADDR parameter.
2, the Transport instruction control transmission method. If you want to completely eliminate multicasting, specify UDPU unicast transport parameters. This requires a list of members to be specified with NodeList. The transport default is UDP, or it can be set to UPDU or IBA.
3, nodelist Each node, must have at least one ring0_addr field, other possible options are RING{X}_ADDR and nodeid,{x} is the ring ordinal, ring{x}_addr the specified node IP, Nodeid is specified only when you need to use both IPV4 and IPv6.


Votequorum Mode configuration
The Votequorum Library is part of the Corosync project. The use of votequorum is to avoid brain fissures, as well:
1, query quorum status;
2. Obtain the list of nodes known to the quorum service;
3, receive the quorum status change notice;
4. Change the number of votes, and assign a node to the domain (the changes of the numbers of votes assigned to a node)
5. Change the number of expected votes for a cluster to be quorate
6. Connect an additional quorum device to allow small clusters remain quorate during node outages
The Votequorum library was created to replace and replace the Qdisk (voting disc).
The corosync.com with the Votequorum configuration are as follows:
Quorum {
Provider:corosync_votequorum (1)
Expected_votes:7 (2)
Wait_for_all:1 (3)
Last_man_standing:1 (4)
last_man_standing_window:10000 (5)
}
Attention:
Corosync_votequorum started the votequorum.
Expected_votes is 7, 7 nodes, and quorum is 4. If the nodelist parameter is set, Expected_votes is not valid.
A wait_for_all value of 1 means that when the cluster is started, the cluster quorum is suspended until all nodes are online and joined to the cluster, and this parameter is added to Corosync 2.0.
Last_man_standing is 1, enabling the LMS feature. This feature is off by default, which is a value of 0. When this parameter is turned on, when the cluster is at the voting edge (such as expected_votes=7, while the current online nodes=4) is in the voting edge state longer than the Last_man_standing_window parameter specifies, the quorum is recalculated, Until online nodes=2. If you want online nodes to be equal to 1, you must enable the Auto_tie_breaker option, which is not recommended for production environments.
The Last_man_standing_window unit is in milliseconds. After one or more hosts are lost from the cluster, recalculate the quorum.

OpenStack Controller HA test environment build record (ii)--Configure Corosync and pacemaker