MHA automatic failover steps and Process Analysis

MHA is the best choice for many enterprises that use MySQL databases for high availability. It is easy to use and has powerful functions. It implements automatic master-slave failover Based on the MySQLreplication architecture.

MHA is the best choice for many enterprises that use MySQL database for high availability. It is easy to use and powerful, and implements automatic master-slave failover Based on the MySQL replication architecture.

MHA is the best choice for many enterprises that use MySQL databases for high availability. It is easy to use and powerful, and implements automatic master-slave failover Based on the MySQL replication architecture, this article describes the steps for MHA automatic switching, demonstrates the switching process, and makes appropriate analysis for your reference and understanding of the principles of MHA and MySQL.

1. MHA automatic failover procedure
A. MHA manager validation phase at startup
Verify the replication configuration based on the configuration file and identify the current master
Cause monitoring termination: Replication configuration exception, abnormal slave, and some required script exceptions
Before MHA manager is started, it can be detected by masterha_check_ssh and masterha_check_repl.

B. master server monitoring stage
After MHA manager is started successfully, the output log [info] Ping (SELECT) succeeded, waiting until MySQL doesn't respond ..
MHA will regularly ping the master. The default interval is 1 second until the master dies.
MHA manager does not monitor the slave status, and any Stopping/Restarting/Adding/Removing slaves does not affect the current MHA monitoring.
When adding or removing slave nodes, we recommend that you modify the configuration file and restart MHA manager.

C. Detect master server exceptions
Failed to ping the master three times consecutively. The parameter secondary_check_script can be used for double check.

D. Start failover.
MHA reads the configuration file again (to prevent the master-slave configuration from being changed) and verifies the slave database of the master and the current masetr.
MHA also checks whether the slave database has different master nodes, such as replication exceptions.
Start failover (exclude the failure of the last failover or the interval between failover is too short)

E. Isolate the master server stage (optional)

If the configuration file defines an IP address drift script, the master IP address is disabled.
If the configuration file defines to close the master script, the script is called to close the master to avoid split-brain

F. Restore a new master.
Obtain the latest end_log_pos (Read_Master_Log_Pos) from the slave node.
Read and save all the binary log files of the crashed master from the location obtained in the previous step (ssh accessible)
Determine the new master (based on the priority defined in the configuration file)
Generate binary/relay log events of the difference and apply it to the new master

G. Activate a new master
If the configuration file defines the IP address drift script, the IP address will be moved to the new master.

I. Restore the remaining slave
Parallel generation of binary/relay log events for all slave Databases
Use the apply difference binary/relay log events to each slave Database
Start Replication

J. Switch result (optional)
Email sending
Backup job, management tool adjustment, etc.

2. MHA configuration information
[Root @ vdbsrv4 ~] $ More/etc/masterha/app1.cnf
[Server default]
Manager_workdir =/var/log/masterha/app1
Manager_log =/var/log/masterha/app1/manager. log

User = mha
Password = xxx
Ssh_user = root
Repl_user = repl
Repl_password = repl
Ping_interval = 1
Shutdown_script = ""
Master_ip_online_change_script = ""
Report_script = ""
Master_ip_failover_script =/tmp/master_ip_failover

[Server1]
Hostname = vdbsrv1
Master_binlog_dir =/data/mysqldata

[Server2]
Hostname = vdbsrv2
Master_binlog_dir =/data/mysqldata

[Server3]
Hostname = vdbsrv3
Master_binlog_dir =/data/mysqldata/
# Candidate_master = 1

[Root @ vdbsrv1 ~] # More/etc/hosts
127.0.0.1 localhost. localdomain localhost
192.168.1.6 vdbsrv1 # master
192.168.1.7 vdbsrv2 # slave1
192.168.1.8 vdbsrv3 # slave2
192.168.1.12 vdbsrv4 # manager

Mysql> show slave hosts;
+ ----------- + --------- + ------ + ----------- + -------------------------------------- +
| Server_id | Host | Port | Master_id | Slave_UUID |
+ ----------- + --------- + ------ + ----------- + -------------------------------------- +
| 1001 | vdbsrv2 | 3306 | 1 | 75bef614-e342-11e4-921d-000c295fb2eb |
| 1002 | vdbsrv3 | 3306 | 1 | 091f79b8-e386-11e4-93d5-000c2943c830 |
+ ----------- + --------- + ------ + ----------- + -------------------------------------- +
2 rows in set (0.00 sec)

3. Demo Switching
### Enable master database VIP
[Root @ vdbsrv4 ~] # Ssh vdbsrv1 "/sbin/ifconfig eth0: 0 192.168.1.13 netmask 255.255.255.0 up"

### Enable MHA manager
[Root @ vdbsrv4 ~] # Nohup masterha_manager -- conf =/etc/masterha/app1.cnf &

### Use sysbench to generate transactions
[Root @ vdbsrv1 ~] # Sysbench -- test = oltp \
> -- Oltp-table-size = 5000000 \
> -- Oltp-read-only = off \
> -- Init-rng = on \
> -- Num-threads = 16 \
> -- Max-requests = 0 \
> -- Oltp-dist-type = uniform \
> -- Max-time = 180 \
> -- Mysql-user = root \
> -- Mysql-socket =/tmp/mysql. sock \
> -- Mysql-password = ''\
> -- Db-driver = mysql \
> -- Mysql-table-engine = innodb \
> -- Oltp-test-mode = complex \
> -- Mysql-db = replicatedb run
Sysbench 0.4.12: multi-threaded system evaluation benchmark