Monitor MHA Masterha_manager processes with supervisor

We need to consider how to keep the Masterha_manager monitoring process in a normal state when we use MHA's Masterha_manager script to do automatic failover of MySQL main library. The supervisor can be a good solution to this problem, it can be a normal command-line process into the background daemon, and monitor the status of the process, abnormal exit can be automatically restarted.

Here's a list of deployment Essentials and management commands

One, Supervisor installation:

sudo pip install Supervisor

Two, supervisor configuration:

Mkdir-p/etc/supervisor/conf.d/

Build configuration file
echo_supervisord_conf >/etc/supervisor/supervisord.conf

Cat/etc/supervisor/supervisord.conf

[unix_http_server]
File=/tmp/supervisor.sock; the path to the socket file

[supervisord]
Logfile=/tmp/supervisord.log; main log file; default $CWD/supervisord.log
logfile_maxbytes= 50MB; Max main logfile bytes b4 rotation; Default 50MB
logfile_backups=10; # of main logfile backups; 0 means none, default ten
Loglevel=info; log level; def Ault info; Others:debug,warn,trace
Pidfile=/tmp/supervisord.pid; supervisord pidfile; default Supervisord.pid
nodaemon= false; Start in foreground if true; Default false
minfds=1024; min. avail startup file descriptors; default 1024x768
minprocs=200; min. avail Process desc Riptors;default

[Rpcinterface:supervisor]
Supervisor.rpcinterface_factory = Supervisor.rpcinterface:make_main_rpcinterface

[Supervisorctl]
Serverurl=unix:///tmp/supervisor.sock; Use a unix://URL for a UNIX socket

[include]
Files =/etc/supervisor/conf.d/*.conf

Configuration essentials:
1, the other configuration will be able to use the generated default configuration, but user needs to do password-free login users, such as the dbadmin here, or Masterha_manager startup will be wrong, because MHA password-free login is all the Dbadmin account
2, the configuration of the management process can be placed directly in the [Program:xxx] segment of the Supervisor master configuration file, but it is best to prepare a profile for each process for easy administration, specifying the profile directory through the file configuration of the [include] segment.

Third, take masterha_manager monitoring test as an example to see Supervisor Operation 1, prepare Masterha_manager monitoring test configuration file

Cat/etc/supervisor/conf.d/masterha_manager_test.conf
[Program:masterha_manager_test]
Command=masterha_manager--conf=/etc/mha/test.cnf--ignore_last_failover; Start command
Stdout_logfile=/tmp/manager.log; StdOut Log Output location
Stderr_logfile=/tmp/manager.log; STDERR Log Output location
Autostart=true; Auto-start when Supervisord is started
Autorestart=true; Automatic restart after program exits unexpectedly
startsecs=10; No exception exits after 10 seconds of startup, as if it had started normally

2, start the Supervisord process

# supervisord-c /etc/supervisor/supervisord.conf
# Ps-ef | grep Super
Dbadmin 11892 1 0 02:56? 00:00:00/usr/bin/python/usr/bin/supervisord
Root 13340 31610 0 02:56 pts/0 00:00:00 grep Super

3. View the status of monitored processes

# SUPERVISORCTL Status

Masterha_manager_test RUNNING pid 11912, uptime 0:03:08

# Ps-ef | grep Master
Root 1343 31610 0 02:59 pts/0 00:00:00 grep Master
Root 3228 1 0 2016? 00:01:33/usr/libexec/postfix/master
Dbadmin 11912 11892 0 02:56? 00:00:00 Perl/usr/local/bin/masterha_manager--conf=/etc/mha/test.cnf--ignore_last_failover
You can see the Masterha_manager has started.

4, test

Kill the Masterha_manager process directly to simulate the Masterha_manager exception exit:
# Ps-ef | grep Master
Root 1343 31610 0 02:59 pts/0 00:00:00 grep Master
Root 3228 1 0 2016? 00:01:33/usr/libexec/postfix/master
Dbadmin 11912 11892 0 02:56? 00:00:00 Perl/usr/local/bin/masterha_manager--conf=/etc/mha/test.cnf--ignore_last_failover

# kill-9 11912

# Ps-ef | grep Master
Dbadmin 1707 11892 5 03:30? 00:00:00 Perl/usr/local/bin/masterha_manager--conf=/etc/mha/test.cnf--ignore_last_failover
Root 2054 31610 0 03:30 pts/0 00:00:00 grep Master
Root 3228 1 0 2016? 00:01:33/usr/libexec/postfix/master

You can see the supervisor and restart the Masterha_manager monitoring process.

5, common management commands

Supervisord: Initial start Supervisord, start, manage the process set in configuration;
Supervisorctl Stop (start, restart) xxx, stop (start, restart) a certain process (XXX);
Supervisorctl Reread: Loads only the latest configuration files and does not restart any processes;
Supervisorctl Reload: Load the latest configuration file, stop all the original processes and start managing all processes as new configuration;
SUPERVISORCTL Update: According to the latest configuration file, start a new configuration or a modified process, configure the unchanged process will not be affected and restart;

6, add Supervisord for Linux system service, boot up automatically

Prepare the startup script supervisord.sh

# chmod +x supervisord.sh

# MV Supervisord.sh/etc/init.d/supervisord

# chkconfig--add Supervisord

# chkconfig--level 345 Supervisord on

Cat/etc/rc.d/init.d/supervisord

#!/bin/sh

#

#/etc/rc.d/init.d/supervisord

#

# Supervisor is a client/server system

# allows its users to monitor and control a

# Number of processes on unix-like operating

# systems.

#

# Chkconfig:-64 36

# Description:supervisor Server

# Processname:supervisord

# Source init functions

. /etc/rc.d/init.d/functions

Prog= "Supervisord"

Prog_bin= "/usr/bin/supervisord"

Pidfile= "/tmp/supervisord.pid"

Confile= "/etc/supervisor/supervisord.conf"

Start ()

{

echo-n $ "Starting $prog:"

daemon $prog _bin-c $CONFILE--pidfile $PIDFILE

[-F $PIDFILE] && success $ "$prog Startup" | | Failure $ "$prog startup"

Echo

}

Stop ()

{

echo-n $ "shutting down $prog:"

[-F $PIDFILE] && Killproc $prog | | Success $ "$prog shutdown"

Echo

}

Case "$" in

start)

Start

 ;;

stop)

Stop

 ;;

status)

Status $prog

 ;;

Restart)

Stop

Start

 ;;

 *)

echo "Usage: $ {Start|stop|restart|status}"

 ;;

Esac

Monitor MHA Masterha_manager processes with supervisor