High Availability of Redis-Failover transition Solution

From the official apsaradb for Redis roadmap, Cluster will be officially supported around Redis3.0. However, even optimistic estimates have to wait for at least a few months. In order to ensure high availability for my applications, I implemented a Failover transition scheme based on the master-slave server.

 

Theoretically, once the master server goes offline, you can select a new master server from the server and reset the master-slave relationship, in addition, the server can be automatically added to the master-slave relationship after the server goes online again. The content is as follows:

<?phpclass RedisFailover{    public $config = array();    public $map    = array();    const CONFIG_FILE = 'config.php';    const MAP_FILE    = 'map.php';    public function __construct()    {        $config = include self::CONFIG_FILE;        foreach ((array)$config as $name => $nodes) {            foreach ($nodes as $node) {                $node = new RedisNode($node['host'], $node['port']);                if ($node->isValid()) {                    $this->config[$name][] = $node;                }            }            if (empty($this->config[$name])) {                throw new Exception('Invalid config.');            }            $this->map[$name] = $this->config[$name][0];        }        if (file_exists(self::MAP_FILE)) {            $map = include self::MAP_FILE;            foreach ((array)$map as $name => $node) {                $node = new RedisNode($node['host'], $node['port']);                $this->map[$name] = $node;            }        }    }    public function run()    {        $set_nodes_master = function($nodes, $master) {            foreach ($nodes as $node) {                $node->setMaster($master->host, $master->port);            }        };        foreach ($this->config as $name => $nodes) {            $is_master_valid = false;            foreach ($nodes as $node) {                if ($node == $this->map[$name]) {                    $is_master_valid = true;                    break;                }            }            if ($is_master_valid) {                $set_nodes_master($nodes, $this->map[$name]);                continue;            }            foreach ($nodes as $node) {                $master = $node->getMaster();                if (empty($master)) {                    continue;                }                if ($master['master_host'] != $this->map[$name]->host) {                    continue;                }                if ($master['master_port'] != $this->map[$name]->port) {                    continue;                }                if ($master['master_sync_in_progress']) {                    continue;                }                $node->clearMaster();                $set_nodes_master($nodes, $node);                $this->map[$name] = $node;                break;            }        }        $map = array();        foreach ($this->map as $name => $node) {            $map[$name] = array(                'host' => $node->host, 'port' => $node->port            );        }        $content = '<?php return ' . var_export($map, true) . '; ?>';        file_put_contents(self::MAP_FILE, $content);    }}class RedisNode{    public $host;    public $port;    const CLI = '/usr/local/bin/redis-cli';    public function __construct($host, $port)    {        $this->host = $host;        $this->port = $port;    }    public function setMaster($host, $port)    {        if ($this->host != $host || $this->port != $port) {            return $this->execute("SLAVEOF {$host} {$port}") == 'OK';        }        return false;    }    public function getMaster()    {        $result = array();        $this->execute('INFO', $rows);        foreach ($rows as $row) {            if (preg_match('/^master_/', $row)) {                list($key, $value) = explode(':', $row);                $result[$key] = $value;            }        }        return $result;    }    public function clearMaster()    {        return $this->execute('SLAVEOF NO ONE') == 'OK';    }    public function isValid()    {        return $this->execute('PING') == 'PONG';    }    public function execute($command, &$output = null)    {        return exec(            self::CLI . " -h {$this->host} -p {$this->port} {$command}", $output        );    }}?>

Two files are mentioned here. Let's talk about config. php first:

<?phpreturn array(    'redis_foo' => array(        array('host' => '192.168.0.1', 'port' => '6379'),        array('host' => '192.168.0.2', 'port' => '6379'),        array('host' => '192.168.0.3', 'port' => '6379'),    ),);?>

Note: Each alias corresponds to a group of servers. One of these servers is the master server, and the other is the slave server. Do not hard-code the master-slave relationship in the configuration file, the SLAVEOF command should be used for dynamic setting.

The content of the map. php file is as follows:

<?phpreturn array (    'redis_foo' => array (        'host' => '192.168.0.1', 'port' => '6379'    ),);?>

Note: The alias corresponds to the currently valid server. Note that this file is automatically generated! When using Redis, the program is configured as an alias. The specific host and port are obtained through this file ing.

After understanding the above Code, it is easy to run:

<?php$failover = new RedisFailover();$failover->run();?>

Note: In actual deployment, the strictest way is to run the program in the daemon mode. However, if the requirement is not very harsh, CRON is enough. During the test, you can manually kill the master server process and view the effect through INFO.

I will add some instructions on command line usage. This article uses redis-cli to send commands, which is usually the best choice. However, if redis-cli is not available for some reason, you can also use the ncnetcat command to implement a simple client tool according to the Redis protocol. For example, the PING command can be implemented as follows:

shell> (echo -en "PING\r\n"; sleep 1) | nc localhost 6379

Note: sleep is required because Redis's request response mechanism is Pipelining.

Now that we have mentioned this, we have to pay another 10 yuan. Generally, we can use telnet commands to interact with the service. However, what is unpleasant about telnet is that the command line does not support up/down key history, fortunately, you can use rlwrap to achieve this goal. Depending on the operating system, you can easily use APT or YUM for installation. The operation is also very simple:

shell> rlwrap telnet localhost 6379

Note: Using rlwrap not only supports the up and down key history, but also supports Ctrl + r search!

...

Before Redis Cluster release, I hope this script can help you. In fact, other services can use similar solutions, such as MySQL, but the complexity will increase a lot, fortunately, there are already MHA-like solutions.