Introduction, start-up and communication of Redis database Sentinel

Sentinel (Redis 3.0.0-rc1)

Sentinel is a sentinel system of Redis ha scenarios, one or more sentinel instances, that can monitor any number of primary servers (master).
And all the subordinate servers (slave) of these primary servers, and when the monitored primary server enters the offline state, automatically upgrades to a new primary server from the server under the primary server.
The new primary server then replaces the offline primary server to continue processing the command request.

BASIC Data Structure

The code is as follows

Copy Code

typedef struct SENTINELREDISINSTANCE { The type and status of the current instance (master, slave, Sentinel, offline) int flags; Host name, Ip:port Char *name; Runid of the instance Char *runid; Configure ERAs uint64_t Config_epoch; The address of the instance Sentineladdr *addr; Redisasynccontext *cc; /* Hiredis context for commands. */ Redisasynccontext *pc; /* Hiredis context for pub/sub. */ int pending_commands; /* Number of commands sent waiting for a reply. */ mstime_t Cc_conn_time; /* CC connection time. */ mstime_t Pc_conn_time; /* PC connection time. */ mstime_t pc_last_activity; /* Last time we received any message. */ mstime_t Last_avail_time; /* The instance replied to ping with A reply we consider valid. */ mstime_t Last_ping_time; /* Last time a pending ping is sent in the Context of the current command connection With the instance. 0 if still not sent or If Pong already received. */ mstime_t Last_pong_time; /* Last time the instance replied to Ping, Whatever the reply was. That ' s used to check If the link is idle and must be reconnected. */ mstime_t Last_pub_time; /* Last time we sent hello via pub/sub. */ mstime_t Last_hello_time; /* Only used if Sri_sentinel is set. Last time We received a Hello from this Sentinel Via Pub/sub. */ mstime_t Last_master_down_reply_time; /* Time of last reply to SENTINEL is-master-down command. */ mstime_t S_down_since_time; /* subjectively down since time. */ mstime_t O_down_since_time; /* Objectively down since time. */ No response how many milliseconds after entering the subjective offline mstime_t Down_after_period; mstime_t Info_refresh; /* Time in which we received INFO output from it. */ /* Role and the observed it. * This is useful in order to delay replacing what the instance reports * with our own configuration. We need to always wait some * To give a chance to the leader to the new configuration before * We do silly things. */ int role_reported; mstime_t Role_reported_time; mstime_t Slave_conf_change_time; /* Last time slave master addr changed. */ /* Master specific. */ Dict *sentinels; /* Other Sentinels monitoring the same master. */ Dict *slaves; /* Slaves for this master instance. */ To determine the number of support votes required for objective off-line unsigned int quorum; Number of slave that can be synchronized with the new master at the time of failover int Parallel_syncs; /* How many slaves to reconfigure in same time. */ Char *auth_pass; /* Password to use for AUTH against master & Slaves. */ /* Slave specific. */ mstime_t Master_link_down_time; /* Slave replication link down time. */ int slave_priority; /* Slave Priority according to its INFO output. */ mstime_t Slave_reconf_sent_time; /* Time at which we sent SLAVE of <new> */ struct Sentinelredisinstance *master; /* Master instance if it ' s slave. */ Char *slave_master_host; /* Master host as reported by INFO */ int slave_master_port; /* Master Port as reported by INFO */ int slave_master_link_status; /* Master link status as reported by INFO */ unsigned long long slave_repl_offset; /* Slave replication offset. */ * Failover * * Char *leader; /* If is a master instance, this is the Runid of The Sentinel that should perform the failover. If This is a Sentinel and this is the Runid of the Sentinel That is Sentinel voted as leader. */ uint64_t Leader_epoch; /* Epoch of the ' leader ' field. */ uint64_t Failover_epoch; /* Epoch of the currently started failover. */ int failover_state; /* sentinel_failover_state_* defines. */ mstime_t Failover_state_change_time; mstime_t Failover_start_time; /* Last failover attempt start time. */ Maximum time limit for refreshing failed migration status mstime_t failover_timeout; /* Max time to refresh failover state. */ mstime_t failover_delay_logged; /* For what failover_start_time value we Logged the failover delay. */ struct Sentinelredisinstance *promoted_slave; /* promoted slave instance. */ /* Scripts executed to notify admin or reconfigure Clients:when they * are set to NULL no script is executed. */ Char *notification_script; Char *client_reconfig_script; } sentinelredisinstance; struct Sentinelstate { Current era uint64_t Current_epoch; Master dictionary for monitoring, key is master name, value is Sentinelredisinstance object Dict *masters; is in tilt mode int tilt; Number of scripts currently being executed int running_scripts; Enter TITL mode time mstime_t Tilt_start_time; Time of last execution time processor mstime_t Previous_time; User Script Execution Queue List *scripts_queue; } Sentinel;

Sentinel Initialize

Start command:

redis-sentinel/path/to/sentinel.conf


or When

redis-server/path/to/sentinel.conf--sentinel


Sentinel is started, you must specify a configuration file that is similar to the minimum configuration:

Sentinel Monitor MyMaster 127.0.0.1 6379 2
Sentinel down-after-milliseconds mymaster 60000
Sentinel Failover-timeout Mymas ter 180000
Sentinel parallel-syncs mymaster 1


This configuration file indicates that the current sentinel monitors a Redis master (MyMaster), IP is 127.0.0.1, the port is 6379, and
requires two sentinel declarations to be off the main standby switch. MyMaster 60000ms is not responding to the mark as invalid.

in the main function, the special configuration is started directly on the Sentinel:

The code is as follows	Copy Code
int main (int argc, char **argv) { ... if (Server.sentinel_mode) { Initsentinelconfig (); Initsentinel (); } ... }

The first is to overwrite the port settings of Redis Server, Sentinel will be listening on port 26379 by default:

The code is as follows	Copy Code
#define Redis_sentinel_port 26379 void Initsentinelconfig (void) { Server.port = Redis_sentinel_port; }

Then overwrite the server's command table and initialize the Sentinel object:

The code is as follows

Copy Code

void Initsentinel (void) { unsigned int J; /* Remove usual Redis commands from the command table, then just add * The SENTINEL command. */ Empty the command form supported by the Redis and change it to a command form supported by Sentinel Dictempty (Server.commands,null); for (j = 0; J < sizeof (SENTINELCMDS)/sizeof (Sentinelcmds[0)); J + +) { int retval; struct Rediscommand *cmd = sentinelcmds+j; retval = Dictadd (Server.commands, Sdsnew (cmd->name), CMD); Redisassert (retval = = DICT_OK); } /* Initialize various data structures. */ Sentinel.current_epoch = 0; Sentinel.masters = Dictcreate (&instancesdicttype,null); sentinel.tilt = 0; Sentinel.tilt_start_time = 0; Sentinel.previous_time = Mstime (); sentinel.running_scripts = 0; Sentinel.scripts_queue = Listcreate (); Sentinel.announce_ip = NULL; Sentinel.announce_port = 0; }

Sentinel can only support monitoring related commands and cannot perform the usual Redis commands, sentinel the command forms that are supported are:

code is as follows

copy code

struct Rediscommand sentinelcmds[] = {     {"Ping", pingcommand,1, "", 0,null,0,0,0,0,0},      {"Sentinel", Sentinelcommand,-2, "", 0,null,0,0,0,0,0},     {"Subscribe", Subscribecommand, -2, "", 0,null,0,0,0,0,0},     {"Unsubscribe", Unsubscribecommand,-1, "", 0,null,0,0,0,0,0},     {"Psubscribe", Psubscribecommand,-2, "", 0,null,0,0,0,0,0},     {"Punsubscribe ", Punsubscribecommand,-1," ", 0,null,0,0,0,0,0},     {" Publish ", sentinelpublishcommand,3," ", 0, null,0,0,0,0,0},     {"Info", Sentinelinfocommand,-1, "", 0,null,0,0,0,0,0},      {"Role", sentinelrolecommand,1, "L", 0,null,0,0,0,0,0},     {"Shutdown", shutdowncommand,-1 , "", 0,null,0,0,0,0,0} };

At the same time, because Sentinel does not accept Redis common commands, it does not load raw data such as RDB files when initialized.



Sentinel Communication


Communication

After initialization is complete, Sentinel will actively communicate with master, slave, to obtain their information.
Get Primary Server information

First, Sentinel will establish two connections with master, which are command connections and subscription connections (stored in the CC and PC fields of Sentinelredisinstance, respectively).

The code is as follows	Copy Code
void Sentinelhandleredisinstance (Sentinelredisinstance *ri) { Sentinelreconnectinstance (RI); ... } #define Sentinel_hello_channel "__sentinel__:hello"

The command connection is used for subsequent access to master's information (including slave information), and subscriptions (Sentinel:hello Channels) are used to push messages such as the drop state of master.

When the connection is complete, Sentinel sends a message periodically:

The code is as follows

Copy Code

void Sentinelsendperiodiccommands (Sentinelredisinstance *ri) { mstime_t now = Mstime (); mstime_t Info_period, Ping_period; int retval; /* Return ASAP If we have already a PING or INFO already pending, or * In the case, the instance is not properly connected. */ if (Ri->flags & sri_disconnected) return; /* for INFO, PING, PUBLISH this are not critical commands to send we * Also have a limit of sentinel_max_pending_commands. We don ' t * Want to use a lot of memory just because a link isn't not working * Properly (note that anyway there are a redundant protection about this, * That is, the link would be disconnected and reconnected if a long * Timeout condition is detected. */ if (ri->pending_commands >= sentinel_max_pending_commands) return; /* If This is a slave of the A master in o_down condition we start sending * It INFO every second, instead of the usual sentinel_info_period * Period. In the We want to closely monitor slaves in case they * are turned into Masters by another Sentinel, or by the sysadmin. */ INFO command The default send interval is 10s #define SENTINEL_INFO_PERIOD 10000 If you are a slave for a master that has been down, change the 1s if (Ri->flags & Sri_slave) && (Ri->master->flags & (sri_o_down| sri_failover_in_progress))) { Info_period = 1000; } else { Info_period = Sentinel_info_period; } /* We Ping instances every time, received Pong is older than * The configured ' Down-after-milliseconds ' time, but every second * Anyway if ' down-after-milliseconds ' is greater than 1 second. */ PING command default interval 1s #define SENTINEL_PING_PERIOD 1000 Ping_period = ri->down_after_period; if (Ping_period > Sentinel_ping_period) Ping_period = Sentinel_ping_period; if (Ri->flags & Sri_sentinel) = = 0 && (Ri->info_refresh = 0 | | (Now-ri->info_refresh) > Info_period)) { /* Send INFO to Masters and slaves, not sentinels. */ retval = Redisasynccommand (RI->CC, Sentinelinforeplycallback, NULL, "INFO"); if (retval = = REDIS_OK) ri->pending_commands++; else if ((now-ri->last_pong_time) > ping_period) { /* Send PING to all three kinds of instances. */ Sentinelsendping (RI); else if ((now-ri->last_pub_time) > Sentinel_publish_period) { /* PUBLISH Hello messages to the three kinds of instances. */ Sentinelsendhello (RI); } }

Handling the return of info message:

code is as follows	copy code
void Sentinelinforeplycallback (Redisasynccontext *c, void *reply, void *privdata) {     Sentinelredisinstance *ri = c->data;     redisreply *r;     redis_notused (privdata);     if (RI) ri->pending_commands--     if (!reply | |!ri) return;     r = reply;     if (R->type = redis_reply_string) {          Sentinelrefreshinstanceinfo (RI,R->STR);    } }

This is for Info,sentinel sent to master:
1. Get Master run_id Records, check master role, update the master list of your own maintenance
2. Copy the fields, get master's slave list, update the slave list that you maintain
Get slave information

Similarly, Sentinel sends the info command to the slave at the same frequency and extracts the following parameters:
* run_id
* Role
* Master's host and port
* The connection status of the master-slave server (master_link_status)
* Slave priority (slave_priority)
* Slave replication offset (slave_repl_offset)
and update the SENTINELREDISINSTANCE structure that you maintain.
Send and receive subscription information

Sentinel sends information to all master and slave through a command connection per second.

The code is as follows

Copy Code

int Sentinelsendhello (sentinelredisinstance *ri) { Char Ip[redis_ip_str_len]; Char payload[redis_ip_str_len+1024]; int retval; Char *announce_ip; int announce_port; Sentinelredisinstance *master = (ri->flags & Sri_master)? ri:ri->master; Sentineladdr *master_addr = sentinelgetcurrentmasteraddress (master); if (Ri->flags & sri_disconnected) return redis_err; /* Use the specified announce address if specified, otherwise try to * Obtain our own IP address. */ if (SENTINEL.ANNOUNCE_IP) { Announce_ip = sentinel.announce_ip; } else { if (Anetsockname (ri->cc->c.fd,ip,sizeof (IP), NULL) = =-1) return redis_err; ANNOUNCE_IP = IP; } Announce_port = Sentinel.announce_port? Sentinel.announce_port:server.port; /* Format and send the Hello message. */ snprintf (payload,sizeof (payload), "%s,%d,%s,%llu,"/* Info about this Sentinel. */ "%s,%s,%d,%llu",/* Info about current master. */ Announce_ip, Announce_port, Server.runid, (unsigned long long) Sentinel.current_epoch, /* --- */ Master->name,master_addr->ip,master_addr->port, (unsigned long long) master->config_epoch); retval = Redisasynccommand (RI->CC, Sentinelpublishreplycallback, NULL, "PUBLISH%s%s", Sentinel_hello_channel,payload); if (retval!= REDIS_OK) return redis_err; ri->pending_commands++; return REDIS_OK; }

Send content includes:
* Sentinel IP (ANNOUNCE_IP)
* Sentinel Port (Announce_port)
* Sentinel Run ID (server.runid)
* Sentinel Configuration era (Sentinel.current_epoch)
* Master Name (master->name)
* Master IP (MASTER_ADDR->IP)
* Master Port (Master_addr->port)
* Master era (Master->config_epoch)

At the same time, all Sentinel connected to this master will receive the message and then respond:
* Update Sentinels Dictionary
* Create a command connection to connect to other Sentinel