Summary of Redis monitoring tips

The most straightforward approach to redis monitoring, of course, is to use the Info command provided by the system, and you only need to execute the following command to get a status report for the Redis system.

REDIS-CLI Info

Memory usage

If the memory used by Redis exceeds the available physical memory size, Redis is likely to be killed by Oom killer. For this, you can use the info command to monitor used_memory and Used_memory_peak, set thresholds for the amount of memory used, and set the appropriate alarm mechanism. Of course, the alarm is only a means, the important thing is that you have to plan ahead, when the memory usage is too large, you should do something to clean up some useless cold data, or to move Redis to a more powerful machine.

Persistence of

If Redis crashes because of problems with your machine or redis itself, your only lifeline might be the Rdb file that you dump, so it's important to monitor the Redis dump file. You can monitor the rdb_last_save_time to see when your last dump data was operating, and you can also monitor the rdb_changes_since_last_save for knowledge of how much data you will lose if it fails.

Master-slave replication

If you set the master-slave replication mode, then you'd better do some monitoring of the situation of replication, mainly for the master_link_status in the info output to monitor, if this value is up, then the synchronization is normal, if it is down, Then you need to look at some other diagnostic information for the output. For example, the following:

Role:slave master_host:192.168.1.128 master_port:6379 Master_link_status:down Master_last_io_seconds_ago:-1 master_sync_in_progress:0 master_link_down_since_seconds:1356900595

Fork Performance

When Redis persists data to disk, it makes a fork operation, and the most inexpensive memory mirroring is implemented by fork to the copy on write mechanism of memory. But while the memory is copy on write, the virtual memory table needs to be allocated at the moment of the fork, so the fork causes a short lag of the main thread (stop all read and write operations), and this lag time is related to the current Redis memory usage. Typically, a gigabyte-grade Redis is used for a fork operation at a millisecond level. You can monitor the latest_fork_usec of the info output to see how long the last fork operation caused the lag.

Consistent configuration

Redis supports the use of Config set operations to implement run-real configuration modifications, which is convenient, but also causes a problem. The configuration that is dynamically modified by this command is not synced to your configuration file. So when you restart Redis for some reason, the configuration changes you make with Config set are lost, so we'd better make sure that the configuration files are changed along with the config file each time we modify the configuration with the set. To prevent human error, we'd better monitor the configuration, use the config get command to get the current runtime configuration, and compare it to the configuration values in redis.conf, and start the alarm if you find that neither side is up.

Slow Log

Redis provides slowlog instruction to get the most recent slow log, Redis slow log is directly in memory, so it's slow log cost is not big, in practice, we use crontab task Execute Slowlog command to get slow log, then save slow log to file, Use Kibana to generate real-time performance graphs for performance monitoring.

It is worth mentioning that the slow logging time of Redis includes only the time that Redis itself executes on a single command, excluding IO time, such as receiving client data and sending client data. In addition, the slow log of Redis and other database slow log is a little different, other databases occasionally 100ms slow log may be more normal, because the general database is multi-threaded concurrent execution, a thread execution of a command performance may not represent the overall performance, but the Redis, it is single-threaded, once the slow log, you may need to get attention immediately, it is better to find out what the specific reason.

Monitoring Services

-sentinel

Sentinel is a Redis-brought tool that monitors Redis master-slave replication and implements automatic failover after the main hang-out. In the process of transfer, it can also be configured to execute a user-defined script, in the script we can implement alarm notification and other functions.

-redis Live

Redis Live is a more general Redis monitoring scheme that implements the Monitor command on Redis periodically to get the commands currently being executed by Redis, and to generate a visual analysis report of the Web page through statistical analysis.

-redis Faina

Redis Faina is a redis monitoring service developed by the famous image sharing app Instagram, which works like Redis Live and is done with monitor.

Data distribution

It's hard to figure out the data storage distribution in Redis, such as you want to know what type of key value takes up the most memory. Here are some tools that can help you analyze Redis datasets.

-redis-sampler

Redis-sampler is a tool developed by REDIS authors that allows you to understand the approximate type, data, and distribution of data in the current Redis.

-redis-audit

Redis-audit is a script that allows us to know the amount of memory each type of key uses. It can provide data such as: the frequency of access to a certain type of key value, how many values set the expiration time, a certain type of key value to use the size of memory, it is convenient for us to find out which keys are not used or at all.

-redis-rdb-tools

Redis-rdb-tools is similar to the Redis-audit function, but it is obtained by analyzing the Rdb file for statistical data.

Redis Monitoring Tips Summary