A summary of the various persistence methods of Redis

Topic: Multiple Persistence modes for Redis:

Redis is an in-memory database that supports persistence, and Redis needs to frequently synchronize the in-memory data to disk to ensure persistence.

1. Rdb mode (snapshotting default snapshot mode):

1.1) Configuration:

Save 1 #在900秒 (15 minutes), dump memory Snapshot if at least 1 key changes occur. After save #在300秒 (5 minutes), dump memory Snapshot if at least 10 key changes occur. After save 10000 #在60秒 (1 minutes), dump memory Snapshot if at least 10,000 key changes occur.

1.2) Working principle:

1.2.1) Redis uses the fork function to copy a copy of the current process (the parent process) (the child process);

1.2.2) The parent process continues to receive and process commands from the client, and the child process begins to write the in-memory data to the temporary file on the hard disk;

1.2.3) When the child process has finished writing all the data, it replaces the old Rdb file with the temporary file, and this time the snapshot operation is complete.

1.3) View Dump.rdb:

127.0.0.1:7000> config get dir1) "dir" 2) "/usr/local/redis/db" 127.0.0.1:7000>

1.4) Advantages:

1.4.1) RDB is a very compact file that holds Redis data sets at a certain point in time, enabling us to backup and recover Redis databases and disaster recovery by regularly backing up the RDB files, or transferring them to other datacenters for storage.

1.4.2) Rdb maximizes the performance of Redis, only fork a subprocess when performing an RDB persistence, and persist with child processes, and the parent process does not need to process any disk I/O operations.

1.4.3) An RDB is faster than aof to recover a large data set and has much higher startup efficiency.

1.4.4) An RDB file is a binary format that is compressed (you can configure the Rdbcompression parameter to disable compression to conserve CPU), so the space occupied is less than the size of the in-memory data and is more conducive to transmission.

1.5) Disadvantages:

1.5.1) Each snapshot persistence is a full write of the memory data to disk once, not the incremental synchronous increment of data only. If the amount of data is large, and more write operations, it will inevitably cause a large number of disk IO operations, may seriously affect performance.

1.5.2) Snapshot mode is done at a certain interval, so if Redis accidentally down, it will lose all the changes after the last snapshot, there is some risk of data loss.

1.5.2) client's Save or Bgsave command notifies Redis that a snapshot persistence is not recommended.

127.0.0.1:7000> saveok127.0.0.1:7000>

Cause: The Save action saves the snapshot in the main thread, which blocks all client requests because Redis uses a main thread to process all client requests. So it is not recommended.

2, Append-only file (abbreviated AOF) way:

2.1) configuration

AppendOnly Yes #开启aofappendfilename "appendonly.aof" #名字 # Appendfsync Always # synchronization is performed every time the write is performed, the safest and slowest appendfsync everysec # One synchronization operation per second # Appendfsync no #不主动进行同步操作, but is entirely done by the operating system (that is, every 30 seconds), the fastest and least secure. #配置写入AOF文件后, a mechanism that requires the system to flush the hard disk cache Auto-aof-rewrite-percentage 100 # when the current aof file size exceeds the AoF file size of the last rewrite, it is rewritten again, if not previously rewritten, The aof file size at startup is based on the Auto-aof-rewrite-min-size 64MB # allowed to override the minimum aof file size

2.2) Working principle:

2.2.1) Redis Call Fork, now has a parent-child two processes

2.2.2) A child process writes a command to a temporary file to rebuild the state of a database based on a database snapshot in memory

2.2.3) The parent process continues to process the client request, in addition to writing the write command to the original aof file. Cache the received write commands at the same time. This will ensure that if the child process rewrite fails, it will not be problematic.

2.2.4.) When a child process writes the snapshot content to a temporary file, the child process signals the parent process. The parent process then writes the cached write command to the temporary file as well.

2.2.5) Now the parent process can replace the old aof file with the temporary file and rename it, and the subsequent write commands are also started to append to the new aof file.

2.3) View AoF

127.0.0.1:7000> config get dir1) "dir" 2) "/usr/local/redis/db" 127.0.0.1:7000>[[email protected] db]# ll total dosage 8- rw-r--r--1 root root 146 February 1 08:36 appendonly.aof-rw-r--r--1 root root 74 February 1 09:20 dump.rdb[[email protected] db ]#

2.4) Advantages:

2.4.1) This mechanism can lead to higher data security, i.e. data persistence.

2.4.2) Because this mechanism writes to the log file in append mode, it does not break the existing content in the log file even if there is an outage during the writing process. However, if we only write half of the data this time there is a system crash problem, do not worry, before the next boot of Redis, we can use the Redis-check-aof tool to help us solve the problem of data consistency.

2.4.3) If the log is too large, Redis can automatically enable the rewrite mechanism. That is, Redis continuously writes the modified data to the old disk file in Append mode, and Redis also creates a new file to record which modification commands are executed during this period. Therefore, the data security can be better ensured when the rewrite is switched.

2.4.4) aof contains a well-formed, easy-to-understand log file for recording all modification operations. In fact, it is possible to complete the reconstruction of the data through this file.

2.5: Disadvantages:

2.5.1) for the same number of datasets, the AoF file is usually larger than the Rdb file, and the persistence file becomes larger.

2.5.2) Depending on the synchronization strategy, the AOF is often slower than the RDB in terms of operational efficiency.

3. Other

Virtual memory mode and Diskstore mode. : (not recommended, and virtual memory is said to be deprecated after version 2.4, Diskstore is not used)

Related configuration

Persistent files can become more and more large. vm-enabled Yes #开启vm功能vm-swap-file/tmp/redis.swap #交换出来的value保存的文件路径/tmp/redis.swapvm-max-memory 1000000 #red is the maximum memory limit that is used, after which Redis begins to Exchange value to disk File Vm-page-size #每个页面的大小32个字节vm-pages 134217728 #最多使用在文件中使用多少页面, size of swap file = Vm-page-size * Vm-pagesvm-max-threads 4 #用于执行value对象换入换出的工作线程数量, 0 means no worker threads are used

Summarize:

1. Redis allows aof and RDB to be enabled at the same time, ensuring data security and making backup operations easy. After you restart Redis, Redis uses the aof file to recover the data, because the aof way of persisting may lose less data.

2, read and write separation through replication can achieve read and write separation to improve the load capacity of the server. In a common scenario, the frequency of reading is greater than the write, when a single-machine redis cannot cope with a large number of Read requests (especially the more resource-intensive requests, such as the sort command, etc.) can be established through the replication function from the database, the primary database only write operations, and from the database is responsible for read operations.

3. Persistent persistence from a database is often relatively time-consuming, in order to improve performance, you can create one (or several) from the database through replication, and enable persistence from the database while disabling persistence in the primary database. Restarting the base database when it crashes from the database automatically synchronizes the data so there is no need to worry about data loss. When the primary database crashes, you need to use the slaveof NO one command from the database to promote the service from the database to the primary database, and then use the slaveof command to set it to the new primary database from the database after the original primary database is started, and then synchronize the data back.

This article is from "Never give up!" Ningzhiyuan "blog, reprint please contact the author!

A summary of the various persistence methods of Redis