The RDB and aof of Redis

1. Data snapshot Rdb

1.1 Principle

(1) An RDB is a way to persist data from a moment to disk, which is a snapshot.

(2) Redis in the process of data persistence, the data will be written to a temporary file, until the persistence process is over, the temporary file will be used to replace the last persisted file. It is this feature that allows us to make backups at any time, and Redis is in the running state;

(3) for RDB mode, Redis creates (fork) a sub-process for persistence, and the main process does not do any IO operations, thus ensuring very high performance for Redis.

(4) If large-scale data recovery is required and is not very sensitive to the integrity of data recovery, the RDB approach is more efficient than the AOF approach.

If you are sensitive to the integrity of your data, then the RDB approach is not for you, because even if you persist every 5 minutes, there will still be nearly 5 minutes of data loss when Redis fails. So, Redis also offers another way to persist, which is aof.

1.2 Several ways to generate snapshots

Manual trigger:

(1) The Save command is used to create a backup of the current database, which will create a Dump.rdb file under the Redis installation directory dir.

If you need to recover data, simply move the backup file Dump.rdb to the Redis installation directory and start the service. Get the Redis directory using the config get dir command

(2) Bgsave is executed in the background;

(3) Shutdown save, when the service is closed, shutdown has two options, nosave|save, if not added, the default is save;

Auto Trigger:

(4) Settings in the configuration file redis.conf:

Save 900 1

Save 300 10

Save 60 10000

Dbfilename Dump.rdb

1.3 Using an RDB file for restore testing

Note: You need to turn off the AOF function when restoring, otherwise redis will load appendonly.aof This log file when it is started, so it is not dump.rdb content, but the aof log of the application.

#使用redis-benchmark load the test data and close aof:

Src/redis-benchmark-h 127.0.0.1-p 6379-n 200000-c 20-d 4-k 1--csv > redis_benchmart_$ (Date +%Y%m%d). Log 2>&A Mp;1

[Email protected] redis]# SRC/REDIS-CLI

127.0.0.1:6379> Config get dir

1) "Dir"

2) "/usr/local/redis"

127.0.0.1:6379> Config Get appendonly

1) "AppendOnly"

2) "No"

127.0.0.1:6379> keys *

1) "Key1"

2) "Key2"

3) "key:__rand_int__"

4) "MyList"

5) "counter:__rand_int__"

127.0.0.1:6379> Shutdown Save

[Email protected] redis]# MV Dump.rdb Dump.rdb.bak

[Email protected] redis]# Src/redis-server redis.conf

[Email protected] redis]# SRC/REDIS-CLI

127.0.0.1:6379> keys *

(empty list or set)

127.0.0.1:6379> shutdown Nosave

Place the backup RDB file in the specified location and restart Redis so that the data is restored

[Email protected] redis]# MV Dump.rdb.bak Dump.rdb

[Email protected] redis]# Src/redis-server redis.conf

[Email protected] redis]# SRC/REDIS-CLI

127.0.0.1:6379> keys *

1) "key:__rand_int__"

2) "MyList"

3) "Key2"

4) "Key1"

5) "counter:__rand_int__"

2.AOF (append only file)

2.1 AOF

(1) Only allow Append, not allowed to change the file

Open method: AppendOnly Yes

The AoF method is to record the executed write instruction and execute the instruction again in the order of the data recovery.

In the case of the same data set, the AoF file is larger than the size of the Rdb file. Also, the AOF mode is slower than the Rdb method.

We can turn on the aof feature by configuring AppendOnly Yes in redis.conf. If there is a write operation (such as set, etc.), Redis is appended to the end of the aof file.

The default AOF persistence policy is fsync once per second (Fsync refers to logging the write instruction in the cache to disk), because in this case, Redis can still maintain good processing performance, even if Redis fails, it will only lose the last 1 seconds of data.

(2) if the log is appended with a full, inode full, or power outage, which results in incomplete log writes, Redis provides a redis-check-aof tool that can be used for log repair:

• make A backup copy of your AOF file.

• fix the original file using the Redis-check-aof tool that ships with redis:  $ redis-check-aof--fix appendonly.aof

• optionally use Diff-u to check which is the difference between the files.

• restart the server with The fixed file.

(3) Restore test via appendonly.aof file

127.0.0.1:6379> Config Get appendonly

1) "AppendOnly"

2) "Yes"

127.0.0.1:6379> mset key1 1 key2 2 Key3 3

Ok

127.0.0.1:6379> keys *

1) "Key3"

2) "Key1"

3) "Key2"

[email protected] redis]# CP appendonly.aof Appendonly.aof.bak

127.0.0.1:6379> Flushall

Ok

127.0.0.1:6379> keys *

(empty list or set)

127.0.0.1:6379> shutdown

[Email protected] redis]# RM-RF appendonly.aof

[Email protected] redis]# MV Appendonly.aof.bak appendonly.aof

[Email protected] redis]# Src/redis-server redis.conf

[Email protected] redis]# SRC/REDIS-CLI

127.0.0.1:6379> keys *

1) "Key1"

2) "Key2"

3) "Key3"

2.2 aof File Rewrite

(1) Rewrite principle

Because of the Append method, if you do not do any processing, the AoF file will become more and more large, for this reason, Redis provides the aof file rewriting (rewrite) mechanism, that is, when the aof file size exceeds the threshold value, Redis will start the AoF file content compression, Keep only the smallest set of instructions that can recover data. If we call 100 times the incr instruction, 100 instructions are stored in the AoF file, but this is obviously very inefficient, it is perfectly possible to combine the 100 instructions into a set instruction, which is the principle of the rewriting mechanism.

In the AoF rewrite, is still the use of the first write temporary files, complete and then replace the process, so power outages, disk full and other issues will not affect the availability of aof files.

Another benefit of the AOF approach is that we illustrate it through a "scene rendition". A classmate in the operation of Redis, accidentally executed Flushall, resulting in redis memory data are all emptied, as long as Redis configured aof persistence mode, and AoF file has not been rewritten (rewrite), We can pause Redis and edit the AoF file as quickly as possible, delete the last line of the Flushall command, and then restart Redis to restore all of the Redis data to the previous state of the Flushall. However, if the AoF file has been rewritten, it is not possible to recover the data in this way.

(2) method of triggering rewrite

The first method: manually triggered using the bgrewriteaof command;

The second method: controlled by the configuration file

Auto-aof-rewrite-percentage 100

Auto-aof-rewrite-min-size 64MB

For example, the above parameter setting meaning: When the appendonly.aof is less than 64M, will not trigger rewrite, when the file 64M, the growth rate of 100%, that is 128M, triggered once rewrite, This time Redis remembers the size of the file after rewrite, if it is 80M, it will only trigger the next time after the file has risen again to 160M, and so on.

3 Summary:

It is recommended that both of these backup strategies be opened at the same time to ensure more secure data;

If your business can accept the loss of certain data, pay more attention to performance, you can only open the RDB;

If you only use Redis as a cache, you do not need to open rdb and aof;

Reference links

Https://redis.io/topics/persistence

The RDB and aof of Redis