How Redis memory data is persisted

[+] Overview

The power of Redis is largely due to the fact that all data is stored in memory, but when Redis restarts, all data stored in memory is lost and in many cases it is intolerable. So, we need to persist the in-memory data. The typical scenario for persisting data is as follows: Use Redis as a database, use Redis as a cache server, but cache miss can have a significant performance impact when all caches fail simultaneously, causing service avalanche and inability to respond.

This article describes two ways that Redis supports data persistence. Redis Data Persistence

Redis supports two methods of data persistence: Rdb and AoF. The former will persist the in-memory data to the hard disk according to the configured rules, while the latter will log the command after each execution of the write command. The two persistence modes can be used alone, but they are usually used in combination. RDB Mode

The persistence of the RDB mode is done by means of a snapshot. When a certain rule is met, a copy of the in-memory data is stored to the hard disk, which is called "snapshot", and Redis takes a snapshot of the data in the following cases: Automatic snapshot According to configuration rules, user performs save, Bgsave command, executes Flushall command; When replication (replication) is performed. scenes that perform snapshots automatic snapshots based on configuration

Redis allows users to customize snapshot conditions and automatically perform snapshots when conditions are met, and the snapshot rules are configured as follows:

1 2 3

Save 1 Save 60 10000

Each snapshot condition is exclusive to one row, between them is or (| | ) relationship, take a snapshot as long as any one is satisfied. The first parameter T after the save is the time, the unit is seconds, the second parameter m is the number of keys changed, meaning: when the number of keys changed within the time t is greater than M, the snapshot is taken automatically. For example, save 900 1 means that the number of keys changed within 15 minutes (900s) is greater than 1 o'clock, and the snapshot operation is performed automatically. Execute Save or bgsave command

In addition to having Redis take snapshots automatically, when we need to restart, migrate, and back up Redis, we can also manually execute the Save or Bgsave command to take the snapshot. Save command: When you execute the Save command, Redis synchronizes the snapshot operation, which blocks all requests from the client, so you should avoid using the command when you put more database data; bgsave command: from the command name can be seen, The difference between this command and the Save command is that the snapshot operation of the command is performed asynchronously in the background, while the snapshot operation can also handle requests from the client. After executing the bgsave command, Redis returns OK immediately to start the snapshot operation, and if you want to know if the snapshot operation is complete, you can use the Lastsave command to return the last time the snapshot was successfully executed, and the result is a UNIX timestamp. Execute flushall Command

When you execute the flushall command, Redis clears all data from the database. Note that regardless of whether the process of emptying the database triggers an automatic snapshot condition, Redis performs a snapshot operation as long as the automatic snapshot condition is not NULL, and the Execute Flushall command does not take a snapshot operation when no automatic snapshot condition is defined. Perform replication

When master-slave mode is set, Redis initiates an automatic snapshot of the replication initialization. Snapshot principle

Redis defaults to storing the snapshot files in the Dump.rdb file in the working directory of the Redis current process, and can specify the storage path and file name of the snapshot file by using the Dir and dbfilename two parameters in the configuration file, for example:

1 2	Dbfilename Dump.rdb Dir/opt/soft/redis-3.0.4/cache

The process of snapshot execution is as follows: Redis uses the fork function to copy a copy of the current process (the parent process), the parent process continues to process requests from the client, and the child process begins to write the in-memory data to a temporary file on the hard disk, and when the child process finishes writing all the data, Replace the old Rdb file with this temporary file, at which time a snapshot operation is completed.

It is important to note that:

When the Fork is executed the operating system (Unix-like operating system) uses a write-time copy (Copy-on-write) policy, that is, the moment the fork function occurs, the parent process and the child process share the same block of memory data, when the parent process needs to modify one of the pieces of data, such as executing a write command. The operating system copies the piece of data to ensure that the child process is not affected, so the Rdb file stores the memory data at the moment the fork operation is performed. So the RDB approach is theoretically a case of data loss (those modified after fork are not written into the Rdb file).

Through the above introduction, you can know that the snapshot does not modify the Rdb file, and only when the completion of the temporary file replacement of the old Rdb file, so that any time the Rdb file is complete. This allows us to make backups of Redis data by regularly backing up the RDB files. An RDB file is a compressed binary file, so the space used is less than the size of the in-memory data and is more advantageous for transmission.

When Redis starts, it automatically reads the RDB snapshot files, loads the data from the hard disk into memory, and depending on the number, the process lasts a different amount of time, typically a record of 10 million string type keys, and a snapshot file size of 1GB takes 20-30 seconds to load into memory. Example

The following is a demonstration of the RDB approach to persistence, with the following snapshot rules configured first:

1 2 3) 4 5

Save 1 Save ten 10000 dbfilename dump.rdb Dir/opt/soft/redis-3.0.4/cache

Configuration file/opt/soft/redis-3.0.4/conf/redis.conf to start the Redis service:

Then set a key value via the client:

1 2 3 4 5 6

[qifuguang@mac~]$/opt/soft/redis-3.0.4/src/redis-cli-p 6379 127.0.0.1:6379> set Test-rdb HelloWorld OK 127.0.0.1:6379> get Test-rdb "HelloWorld" 127.0.0.1:6379>

Now forcibly kill the Redis service:

Now to the/opt/soft/redis-3.0.4/cache directory, the Redis snapshot file Dump.rdb appears in the directory:

1 2	[qifuguang@mac/opt/soft/redis-3.0.4/cache]$ ls dump.rdb

Now restart Redis:

Then use the client connection to check if the previously set key is still present:

1 2 3 4

[qifuguang@mac~]$/opt/soft/redis-3.0.4/src/redis-cli-p 6379 127.0.0.1:6379> get Test-rdb "HelloWorld" 127.0.0.1:6379>

It can be found that the previously set key was recovered by the snapshot file Dump.rdb after the Redis reboot. AoF Way

When using Redis to store non-temporal data, it is generally necessary to turn on aof persistence to reduce data loss due to process termination, AOF can append each write command that Redis executes to the hard disk file, which obviously reduces the performance of Redis, but in most cases this effect is acceptable , in addition, the use of a faster hard drive can improve the performance of aof. Open AoF

By default, Redis does not have the AOF (append only file) persistence feature enabled by configuring it in the configuration file as follows:

1	AppendOnly Yes

When on, Redis writes the command to the AoF file on the hard disk each time a write command is executed. AOF file save path and Rdb file path are consistent, all through the dir parameter configuration, the default file name is: appendonly.aof, can be configured by the Appendonlyfilename parameter modification, for example:

1	Appendonlyfilename appendonly.aof

implementation of AOF persistence

AoF writes the write commands that Redis executes, such as executing the following command in the case of opening aof persistence, in the form of plain text:

1 2 3 4 5 6 7 8 9

[qifuguang@mac/opt/soft/redis-3.0.4]$./SRC/REDIS-CLI 127.0.0.1:6379> 127.0.0.1:6379> 127.0.0.1:6379> 127.0.0.1:6379> set Aof1 value1 OK 127.0.0.1: