Redis Data Summary (vii) persistence

Redis is an in-memory database that supports persistence, which means that Redis often needs to synchronize in-memory data to disk to ensure persistence. Redis supports two persistence modes, one is snapshotting (snapshot) is the default, and the other is the way Append-only file (abbreviated AOF). The following are respectively

Snapshotting

Snapshots are the default persistence mode. This is the way in which the in-memory data is written to the binary in a snapshot, the default file name is Dump.rdb. You can automatically make snapshot persistence by configuring settings. We can configure Redis to take snapshots automatically if more than M key is modified in n seconds, the following is the default snapshot save configuration

Save 1 #900秒内如果超过1个key被修改 to initiate a snapshot save
Save #300秒内容如超过10个key被修改, the snapshot save is initiated
Save 60 10000

The detailed snapshot save procedure is described below

1. Redis calls fork and now has child and parent processes.
2. The parent process continues to process the client request, which is responsible for writing the memory contents to the temporary file. Because the OS's write-time replication mechanism (copy on write) will share the same physical page, when the parent process processes the write request, the OS creates a copy of the page to be modified by the parent process, rather than writing the shared page. So the data in the child process's address space is a snapshot of the entire database at fork time.
3. When the child process finishes writing the snapshot to the temporary file, replace the original snapshot file with a temporary file, and the child process exits.

The client can also use the Save or Bgsave command to notify Redis to do a snapshot persistence. The save action is to save the snapshot in the main thread, which blocks all client requests because Redis uses a main thread to process all client requests. So it is not recommended. It is also important to note that each snapshot persistence is a full write of the memory data to disk once, not the incremental synchronization of only dirty data. If the amount of data is large, and more write operations, it will inevitably cause a large number of disk IO operations, may seriously affect performance.

In addition, because the snapshot is done at a certain interval, if Redis is accidentally down, all changes after the last snapshot will be lost. You can use AOF persistence if your application requires that you cannot lose any modifications. Described below

Append-only file

AOF is more persistent than snapshot mode, because Redis appends each received write command to a file by using the Write function (default is appendonly.aof) when aof persistence is used. When Redis restarts, the contents of the entire database are rebuilt in memory by re-executing the write commands saved in the file. Of course, because the OS caches write modifications in the kernel, it may not be written to disk immediately. The persistence of this aof method is also likely to lose some of the modifications. But we can tell Redis through the configuration file that we want to force the OS to write to disk through the Fsync function. There are three ways to do this (the default is: Fsync once per second)
AppendOnly Yes//enable AOF persistence mode
# Appendfsync always//write command is immediately forced to write to disk, the slowest, but guaranteed full persistence, not recommended
Appendfsync everysec//force write disk once per second, a good compromise in performance and persistence, recommended
# Appendfsync no//completely dependent on OS, best performance, no guarantee of persistence

The AOF approach also poses another problem. Persistent files can become more and more large. For example, we call the INCR Test command 100 times, the file must save all 100 commands, in fact, 99 are redundant. Because you want to restore the state of the database, it is enough to save a set test 100 in the file. In order to compress the aof persistence file. Redis provides the bgrewriteaof command. Receive this command Redis will use a snapshot-like method to save the in-memory data to a temporary file in the form of a command, and finally replace the original file.

The specific process is as follows
1. Redis Call Fork, now has a parent-child two processes
2. The child process writes to the temporary file the command to rebuild the database state based on the database snapshot in memory
3. The parent process continues to process the client request, in addition to writing the write command to the original aof file. Cache the received write commands at the same time. This will ensure that if the child process rewrite fails, it will not be problematic.
4. When a child process writes the snapshot content to a temporary file, the child process signals the parent process. The parent process then writes the cached write command to the temporary file as well.
5. Now the parent process can replace the old aof file with the temporary file and rename it, and the subsequent write commands are also started to append to the new aof file.
Note that the operation to rewrite the aof file does not read the old aof file, but instead overwrites the entire in-memory database content with a new aof file in the form of a command, which is a bit similar to a snapshot.

Redis Data Summary (vii) persistence