Verify Redis snapshot and AOF

Redis persistence introduction:

Redis is a memory database that supports persistence. That is to say, redis often needs to synchronize data in the memory to the disk to ensure persistence. Redis supports two persistence Methods: Snapshotting (snapshot) and Append-only file (aof.

The following describes

Snapshotting
Snapshots are the default persistence method. In this way, data in the memory is written to the binary file as a snapshot. The default file name is dump. rdb. You can configure and set Automatic snapshot persistence. We can configure redis to automatically create snapshots if more than m keys are modified in n seconds. The following is the default snapshot storage configuration.

Save 900 1 # if more than one key is modified within 900 seconds, the snapshot is saved.
Save 300 10 #300 seconds if more than 10 keys are modified, the snapshot is saved.
Save 60 10000

The following describes the snapshot retention process in detail.

1. redis calls fork and now has sub-process and parent process.

2. The parent process continues to process client requests. The child process is responsible for writing the memory content to the temporary file. Because the copy on write mechanism of the OS Parent and Child processes share the same physical page, when the parent process processes write requests, the OS creates a copy of the page to be modified by the parent process, instead of writing shared pages. Therefore, the data in the address space of the sub-process is a snapshot of the entire database at the fork moment.

3. After the sub-process writes the snapshot to the temporary file, it replaces the original snapshot file with the temporary file, and then the sub-process exits.

The client can also use the save or bgsave command to notify redis to perform snapshot persistence. The save operation saves snapshots in the main thread. Because redis uses a main thread to process all client requests, this method will block all client requests. Therefore, it is not recommended. Note that each snapshot persistence completely writes the memory data to the disk, not incremental synchronization of only dirty data. If the data volume is large and there are many write operations, a large number of disk I/O operations will inevitably occur, which may seriously affect the performance.

In addition, because the snapshot method is performed once at a certain interval, if redis accidentally goes down, all modifications after the last snapshot will be lost. If the application requires no loss of any modifications, you can use the aof persistence method. The following describes

Append-only file

Aof is more persistent than snapshot, because when aof persistence is used, redis will append every write command received to the file through the write function (appendonly by default. aof ). When redis is restarted, it re-executes the write commands saved in the file to recreate the entire database content in the memory. Of course, because the OS caches write modifications in the kernel, it may not be immediately written to the disk. In this way, the persistence of the aof method may also lose some modifications. However, we can use the configuration file to tell redis when we want to force the OS to be written to the disk through the fsync function. There are three methods as follows (default: fsync once per second)

Appendonly yes // enable the aof persistence Method
# Appendfsync always // immediately write data to the disk every time you receive the write command, which is the slowest, but ensures full persistence and is not recommended.
Appendfsync everysec // forcibly writes data to the disk once per second, which makes a good compromise between performance and persistence. We recommend that you
# Appendfsync no // fully dependent on OS, which has the best performance and is not guaranteed for persistence

The aof method also brings about another problem. Persistent files become larger and larger. For example, if we call the incr test command 100 times, all the 100 commands must be saved in the file. In fact, 99 of them are redundant. Because it is enough to restore the database status to save a set test 100 in the file. To Compress aof persistent files. Redis provides the bgrewriteaof command. After receiving this command, redis will save the data in memory to a temporary file in a similar way as a snapshot, and finally replace the original file. The procedure is as follows:

1. redis calls fork. Now there are two processes: parent and child.
2. The sub-process writes the command to the temporary file to rebuild the database status based on the database snapshot in the memory.
3. The parent process continues to process client requests, except for writing commands to the original aof file. Cache the received write commands. In this way, we can ensure that if the sub-process rewrite fails, there will be no problems.
4. When a sub-process writes the snapshot content to a temporary file by command, the sub-process sends a signal to notify the parent process. Then the parent process writes the cached write command to the temporary file.
5. Now the parent process can replace the old aof file with a temporary file and rename it. The subsequent write command also begins to append it to the new aof file.

You need to note that the aof file rewriting operation does not read the old aof file, but overwrites a new aof file in the database content in the entire memory using commands, this is similar to snapshot.

1 2 3 4 Next Page