Quick mastery of redis--sixth: persistence (data backup and recovery)

1 What is persistent

Redis is an in-memory database that loses power and transfers data. Persistence is the conversion of memory data to hard disk data.
Of course, you can also drive to memory (the concept of backup, save, restore).

2 How to achieve

Two methods: snapshot Mode (RDB) + log mode (AOF)
Fast + Maximize Redis performance + convenience: RDB mode
More durable: aof mode
Recommendation: Use both of these methods reasonably.

2.1 Rdb Snapshot mode

The

• snapshotting (snapshot) syntax

  Snapshot is the default persistence mode (full-memory copy). This is the way in which the in-memory data is written to the binary in a snapshot, the default file name is Dump.rdb. You can automatically make snapshot persistence by configuring settings. We can configure Redis to take snapshots automatically if more than M key is modified in n seconds, here is the default snapshot save configuration. (Recommended from the bottom up, 60s-300s-900s)

    Save 900 1//900 seconds if more than 1 keys are modified, the snapshot is saved save 300 10//300 seconds content such as more than 10 keys are modified, Save 60 10000//(these 3 options are masked, the RDB is disabled) Stop-writes-on-bgsave-error Yes//background backup process error, the main process stops writing rdbcompression Yes/ If the Rdb file is compressed rdbchecksum Yes//import RBD recovery data, do not verify the integrity of the RDB Dbfilename Dump.rdb//the RDB filename dir.///rdb Placement Path  

• Detailed snapshot Save procedure

  1.redis calls Fork, and now has child and parent processes.

  2. The parent process continues to process client requests, and the child process is responsible for writing the memory contents to the temporary file. Because the OS's write-time replication mechanism (copy on write) will share the same physical page, when the parent process processes the write request, the OS creates a copy of the page to be modified by the parent process, rather than writing the shared page. So the data in the child process's address space is a snapshot of the entire database at fork time.

  3. When the child process finishes writing the snapshot to the temporary file, replace the original snapshot file with a temporary file, and the child process exits.

The client can also use the Save or Bgsave command to notify Redis to do a snapshot persistence. The save action is to save the snapshot in the main thread, which blocks all client requests because Redis uses a main thread to process all client requests. So it is not recommended. It is also important to note that each snapshot persistence is a complete write of the memory data to the disk once, not the incremental synchronous increase of data only. If the amount of data is large, and more write operations, it will inevitably cause a large number of disk IO operations, may seriously affect performance.

In addition, because the snapshot is done at a certain interval, if Redis is accidentally down, all changes after the last snapshot will be lost. You can use AOF persistence if your application requires that you cannot lose any modifications.

2.2 AOF Log Mode

AOF is more persistent than snapshot mode, because Redis appends each received write command to a file by using the Write function (default is appendonly.aof) when aof persistence is used. When Redis restarts, the contents of the entire database are rebuilt in memory by re-executing the write commands saved in the file. Of course, because the OS caches write modifications in the kernel, it may not be written to disk immediately. The persistence of this aof method is also likely to lose some of the modifications. But we can tell Redis through the configuration file that we want to force the OS to write to disk through the Fsync function. There are three ways to do this (the default is: Fsync once per second)

appendonly yes              //启用aof持久化方式# appendfsync always      //每收到写命令就立即强制写入磁盘，最慢的，但是保证完全的持久化，不推荐使用appendfsync everysec     //每秒钟强制写入磁盘一次，在性能和持久化方面做了很好的折中，推荐# appendfsync no         //完全依赖os，性能最好,持久化没保证（操作系统自身的同步）no-appendfsync-on-rewrite  yes  //正在导出rdb快照的过程中,要不要停止同步aofauto-aof-rewrite-percentage 100  //aof文件大小比起上次重写时的大小,增长率100%时,重写auto-aof-rewrite-min-size 64mb //aof文件,至少超过64M时,重写

Rewrite: There is n more data in memory, is the NNN time statement, in order to save the log file space, the memory data into directly optimized statements, not the previous (detailed) operation steps, but the "data" effect is the same.
The AOF approach also poses another problem. Persistent files can become more and more large. For example, we call the INCR Test command 100 times, the file must save all 100 commands, in fact, 99 are redundant. Because you want to restore the state of the database, it is enough to save a set test 100 in the file. In order to compress the aof persistence file. Redis provides the bgrewriteaof command. Receive this command Redis will use a snapshot-like method to save the in-memory data to a temporary file in the form of a command, and finally replace the original file. AOF process:

1.redis调用fork ，现在有父子两个进程2.子进程根据内存中的数据库快照，往临时文件中写入重建数据库状态的命令3.父进程继续处理client请求，除了把写命令写入到原来的aof文件中。同时把收到的写命令缓存起来。这样就能保证如果子进程重写失败的话并不会出问题。4.当子进程把快照内容写入已命令方式写到临时文件中后，子进程发信号通知父进程。然后父进程把缓存的写命令也写入到临时文件。5.现在父进程可以使用临时文件替换老的aof文件，并重命名，后面收到的写命令也开始往新的aof文件中追加。

Note that the operation to rewrite the aof file does not read the old aof file, but instead overwrites the entire in-memory database content with a new aof file in the form of a command, which is a bit similar to a snapshot.

Attention:

• AoF and RDB exist at the same time, limited use aof recover data;
• AoF and Rdb when recovering, who fast, RDB fast, copy data directly, AOF also execute the statement;
• It is recommended to use both AOF and RDB
• The aof file and the Rdb file are executed or loaded at the next Redis boot to achieve the purpose of the backup.

Quick mastery of redis--sixth: persistence (data backup and recovery)