Redis notes-Database persistence strategy

2018-1-17 by Atlas

• Redis Persistence

Keep Redis's in-memory database state to disk to avoid accidental data loss.

• RDB Persistence

• It can be performed either manually or periodically, depending on the server configuration options.
• The generated RDB file is a compressed binary file that allows you to restore the state of the database when the Rdb file was generated.
• When the Rdb file is loaded, the server remains blocked until it is complete.

• Command Save, BGSAVE
  The Save command blocks the Redis server process until the Rdb file is created, and the server cannot process any command requests during server process blocking.
  The Bgsave command derives a child process and then the sub-process is responsible for creating the Rdb file, and the server process (the parent process) continues to process the command request.

• The aof file update frequency is usually higher than the RDB file update frequency.
  So-and
  If AoF is turned on, the server first restores the database using the AoF file;
  The server will use the Rdb file to restore the database only if AOF is turned off.

• Bgsave the state of the server when the command executes:
  (1) During bgsave execution, the Save command sent by the client is rejected by the server.
  (2) During bgsave execution, the Bgsave command sent by the client is rejected by the server.
  (3) Bgsave and bgrewriteaof cannot be executed simultaneously:
  If Bgsave is executing, the bgrewriteaof sent by the client will be deferred to Bgsave after execution;
  If bgrewriteaof is executing, the Bgsave sent by the client will be rejected by the server.

• Save saving condition

save  900  1                          服务器在900秒之内，对数据库进行了至少1次修改save  300  10                        服务器在300秒之内，对数据库进行了至少10次修改save  60    10000                  服务器在60秒之内，对数据库进行了至少10000次修改

struct redisServer {        // ...        // 记录保存条件的数组        struct saveparam *saveparams;        // 修改计数器        long long dirty;        // 上次执行保存的时间        time_t lastsave;        // ...}struct saveparam {        // 秒数        time_t seconds;        // 修改数        int changes;}

Servercron function checks the process of saving a condition

The RDB file structure is discussed together with the Java class file.

• AOF Persistence

• AOF Persistence records the state of a database by saving the write commands that are executed by the Redis server.

• The AoF file content is in plain text format.
  e.g.
  redis> SET msg "Hello"
  Ok
  AOF file contents (the Select command is automatically added by the server):
  2\r\n$6\r\nselect\r\n$1\r\n0\r\n
  3\r\n$3\r\n$3\r\nset\r\n$3\r\nmsg\r\n$5\r\nhello\r\n

• AOF Persistence Implementation steps:
  Command append-to-file-to-file synchronization
• Command append is the end of the aof_buf buffer that appends the executed write command to the server state:

struct redisServer {        // ...        // AOF缓冲区        sds aof_buf;        // ...}

• Writing and synchronizing files
  The Flushappendonlyfile () function considers whether the contents of the Aof_buf are to be written to and saved in the AoF file, and the behavior is determined by the value of the APPENDFSYNC option configured by the server.
• Appendfsync configuration:

Always:aof is the most durable, security-safe, failure to lose only one command data.
Everysec:aof persistent efficiency is fast enough, security means that the failure will only lose one second of command data.
No:aof the fastest persistence and security, the failure loses all the write command data after the last synchronization of the AoF file.

• AOF File loading process

• AOF file Rewrite to resolve aof file volume expansion problem.
• AOF File Rewriting principle:
  AOF file rewriting does not require reading, parsing, or writing to an existing aof file, but is implemented by reading the current database state of the server.
  The aof file rewrite first reads the current value of the key from the database, and then records the key-value pair with a single command instead of the multiple commands that previously recorded the key.
  If the number of elements exceeds the value of the Redis.h/redis_aof_rewrite_items_per_cmd constant, rewrite the value of multiple command record keys.
• AOF file Background Rewriting process (bgrewriteaof principle)

Reference: "Redis design and implementation"

Redis notes-Database persistence strategy